Smart devices are profoundly interactive, with other devices and people, but do they possess ‘environmental awareness’? Not yet. Context-based awareness and computing is still an unexplored realm in smart device technology. But things may soon change, if recent works by a team of researchers at the Carnegie Mellon University are to be believed. Smart devices may pick up specific sound vibrations and map them to human activity; also, they can identify a device and might confirm whether it is on or off. These functionalities have been made possible based on technologies that use acoustic and sensory input, which are identified and then put into contexts, all with an amazingly high accuracy.
Sound-Enabled Activity Recognition System use Pre-existing Microphones in Devices
The findings of the two separate research papers are expected to be presented at the ACM Symposium on User Interface Software and Technology (UIST) in Berlin by the end of October 17, 2018. In the first paper, they discussed the idea of a type of sound-enabled activity recognition system that they call ‘Ubicoustics’ which can identify sounds related to specific locations. Here they utilize the concept of professional sound-effect libraries and use microphones already present in smart devices, such as smartphones, to achieve the end. In the other paper, their work demonstrated how these vibrations can be contextualized in a spatial environment.
The activity recognition system can be integrated to an already existing smart device as a software update. The technology works as a plug-and-play system that use deep-learning models to train itself. Fit for any environment, the system can identify a variety of sounds such as those related to cooking such as blending or chopping and map them to vibrations, which will cause it to function in certain way like giving details of a recipe.
Sounds contextualized to Specific Surfaces using Laser Vibrometry Technology
The other technique that they hope to demonstrate is the idea of detecting vibrations based on laser vibrometry that uses a specialized sensor, and localize them to specific surfaces. Unlike the first technology, the technique doesn’t comprise on privacy and does away with the limitation of vibration overlap caused by picking of sounds from different activities. The sensor in the system can identify the type of device, its movement, and whether it is on or off, with more than 90% accuracy.
