The technology that makes it possible, called semantic hearing, could pave the way in which for smarter hearing aids and earphones, allowing the wearer to filter out some sounds while boosting others. 

The system, which continues to be in prototype, works by connecting off-the-shelf noise-canceling headphones to a smartphone app. The microphones embedded in these headphones, that are used to cancel out noise, are repurposed to also detect the sounds on this planet across the wearer. These sounds are then played back to a neural network, which is running on the smartphone; then certain sounds are boosted or suppressed in real time, depending on the user’s preferences. It was developed by researchers from the University of Washington, who presented the research on the ACM Symposium on User Interface Software and Technology (UIST) last week.

The team trained the network on 1000’s of audio samples from online data sets and sounds collected from various noisy environments. Then they taught it to acknowledge 20 on a regular basis sounds, akin to a thunderstorm, a bathroom flushing, or glass breaking.

It was tested on nine participants, who wandered around offices, parks, and streets. The researchers found that their system performed well at muffling and boosting sounds, even in situations it hadn’t been trained for. Nonetheless, it struggled barely at separating human speech from background music, especially rap music.

Mimicking human ability

Researchers have long tried to resolve the “cocktail party problem”—that’s, to get a pc to give attention to a single voice in a crowded room, as humans are capable of do. This recent method represents a major step forward and demonstrates the technology’s potential, says Marc Delcroix, a senior research scientist at NTT Communication Science Laboratories, Kyoto, who studies speech enhancement and recognition and was not involved within the project.