Within the late 1800s, scientists realized that migratory birds made species-specific nocturnal flight calls—“acoustic fingerprints.” When microphones became commercially available within the Fifties, scientists began recording birds at night. Farnsworth led a few of this acoustic ecology research within the Nineties. But even then it was difficult to identify the short calls, a few of that are at the sting of the frequency range humans can hear. Scientists ended up with 1000’s of tapes they’d to scour in real time while spectrograms that visualize audio. Though digital technology made recording easier, the “perpetual problem,” Farnsworth says, “was that it became increasingly easy to gather an unlimited amount of audio data, but increasingly difficult to research even a few of it.”
Then Farnsworth met Juan Pablo Bello, director of NYU’s Music and Audio Research Lab. Fresh off a project using machine learning to discover sources of urban noise pollution in Recent York City, Bello agreed to tackle the issue of nocturnal flight calls. He put together a team including the French machine-listening expert Vincent Lostanlen, and in 2015, the BirdVox project was born to automate the method. “Everyone was like, ‘Eventually, when this nut is cracked, that is going to be a super-rich source of knowledge,’” Farnsworth says. But to start with, Lostanlen recalls, “there was not even a touch that this was doable.” It seemed unimaginable that machine learning could approach the listening abilities of experts like Farnsworth.
“Andrew is our hero,” says Bello. “The entire thing that we wish to mimic with computers is Andrew.”
They began by training BirdVoxDetect, a neural network, to disregard faults like low buzzes brought on by rainwater damage to microphones. Then they trained the system to detect flight calls, which differ between (and even inside) species and might easily be confused with the chirp of a automobile alarm or a spring peeper. The challenge, Lostanlen says, was much like the one a wise speaker faces when listening for its unique “wake word,” except on this case the gap from the goal noise to the microphone is way greater (which implies rather more background noise to compensate for). And, in fact, the scientists couldn’t select a novel sound like “Alexa” or “Hey Google” for his or her trigger. “For birds, we don’t really make that selection. Charles Darwin made that selection for us,” he jokes. Luckily, they’d plenty of training data to work with—Farnsworth’s team had hand-annotated 1000’s of hours of recordings collected by the microphones in Ithaca.
With BirdVoxDetect trained to detect flight calls, one other difficult task lay ahead: teaching it to categorise the detected calls by species, which few expert birders can do by ear. To cope with uncertainty, and since there isn’t training data for each species, they selected a hierarchical system. For instance, for a given call, BirdVoxDetect might have the option to discover the bird’s order and family, even when it’s undecided concerning the species—just as a birder might at the very least discover a call as that of a warbler, whether yellow-rumped or chestnut-sided. In training, the neural network was penalized less when it mixed up birds that were closer on the taxonomical tree.
