“Meta has done an awesome job having a breadth of various things they support, like text-to-speech, speech-to-text, even automatic speech recognition,” says Chetan Jaiswal, a professor of computer science at Quinnipiac University, who was not involved within the research. “The mere variety of languages they’re supporting is an incredible achievement.”
Human translators are still an important a part of the interpretation process, the researchers say within the paper, because they will grapple with diverse cultural contexts and ensure that the identical meaning is conveyed from one language into one other. This step is vital, says Lynne Bowker of the University of Ottawa’s School of Translation & Interpretation, who didn’t work on Seamless. “Languages are a mirrored image of cultures, and cultures have their very own ways of knowing things,” she says.
Relating to applications like medicine or law, machine translations must be thoroughly checked by a human, she says. If not, misunderstandings may result. For instance, when Google Translate was used to translate public health information concerning the covid-19 vaccine from the Virginia Department of Health in January 2021, it translated “not mandatory” in English into “not obligatory” in Spanish, changing the entire meaning of the message.
AI models have way more examples to coach on in some languages than others. This implies current speech-to-speech models may have the ability to translate a language like Greek into English, where there could also be many examples, but cannot translate from Swahili to Greek. The team behind Seamless aimed to resolve this problem by pre-training the model on hundreds of thousands of hours of spoken audio in numerous languages. This pre-training allowed it to acknowledge general patterns in language, making it easier to process less widely spoken languages since it already had some baseline for what spoken language is alleged to sound like.
The system is open-source, which the researchers hope will encourage others to construct upon its current capabilities. But some are skeptical of how useful it might be compared with available alternatives. “Google’s translation model isn’t as open-source as Seamless, however it’s far more responsive and fast, and it doesn’t cost anything as an instructional,” says Jaiswal.
Essentially the most exciting thing about Meta’s system is that it points to the potential of quick interpretation across languages within the not-too-distant future—just like the Babel fish in Douglas Adams’ cult novel . SeamlessM4T is quicker than existing models but still not quick. That said, Meta claims to have a more moderen version of Seamless that’s as fast as human interpreters.
“While having this type of delayed translation is okay and useful, I believe simultaneous translation shall be much more useful,” says Kenny Zhu, director of the Arlington Computational Linguistics Lab on the University of Texas at Arlington, who isn’t affiliated with the brand new research.
