DeepMind’s Latest AI Generates Soundtracks, Dialogue For Videos

DeepMind introduces V2A, an AI sound director for silent videos. V2A, short for “video-to-audio,” tackles the limitation of muted outputs in current video generation models. . This technology can create music, sound effects, and dialogue synchronized with video content, enhancing the realism of AI-generated media. Trained on a meticulously curated dataset combining audio, dialogue scripts, and video clips, V2A learns to sonically mirror on-screen motion.

While V2A shows promise, it shouldn’t be yet perfect; its audio generation quality can suffer in videos with artifacts or distortions. As a result of potential misuse, DeepMind has decided against immediate public release, choosing further safety assessments and gathering feedback from creators and filmmakers. V2A goals to be a useful tool for archivists and people working with historical footage, while raising essential considerations in regards to the impact of generative AI on the film and TV industry. Read more.

Chinese manufacturer Starpery Technology are injecting AI into the “oldest career,” aiming for interactive dolls that converse and, well, connect. These next-generation sexbots, set to debut soon, promise enhanced user experiences by specializing in emotional connections through sophisticated AI models. Starpery’s approach involves training LLMs much like ChatGPT, enabling these dolls to reply intelligently and perform tasks beyond basic dialogue.

Despite technical challenges in achieving lifelike human interaction, similar to realistic movement and speech, Starpery stays committed to innovation. Furthermore, they plan to expand into robotics for household chores and care by 2025, leveraging AI advancements to deal with societal needs. Nevertheless, the industry faces ethical concerns regarding consent, gender stereotypes, and privacy issues on account of the sensitive nature of AI-driven intimate companions. Read more.

Meta’s FAIR team has introduced a series of recent AI models and tools focused on audio synthesis, text-image interactions, and watermarking. Leading the lineup is JASCO, an revolutionary AI that refines musical elements like chords and beats using textual inputs, allowing users to regulate generated sounds through easy text commands. FAIR plans to release JASCO’s inference code and pre-trained models under open-source licenses to encourage collaboration in auditory AI development.

Moreover, Meta has launched AudioSeal, a tool for applying digital watermarks to AI-generated speech, enhancing the power to detect synthetic content inside real-world audio recordings. This advancement is aimed toward improving detection speeds for AI authenticity verification.

Meta’s Chameleon model, designed for tasks like image captioning using text, is currently accessible just for text-related functionalities. These releases underscore Meta’s commitment to advancing AI capabilities while promoting transparency and innovation in the sphere. Read more.