Everyone knows sound is Submission Season 1 Episode 1 (2016)a critical component to most films and videos. After all, even when films were silent, there was still a musical accompanist letting the audience know how to feel.
This natural law remains the same for the new crop of generative AI videos, which emerge eerily silent. That's part of why Google has been working on "video-to-audio" technology (V2A) which "makes synchronized audiovisual generation possible." On Monday, Google's AI lab, DeepMind, shared progress on generating such audio including soundtracks and dialogue that automatically match up with AI-generated videos.
Google has been hard at work developing multimodal generative AI technology to compete with rivals. OpenAI has its AI video generator Sora (yet to be publicly released) and GPT-4o, which creates AI voice responses. Companies like Meta and Suno have been exploring AI-generated audio and music, but pairing audio with video is relatively new. ElevenLabs has a similar tool that matches audio to text prompts, but DeepMind says V2A is different because it doesn't require text prompts.
V2A can be paired with AI video tools like Google Veo or existing archival footage and silent films. This can be used for soundtracks, sound effects, and even dialogue. It works by using a diffusion model trained with visual inputs, natural language prompts, and video annotations to gradually refine random noise into audio that fits the tone and context of videos.
Google DeepMind says V2A can "understand raw pixels" therefore you don't actually need a text prompt to generate the audio, but it does help with the accuracy. The model can also be prompted to make the tone of the audio sound positive or negative. Along with the announcement, DeepMind released some demo videos, including a video of a dark, creepy hallway accompanied by horror music, a lone cowboy at sunset scored to a mellow harmonica tune, and an animated figure talking about its dinner.
V2A will include Google's SynthID watermarking as a safeguarding measure against misuse, and Deepmind's blog post says the feature is currently undergoing testing before it's released to the public.
Topics Artificial Intelligence Google
Previous：The 10 Most Anticipated PC Games of 2017
Next：How This Long
Best travel deal: Take 30% off Southwest flights
'Final Fantasy XVI' review: Being different is fine
Newest luxury sex toy is a real innovation in suction stimulation
Charlotte Brontë Poem at Auction, and Other News by Sadie Stein
New MIT report reveals energy costs of AI tools like ChatGPT
Honors Galore, and Other News by Sadie Stein
Lady Liberty by Sadie Stein
Apple ID supports passkeys on iOS 17, iPad OS 17, and macOS Sonoma. Here's how to test it out.
Barcelona Open 2025 livestream: Watch live tennis for free
In the Margins by Sadie Stein
Best robot vacuum deal: Save $140 on roborock Q7 Max Robot Vacuum
The Bookstore of the Year, and Other News by Sadie Stein
What We’re Loving: Aliens and Birds by The Paris Review
The 20 best TV shows streaming on Max
10 Tech Predictions for 2017
How to watch 'Infinity Pool' with the best streaming deals
Painting More Animals on Rocks by Sadie Stein
Rumors of the Death of the Book Greatly Exaggerated, and Other News by Sadie Stein
Waymo data shows humans are terrible drivers compared to AI
Apple ID supports passkeys on iOS 17, iPad OS 17, and macOS Sonoma. Here's how to test it out.
Apple reportedly swapping in new iPhone XR colors: green and lavenderCleganebowl was a bright spot in a dismal 'Game of Thrones' seasonBeastie Boys singer designs 'vegan' sneaker for Planned ParenthoodEverything you need to know about Global Accessibility Awareness Day'Game of Thrones' featured a cameo from NFL superstar Aaron RodgersFacebook warns advertisers: You might not like 'clear history'35 things to inspire you during your 2019 selfOur lack of sleep is costing the world billions of dollarsUber tests out no8th grader's Spotify playlist gets him in big trouble with momPsychologists issue powerful new guidelines for treating girls, womenSalvador Dalí deepfake debuts at Florida museum in 'Dalí Lives'Trump might sign an order that effectively bans Huawei in the U.S.Bill Nye uses profanity to stress the enormity of climate changeFirst look at Pokémon's new mobile game, quietly released on AndroidOnePlus 7 Pro has a 6.7'Game of Thrones' star Pilou Asbæk responds to Season 8 criticismEverything you need to know about Global Accessibility Awareness DayJeremy Clarkson launches social media platform for car addictsApple reportedly swapping in new iPhone XR colors: green and lavender Eight Public Cases An Interview with Kerri Pierce Why Write Fiction in 2017? Puerto Rico Sketchbook: The Anarchist Bikers Who Came to Help The Rise of Queer Comics The Paris Review Staff's Favorite Books of 2017 Reimagining Female Identity in a Ukrainian Orphanage The Uncertain Future of the American Mall Emoji Poetry Contest The Schizophrenic Sentence by Jeff Dolven Death’s Footsteps Degas’s Model Tells All Cooking with Sybille Bedford by Valerie Stivers Why an Unemployed Actor Flew Across the Country to Stalk Salinger Listen: Hemingway's Unrequited High School Crush Playing for Ralph Ellison's Little Man at Chehaw Station The Dignified Bot Opera in a Post A Visit to the Musée d’Edith Piaf Staff Picks: Nerds, Necromancers and New Wave Poetry
2.4796s , 10109.703125 kb
Copyright © 2025 Powered by 【Submission Season 1 Episode 1 (2016)】,Evergreen Information Network