What is native audio in AI video?

A model with native audio is trained on clips with their soundtracks, so it learns that a door shuts with a bang, a crowded street hums and lips move with the words they say. When it generates, it makes the frames and the sound together, both steered by the same prompt and laid on the same timeline, so each sound starts on the frame that causes it.

The older way, still common, makes a silent clip first and gives it sound afterward: a voice from a text-to-speech model, effects from a sound model that watches the clip, music chosen in an editor. Each step is separate, so each has to be lined up with the picture by hand or by another model. Google describes Veo 3.1 as generating all of its audio natively, and ByteDance calls Seedance 2.5 an audio-video joint generation model.

Under the hood: two streams that listen to each other

Published joint models run two generators side by side, one for the picture and one for the sound, and let them exchange information at every step. MM-Diffusion (2022) was an early design: two subnetworks that turn noise into a matching clip and soundtrack together. Ovi (2025) joins two transformer towers block by block, sharing timing and meaning between them, which its authors say gives synchronization without a separate alignment step.

The LTX-2 paper (2026), from Lightricks, the makers of LTX-2.3, sets out the same idea at scale: a 14-billion-parameter video stream and a 5-billion-parameter audio stream, joined by attention in both directions, with speech, background sound and foley, and published weights. The other route, sound made after the picture, is still studied too: MMAudio takes a finished video, with optional text, and generates sound aligned to it frame by frame.

A clip’s soundtrack is built from layers, and a prompt works best when it names each layer it wants. Google’s guidance for Veo 3.1 is to say clearly whether you want audio, and describe it in sentences of its own; its prompting guide labels effects “SFX:” and the background “Ambient noise:”.

Ambience and effects are what video models do most reliably: a place’s sound is long and forgiving, and an effect only has to land on one frame. Speech is the hardest layer, because every syllable has to match a mouth.

Native audio is one of several ways to give a video sound:

The difference from lip sync is the order. Lip sync starts from a voice and changes the picture to match it; native audio makes the voice and the moving mouth at the same time, from the words of the prompt, so neither is fitted to the other.

Dialogue is written into the prompt, not recorded. This 30-second take was made on DreamBench with Seedance 2.5 with sound on; the brief is in English and the chef’s one line is in Japanese, quoted in Japanese script with its meaning beside it.

• Put the line in quotes, straight after the person who says it, so the voice belongs to someone on screen.
• Say how it is said: quietly, out of breath, laughing. The delivery is part of the line.
• Name the language when it isn’t the language of the brief, and write the words in that language.
• Keep it short: one or two short sentences fit an 8-second clip; a speech does not.
• One speaker at a time. Two people talking over each other is where voices get mixed up and mouths go wrong.

Google’s model reference lists English as the prompt language for Veo 3.1. A line in another language is worth testing on the model before you rely on it.

Sound without words still needs directing. This clip on Veo 3.1 asks for one sound that changes once, on cue: the brief ends “Muffled underwater ambience, then one clear breath”, and the breath comes as the turtle breaks the surface.

• Tie each effect to an action in the same sentence: “the door slams as she leaves” lands better than a list of sounds at the end.
• Describe the ambience as the place’s own material: wind over a rooftop, a café’s murmur, traffic two streets away.
• Say how the sound ends: it stops on a cut, carries on, or fades.

• Lip sync drift: the mouth and the words start together, then slide apart over a long line, or the mouth moves when no one is speaking.
• Garbled speech: words come out slurred, invented or in the wrong order, more often in long or fast lines. Google says it is still working to remove instances of incoherent speech from Veo, and its guide calls the audio features experimental and actively improving.
• The wrong speaker: with two or three people in frame, a line can come from the wrong mouth, or an extra voice can appear.
• Voices that change between clips: each clip is made separately, so a character described differently in two prompts can sound like two people; see character consistency.
• Music that stops at every cut: a score made inside one clip ends where the clip ends, and the next clip starts a different one.
• Music you can’t vouch for: nothing in a generated clip tells you whether its melody resembles a recording that exists. For anything you publish commercially, use a track whose license you know.
• Loudness that jumps: each clip comes out at the level the model chose, so a cut between two clips can be suddenly louder or quieter. Level the sound in an editor before you publish.

Voices and consent

A generated voice can sound like a real person’s. Don’t prompt for a named person’s voice, and don’t put words in a real person’s mouth without their consent; for a voice you need exactly, record it with permission.

Most current video models generate sound. On DreamBench, some always do and some have a switch, and the table below is read off the composer’s own settings:

Sound is always on with Veo 3.1, Kling 4.0, Wan 3.0, Grok Imagine 1.5, MiniMax H3 and HappyHorse 1.1: their makers offer no way to turn it off. Seedance 2.5, Seedance 2.0, Kling 3.0, Wan 2.7 and LTX-2.3 have a switch. Vidu Q2’s only sound is background music, which a DreamBench film leaves off.

Turning sound off lowers the price only on Kling 3.0, whose provider bills a silent clip for less; on every other model, Seedance 2.0 among them, though ByteDance describes it as a joint audio-video model, the switch changes what you hear, not the credits. Each model’s page on the models list gives its lengths and settings.

On DreamBench you describe the film, and DreamBench writes every clip’s prompt, sound included. Each clip with sound gets a sound line of its own: the ambience of the place, each speaker’s voice, how many lines are spoken, by whom and in which language when your brief names one, and how the sound ends at the cut.

• A character’s voice is decided once and written word for word into every clip they speak in, because a voice described in new words each time comes out as a new voice.
• A film’s clips carry their own sounds and voices, never music: each clip renders on its own, so a score made inside one would stop dead at its cut.
• The Audio switch sits beside the model in the composer. On a model whose sound is always on, it shows on and can’t be turned off.

To write a single prompt with sound yourself, the AI video prompt generator has a sound part, from ambience alone to one spoken line in quotes, and the Veo 3.1 prompt generator writes for a model whose sound is always on. Read more on AI video generation, text to video and multi-shot video.