What is character consistency in AI video?

A video model makes a film one shot at a time, and nothing passes from shot 4 to shot 5 unless you pass it: the model starts again from noise and invents whatever the prompt leaves open. Write “a woman with short blond hair” twice and you get two different women.

So consistency comes from what every shot is given. The same words help. A picture helps more, because a face is easier to show than to describe: models that take one, the mode called reference to video, keep the face, hair and colors they can see. A character sheet, the character from the front, side and back, helps most, because it also shows them turned round.

Starting a shot from a picture of the character, as image to video does, works too: the first frame is fixed, and the further the shot moves from it, the more is invented again.

Under the hood: references, identity embeddings and seeds

A reference picture reaches the model through attention, the same mechanism the text uses. IP-Adapter adds an image as a prompt through cross-attention layers of its own, kept separate from the text’s, in an adapter of about 22 million parameters, with the model itself unchanged.

Identity methods go further and encode the face itself. InstantID works from a single photo of a face: a separate network steers generation with that face’s features and landmarks, so the person stays recognizable while style and pose change. That encoded face is an identity embedding.

The oldest route is to teach the model the character. DreamBooth fine-tunes a model on a few pictures of a subject until it binds a unique word to it, then generates that subject in new scenes, poses, views and light. LoRA, first published for language models, makes such a fine-tune light: the model’s own weights stay frozen and only small added matrices are trained. Both need a model whose weights you can train.

Another idea is to generate the pictures together. StoryDiffusion shares attention across a batch of images, with no training, so a subject agrees across a series; multi-shot video models hold a cast across their own cuts in a similar way, but only within one generation.

Why doesn’t the seed carry the character? A seed fixes the starting noise, so the same seed with the same prompt, length, frame and model repeats the same clip. A new shot changes at least one of them, the same noise is read in a new way, and the last face is stored nowhere.

Both end in a face that changes, so they are easy to mix up. Temporal consistency is inside one clip: does frame 90 agree with frame 1? Character consistency is across clips: is the woman in the kitchen in shot 2 the woman on the train in shot 9? A steady clip can still show the wrong person.

First and last frame can carry one shot’s last frame into the next shot’s first: the join is smooth, but shot 12 still doesn’t know shot 3.

Each model maker has its own way in. Google’s Vertex AI guide takes up to three images of a single person, character or product for Veo 3.1, and says Veo preserves the subject’s appearance in the video; on the standard model, a clip made with subject images is 8 seconds long.

Kling builds characters as elements. Its Element Library guide asks for two to four pictures an element, one main and up to three from other angles, recommends a front-facing main picture, and says Kling 3.0 binds up to three elements to a start frame.

Seedance takes many references at once, with one rule that matters here: BytePlus’s Seedance 2.0 series tutorial says those models don’t accept reference images or videos that directly contain real human faces, and offers its own digital characters and authorized real-person assets instead.

A model copies what a reference shows, wanted or not, so a good sheet is plain on purpose:

• The same face and hair in every view, front, side and back, the skin the same tone in each.
• One character to a sheet: a second face is a second identity to borrow from.
• A plain ground with no cast shadow or color cast, which would turn up in every shot.
• No text, labels or numbers, which a model may write into the shot.
• A clear front view as the main picture, as Kling recommends, with the whole outfit visible.
• Places on sheets of their own, empty of characters.

Why does my character change halfway through a shot? That is drift inside the clip, a temporal consistency failure: a shorter shot, less motion and a cut before a big turn help more than a new reference.

• Make the character once, as a sheet, before any shot.
• Give that sheet to every shot that shows them, or start each shot from a picture made from it.
• Write their few defining details word for word in every prompt.
• Use one model for the whole film where you can: each reads a reference its own way.
• Compare stills from different shots side by side: a change you miss in motion is plain in stills.

A reference picture of a real person is their likeness. Use your own, or one from someone who has given you permission, and say the video is AI-made when it could pass for real footage. Some models, Seedance 2.0 among them, refuse real faces outright.

On DreamBench, uploaded pictures are checked before any credit is used, and a picture that appears to show a real person under 18 is refused: the Terms don’t allow uploads or depictions of real people under 18. In Morph, a video with real people is made only after you check a box saying it is you, or people who gave their permission.

When you describe a film, DreamBench writes the script, then draws the cast before a single shot is rendered: every character, place and prop once, as a sheet, the same in every shot. Add a picture under “Who’s in it?”, tagged as a character, creature, location or prop, and that sheet is drawn from your picture, taking the subject alone and not its light or background; a sheet you made before is used as it is.

Before a sheet is drawn, each character and prop is given an identity lock: the face, the profile, the body and the size, written for the whole cast at once so that no two characters share a face. A short line of identifiers from it is written word for word into every shot that shows the character, with the character’s voice on models that make sound and the number of people on screen.

How a sheet reaches a shot depends on the model. On Seedance 2.5, Seedance 2.0, Veo 3.1 and Vidu Q2, the sheets go with each shot as reference pictures. On the other models, Veo 3.1 Fast and Lite and Kling 3.0 among them, DreamBench first draws an opening still from the shot’s sheets, and the shot starts from that still.

A change the story needs, such as a character soaked by rain, is drawn as a variant on their own sheet. If you choose to review, you meet the cast before anything is shot and can draw any sheet again, or from a picture of yours. A shot that still misses is redone with a note saying what to fix.

Original

With Morph

Change the background to @livingroom, laid out as in @floorplan, but swap the 3 cats for @yickshuncat, @oreo and @mango.