What is text to video?

Text to video works by training a model on a very large number of clips paired with descriptions, so it learns which words go with which sights and movements. To generate, it encodes your prompt, starts from random noise the size of a whole clip, and removes the noise step by step until frames remain that fit the words.

Because nothing but the words constrains it, a text-to-video model has the most freedom of any mode, which is both its strength and its weakness: two runs of the same prompt give two different clips. See AI video generation for the pipeline in depth.

Some tools sold as text to video don’t generate footage. They take a script, pick stock clips to match it, and add an AI voice and captions. Text to speech is narrower still: it turns a script into a voice, with no picture at all. A text-to-video model works differently: it generates every frame, so nothing you see was filmed.

If you already have a picture of what the clip should look like, start from it instead: image to video keeps your picture as the first frame and generates the motion.

A prompt generator is different again: it writes the prompt and generates no video. The AI video prompt generator turns a few choices into a shot description in the order a model reads it, with presets for Veo 3.1, Seedance 2.5 and Kling 3.0; the clip is still made by a video model.

Video models read a prompt like a shot description. The clearer each part, the less the model guesses. A useful order:

• One shot per prompt: a model makes one continuous shot, so describe one camera position and one action.
• Describe what you want to see, not what you don’t: “an empty street” works better than “no people”.
• Name the camera move, or the model will pick one.
• Keep text on screen short, or leave it out and add it afterward.

This clip was made on DreamBench from the brief under it, on Veo 3.1 at 1080p with sound. It is a good brief because every sentence gives the model something to show, and nothing it can’t.

The quoted words are the brief as it was written, in English: a prompt works in any language the model understands.

Most weak prompts are short, not wrong. Each step below adds one kind of information, and each removes a choice the model would otherwise make at random.

A brief can ask for a single unbroken shot, a slow camera move and a spoken line, even in another language. This 30-second take on Seedance 2.5 was written as one continuous shot from ladle to smile, with the chef’s line in Japanese.

• Mood pieces and concept films, to try an idea before spending on a shoot.
• B-roll: cityscapes, nature, textures and crowds to cut between other shots.
• Social clips built around a single striking image or moment.
• Storyboards that move, so a director and a client can see the same scene.
• Backgrounds and loops for presentations, music and live events.

Every video model on DreamBench makes video from text. They differ in sound, in whether they can also take a last frame, and in how sharp they render:

No model is best for every brief. Each model’s page on the models list gives its length, resolution and sound options; test a brief on two and compare.

• It can’t know a specific person, product or logo from words alone; use a reference picture for those, as reference to video does, and keep it in every shot for a consistent character.
• Signs and captions often come out misspelt.
• Complex actions with several steps get shortened, merged or reordered.
• Crowds, hands and things touching are where errors show first.
• Clips flicker or drift when the model loses track of a detail: a failure of temporal consistency.

Why does my clip stop before the action finishes? The clip’s length is fixed before the model reads the prompt, so the action is fitted into it: an action that needs longer is shortened, loses steps, or ends mid-move. Write one action that fits the seconds you ordered, or order a longer clip; the rewrite table above keeps one action per shot for this reason.

On DreamBench you don’t write each prompt yourself. You describe the film, and DreamBench writes the shot list and a full prompt for each clip, with camera, light, action and sound, then renders them on the model you pick, such as Seedance 2.5, Veo 3.1 or Kling 3.0. Add pictures of a character or product, and they are used as references in every shot.