Alibaba · Tongyi Lab

Wan2.2-S2V

The speech-to-video member of the Wan 2.2 family. Given one reference image and an audio track, it generates a video whose lip movement, head motion and gesture follow the audio.

Developer Alibaba Tongyi Lab
Task audio + image → video
Params 14B
License Apache-2.0

Overview

S2V extends the Wan 2.2 video backbone with audio conditioning. Rather than driving only the mouth region, the audio signal conditions the whole frame, so posture and gesture move with the speech as well.

Audio-conditioned motion

Lip shape, head pose and upper-body movement are generated jointly from the audio track instead of being composited from a separate talking-head module.

Identity from one image

A single reference frame supplies appearance and identity; no per-subject fine-tuning or reference video is required.

Speech and song

The checkpoint is trained on both spoken and sung audio, so it is not restricted to conversational delivery.

Long-form generation

Clips are produced in chunks with motion carried across boundaries, which keeps identity stable past the length of a single window.

Inputs

Field	Type	Notes
`image`	URL	Reference frame. A clear, front-facing subject works best.
`audio`	URL	Driving audio track. Determines the clip length.
`prompt`	string	Optional description of scene, framing and camera behaviour.
`seed`	int	Optional. Fix it to make a run reproducible.

Run it

Upload local files first with POST /media/upload/binary and pass the returned URLs, or reference any publicly reachable URL directly.

# 1. submit the job
curl -X POST "https://api.wavespeed.ai/api/v3/wavespeed-ai/wan-2.2/speech-to-video" \
  -H "Authorization: Bearer $WAVESPEED_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "image": "https://example.com/portrait.jpg",
        "audio": "https://example.com/speech.mp3",
        "prompt": "A woman speaking to camera in a sunlit studio, medium shot",
        "enable_sync_mode": false
    }'

# -> {"code": 200, "data": {"id": "<request-id>", "status": "created", ...}}

# 2. poll until status is "completed"
curl "https://api.wavespeed.ai/api/v3/predictions/<request-id>/result" \
  -H "Authorization: Bearer $WAVESPEED_API_KEY"

# -> {"code": 200, "data": {"status": "completed", "outputs": ["https://..."]}}

import os, time, requests

API = "https://api.wavespeed.ai/api/v3"
KEY = os.environ["WAVESPEED_API_KEY"]
HEADERS = {"Authorization": f"Bearer {KEY}"}

# submit
res = requests.post(
    f"{API}/wavespeed-ai/wan-2.2/speech-to-video",
    headers={**HEADERS, "Content-Type": "application/json"},
    json={
      "image": "https://example.com/portrait.jpg",
      "audio": "https://example.com/speech.mp3",
      "prompt": "A woman speaking to camera in a sunlit studio, medium shot",
      "enable_sync_mode": false
    },
    timeout=30,
)
res.raise_for_status()
request_id = res.json()["data"]["id"]

# poll
while True:
    data = requests.get(
        f"{API}/predictions/{request_id}/result",
        headers=HEADERS,
        timeout=30,
    ).json()["data"]

    if data["status"] == "completed":
        print(data["outputs"][0])
        break
    if data["status"] == "failed":
        raise RuntimeError(data.get("error", "generation failed"))
    time.sleep(1.5)

const API = "https://api.wavespeed.ai/api/v3";
const KEY = process.env.WAVESPEED_API_KEY;
const headers = { Authorization: `Bearer ${KEY}` };

// submit
const submit = await fetch(`${API}/wavespeed-ai/wan-2.2/speech-to-video`, {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify({
      "image": "https://example.com/portrait.jpg",
      "audio": "https://example.com/speech.mp3",
      "prompt": "A woman speaking to camera in a sunlit studio, medium shot",
      "enable_sync_mode": false
    }),
});
const { data: { id } } = await submit.json();

// poll
for (;;) {
  const res = await fetch(`${API}/predictions/${id}/result`, { headers });
  const { data } = await res.json();

  if (data.status === "completed") {
    console.log(data.outputs[0]);
    break;
  }
  if (data.status === "failed") throw new Error(data.error ?? "generation failed");
  await new Promise((r) => setTimeout(r, 1500));
}

Requests are asynchronous: POST returns a request id, then you poll /predictions/<id>/result until status is completed. Set enable_sync_mode: true to have the call block and return outputs directly.

API keys are created in the WaveSpeed dashboard.

Open on WaveSpeed API reference