Sound Check: Testing the latest lip-sync models' ability with music-making models

Method

Everything ran on October 7, 2026, through FAL's API, one run per tool per song. We kept every result, including the bad ones. No retries, no picking the best of three.

Who ran it. Claude Opus 5.5 in Claude Code ran the test. It made the songs and singer images, called each model, pulled frames with ffmpeg, scored every clip and wrote the first draft of this post.

The songs. Two songs made the way Music House makes them: our own lyrics and style prompt, planned section by section and rendered by ElevenLabs Music v2. One is a moody female R&B song, the other a male rap song. Both choruses start at 16.3 seconds, so the test window is 16.3 to 31.3 seconds, taken from the song's section times.

The vocal. We split the vocals out of each window with Demucs (htdemucs). Every model heard the same vocal stem, not the full mix. The full mix goes back on top afterwards.

The singers. One AI-generated singer per song, made with FLUX.2 Max. Nobody real, no famous likeness. Each image follows the shot rule we learned in earlier rounds: one front-on close-up, shoulders up, face about a third of the frame, nothing covering the mouth, no hands, no microphone.

Two kinds of tools, identical inputs.

• Patch after tools change the mouth in a video that already exists. They all got the same base clip: Gemini Omni Flash 1.1 animating the singer image, 10 seconds plus a 5 second continuation from its last frame, joined into one 15 second shot with no cut.
• Audio-driven tools make the video from a still image and a vocal. They all got the same singer image and the same vocal stem.

We used default settings. Where a tool requires a setting, we name it: Fabric at 480p, OmniHuman at 720p, H3 Max at 768p, Sync React-1 in face mode with a neutral emotion, LTX at 9:16. LTX, Kling and OmniHuman accept a prompt, and all three got the same short one.

The score. For every frame, MediaPipe face landmarks give how open the mouth is. We correlate that with the loudness of the vocal stem. That is the sync score: higher means the mouth opens when the voice does. We also measure how open the mouth is in the vocal gaps compared with while singing, the gap ratio: lower is better, and 1.0 means the mouth ignores the silence. Then we looked at the frames ourselves, every half second, and wrote down what we saw with timestamps.

Cost and speed. Costs come from FAL's usage API for the run, at FAL's list price. Time is split into queue (waiting for a GPU) and compute (making the video), from FAL's request log.

Use the scores to rank, then watch the clips. A mouth can move in time with the voice and still look wrong, and the reverse.

Every tool, with links and list prices

Patch-after tools also need the base video, so their real cost is the tool plus $1.50. Audio-driven tools need one singer image, $0.07 here. No affiliate links.

VEED published its own lip-sync API comparison, also shared by VEED co-founder Sabba Keynejad. It is a vendor-run test on talking heads, which is a different job from singing.

Results and defects, tool by tool

Sync is the correlation of mouth opening with the vocal at zero lag. Totals include the base video or singer image. Compute and queue are the range over the two songs.

Patch after: the mouth moves, but it does not stop

Gemini Omni already makes the singer "sing" in its own rhythm, unrelated to our song: its sync score is -0.23 on the R&B chorus and 0.06 on the rap. Patching pulls that up. Sync lipsync-2 reached 0.28 on both songs, the best of the patch-after group and the cheapest. But every patch-after tool leaves the mouth open in the silences. In the first second of the R&B clip and the 2.4 to 4.9 second gap in the rap, the patched singers keep mouthing.

What we saw, frame by frame:

• Sync lipsync-2: clean skin and identity. Mouth stays parted in gaps (0 to 1.2 s and 6 to 7.2 s on the R&B). Irregular, gapped teeth at 8.0 to 9.0 s and 12.5 s (R&B) and at 2.0 s (rap).
• Sync React-1: wider, more expressive mouth on the R&B, but on the rap it scored zero: the mouth does not follow the verse and stays as open in the long gap as while rapping (gap ratio 0.95). It costs 3.3 times lipsync-2, takes 5 minutes, and accepts at most 15 seconds per request.
• VEED Lipsync v2: big, visible mouth shapes and the lowest gap ratio of the patch group on the rap (0.47), but the mouth leads the voice by about 125 ms on the R&B.
• HeyGen v3 Precision: changed the mouth only lightly. On the R&B it barely moved the score (0.09). It is built to re-dub speech, and it only accepts a video that has an audio track.

Audio-driven: the singer comes from the song

These models never see a moving video. They get one photo and the vocal, and they make the performance from the sound. All six close the mouth in the gaps better than any patch-after tool.

• FlashTalk: best sync on both songs, and the cleanest silences on the R&B (gap ratio 0.28). Identity holds, head and framing stay steady, eyes stay open. Trade-offs: output is 448 x 768, so it is softer than the rest once scaled up, lips come out slightly redder and teeth whiter than the source on the rap, and fast closures snap shut in one or two frames (R&B 7.5 s, 10.2 s). It also waited 6 to 8 minutes in FAL's queue for under 2 minutes of compute.
• VEED Fabric 1.0: tied FlashTalk on the R&B (0.53) and dropped to 0.28 on the rap. Steady and clean, at 480p. Hands rise into the frame at the end of the rap clip (14.5 s).
• LTX-2.5 Fast: strong sync (0.36 / 0.40) and the fastest high-resolution result (1080p in under a minute). It also moves the most: big head tilts, eyes closed, and the R&B singer looks older and strained by 14 s. Mouth smears on the rap at 2.5 to 4.0 s and 13 s. Skin colour shifts at 12 to 12.6 s (R&B).
• Kling Avatar v2: good sync, but hands come up and gesture in front of the body from about 3.5 s on the R&B, and the singer smiles through a song about waiting for someone.
• MiniMax H3 Max: the most natural motion and lighting, and the fastest overall (about 45 s). Sync is mid-pack, and the mouth runs about 80 ms ahead of the voice.
• OmniHuman 1.5: hands in frame for the whole R&B clip, eyes often closed, and on the rap the face changes and the skin warms from about 5 s. Slowest at almost 7 minutes.

Raw data

Every number, the scoring script and the exact FAL requests: lipsync-benchmark-2026-10.zip (116 KB). The FAL bill for the run was $33.25; the two songs cost about $0.40 on ElevenLabs.

FAQ

How much does AI lip sync cost for a song?

On FAL in October 2026, 15 seconds ranged from $0.31 (FlashTalk) to $2.50 (Sync React-1) for the lip sync alone. Patch-after tools also need a base video, which added $1.50 per 15 seconds here. A full 3 minute song with FlashTalk would be about $3.60 at list price, if you lip sync all of it.

How long can a lip-synced clip be?

It depends on the model. Sync React-1 accepts 15 seconds per request. LTX-2.5 Fast takes 2 to 20 seconds of audio. OmniHuman 1.5 takes up to 60 seconds at 720p and 30 seconds at 1080p. MiniMax H3 Max charges 1.2 times more past 15 seconds. We tested 15 seconds on every model. Longer clips mean several requests and a seam you have to hide.

Can I use my own photo?

Audio-driven models work from any clear, front-on photo, including yours. A neutral face with the mouth closed works best. Avoid glasses glare, hands near the face, and anything covering the mouth. We used AI-generated singers for this test so nobody real is shown.

Do I need permission to lip sync someone's face?

Yes. Only use your own face, or a person who has agreed to it. Do not make a video of someone saying or singing something they did not agree to, and do not use a famous person's likeness.

Why is FlashTalk so cheap but slow?

It is a small model that bills $0.02 per second of output and renders at 448 x 768. In our run the compute took 42 to 117 seconds, but each job waited 6 to 8 minutes in the queue first. Queue time depends on demand and can change from day to day.