Brief Side Quest: Missing War Machines

I've been thinking about how missing bits of otherwise recovered episodes have been dealt with in the past. The War Machines is a good test case because in this clip we have the start and frames of the missing seconds, so we know how it was shot.

The VHS and DVD releases both cut away to WOTAN, using differing duplicated footage. I wanted to see if we could get closer to what was removed by the censors.

I tried three different ways to generate new footage of Krimpton. Hedra output is pretty good, but too much movement, and I have no control. But it's excellent because it generates all the lip movement by following the audio, so that's a huge help.

LivePortrait is very fake-looking but at least it's less frantic, and it uses motion-capture which has pros and cons. Mocap is laborious to setup and requires me to learn the lines and perform the movements, which takes time and space to block out, then record myself, then upload, so it's a faff. But it does mean the character only moves when I want it to.

Kling produced a pretty good result, but it doesn't lip sync to audio, so I had to process that through another thing - Deevid - to get the lips in sync.

The problem with Deevid is that it can't match based on audio, only text input. And unfortunately the text-to-speech voice choses its own pace and intonation, so the timing of the lips doesn't match. I had to do some manual correction to keep it aligned, and it's laborious.

Once I'd done these tests, I then thought of a better way to do it. I should have taken the existing footage of Krimpton and changed the lip-movement on that, and not tried to generate new footage.

I have just learned of a new lip-sync tool called Sync.so, which I'm going to do a little test with shortly. This will redo lips based on audio, which would be invaluable.