Mickmumpitz

[Free Workfow] "LIP SYNC for your AI MOVIES"

Added 2024-02-09 14:12:55 +0000 UTC

I'm currently working on the next episode of my AI series about humanity's future in the cosmos (https://www.youtube.com/watch?v=X1NIjn_kJq8). And this time I wanted to have a talking character, a presenter who becomes the protagonist of the short film.

You may remember that in one of my older videos (https://youtu.be/nTLuH-uRaZ8?si=7ukgU_3pztAFC2ua) I created a pretty complex workflow for this. I used "Wav2Lip", which produces really good LipSync for free, but unfortunately with a very poor image quality.

I was quite happy with the results, but creating these shots took a lot of time because of the many steps involved. That's why I simplified and automated the with ComfyUI.

Download the Workflow here! And don't forget to play around with the strength (codeformer fidelity) and model (Codeformer or GFPGAN)!