Gav Rymill

Image to Video keeps improving...

Morning all,

I'm sincerely sorry for the lack of updates, things are difficult at the moment. One thing I did get the chance to look at was Luma Labs new release. Obviously I'm obliged to caveat any mention of AI with the fact it's evil and we're supposed to hate it etc etc, but wow this is a huge step up.

Yes, clearly the body movement is very off, and that guy's collar and eyes are weird, but the rest of it is incredible. The way it handles the camera move, and the whole room re...

Telesnap #32 Behind the Scenes

Hello!

Here's a little making-of showing how I put together this latest telesnap recreation.

Gav

Telesnap #32 - Doctor von Wer!

Hellooo!

Now, here's one I could really get my teeth stuck into. A very complex composition, with three main performances, plus two background people. Lots of layers to deal with and a prop to think about. What I love about Viggle is how effortlessly is sorts out the three-dimensional nature of the hats, for example. That would have been a huge headache with EbSynth, as is any turn of the head, but provided we don't get too precious about the loss of absolute likeness when the head turn...

Telesnap #15 - Alternate Version

Hello!

This was an earlier version where the entire body had movement which I had wondered if it made him look a bit artificial. So for the version I posted earlier I tweaked it so the arm and kilt had little/no movement. Based one feedback that the rock-solid arm looked strange, here's the unsteady version.

Gav

Telesnap #15 - Shot to Pieces

Evening All,

This was only a moderate success with Viggle, and maybe it would have been better done a different way. I may revisit this one. It was pretty straightforward to do, but a extra little detail which I think helps sell it is that I ensured the shadow across his hairline didn't stay stuck to his head when he moved. All these small things help trick the brain, but take the extra time.

Gav 

Telesnap #10 - TARDIS Disembark

Hello everyone!

Here's my rendition of the TARDIS crew leaving the ship. I acted out the movements myself for each of the three characters, and then used that as motion capture and fed it to Viggle to generate the movement. I separated the telesnap into several layers with the bushes on top, and I created a clean background plate, with the people in the middle. In addition, I added movement to the leaves using stock footage overlaid. Perhaps too subtle? I didn't want to replace the ori...

Telesnap #40 - English?!

Evening all,

Another new software tool called Viggle has presented itself which has shown a lot of potential. This is back to using motion-capture again, so requires me to act out the movement and lip-sync to the audio, which is hard work, but I think the overall movement is enjoyable. It does a pretty decent job of mapping a flat cutout of a person onto 3D movement and although it doesn't maintain the likeness, I think it's quite impressive that it can generate a turn of the head out o...

Telesnap #45 - Disrespect

First attempt at using this new lip-sync on Troughton. It's not great. Part of the problem is that in the original telesnap, he's smiling and his eyes are closed, so it's all a bit uncanny for that reason. We're also just all very familiar with how the Doctor looks so doing the regulars is always going to be the biggest challenge. Other notes: I think the last lines feel wrong on this shot. I'm sure it would have cut away. And the Sergeant is meant to spit, which I'm not sure I can do very ea...

Telesnap #34: Spying a Wench

Each time I do one of these, I work out how I can improve it, but instead of redoing the same ones I'll keep moving forward and apply my ideas to the next one. Eventually I'll circle back round and apply what I've learned to the first ones. Although by then, some new tech will have come along!

Telesnaps #37 & #38

Recently, one of the Machine Learning systems (which everyone is calling AI) called Runway released a little update with a new lip-sync tool. Previously I had experimented with the MyHeritage tool which was fine but it lost a great deal of the the likeness from the original photo.

The Runway tool is pretty cool and will generate lip movement either from a wav file (where it attempts to interpret the words) or a text input (which gives you more accurate mouth shapes). In this clip I've ...

Telesnap #9 TARDIS Landing

Hello!

After my break I've started back on this again and having lots fun with it. This is a nice and easy one to tick off. I know the dolly movement of the camera is almost certainly incorrect for this location work, but I wanted to give it some depth. Do you think it's too subtle?

Another update in a few minutes!

Gav

ps. If anyone is curious where I've been. I've been working away, as a Supporting Artist for a new crime drama. I've so far done four weeks, totalling 1...

Festive Pause

Hello all,

Just a quick update to say that the project is still progressing, but not as quickly I'd hoped so rather than let more time go by without a decent update, I'm pausing subscriptions. I'll get back to working on this properly in January. Thanks all for your continued support and I hope you have a great Christmas and New Year!

Best wishes,

Gav

Wheel in Space Studio Walk-Through (Bonus)

As a small apology for the lack of update, here's a video of what it would have been like to walk to walk round part of the set of Silver Carrier! Sorry about the glitch, I only just spotted it!

Moving Plate 1

Hello!

Sorry for the lack of updates! I did what I said I wouldn't do and jumped ahead to work on the backgrounds of other shots whilst also trying to get that TNA video finishing so I'll upload some more soon. Meanwhile, here's a small update on Telesnap 1 showing how it looks in motion. A surprisingly tricky one to get right and more work is needed to slow it down.

CGI Space Pirates Fuel Store (Bonus!)

To offer you a little something extra, I thought I'd share this set recreation I did of The Space Pirates atomic fuel store and adjacent room as seen in episode six. This is how it appeared in Ealing on February 14th 1969 but I suspect it's not finishing being dressed. My hunch is that those alcoves had the radiation suits in. I've worked out some of the camera positions but I'm really puzzled about how the Doctor was filmed without being constantly obscured.

Not Very Exciting Update!

Hello!

Sorry it's been quiet. I failed to get the mocap for Snap #1 done this week because I want to do it outside and the weather has been shocking. Here's the clear background plate and I've set up the 2.5D environment to generate the depth so I'll hopefully share that in the next couple of days. In the meantime I'll post a short bonus video....

Cheers

Gav

Telesnap #1

Well this is a baptism of fire. I'm determined to be methodical and not skip ahead. But this shot of four figures all interacting and overlapping with what I suspect is a camera dolly ... is tricky to say the least. Anyway, here we go!

20 Seconds of 'Power' with Me as Lesterson

Another experiment I dug up from the depths of my hard drive, done many years ago. I never intended to DeepFake this or anything, I just thought I'd see what a virtual set and virtual Dalek looked like...

The Votes Are In!

Hello and welcome to everyone! Thanks so much for supporting this project. I'm really very grateful as I know times are hard for us all.

The votes have been counted and the winner with 44% is The Highlanders! I hope the remaining folks aren't disappointed. That's democracy for you.

I'm secretly rather pleased, having worked on clips for the other two already, so this will feel fresh and exciting. And there are some glorious shots I can't wait to bring to life. But wait I must, bec...

Wheel in Space Animatic - Half of Episode 1

Using the camera script I have tried to faithfully recreate all the shots. It was tricky to work some of it out because we don't have a studio plan, but by following the camera moves and looking at what the shots showed (and what the camera couldn't see) I've been able to work most of it out. There are some corrections to make and hopefully at some point I'll be able to add more footage.

What Episode Should I Do?

Out of the stories with Tele-Snaps, there are three which I'm particularly keen to work on, as they feel the least familiar and it would be most stimulating to try to resurrect. They're also only four-parters, so if I do manage to complete an episode, then three more might not seem unreasonable.

Proof of Concept #2 - The Smugglers Trailer

So this is the culmination of all the methods I've been playing around with to recreate 50 seconds of The Smugglers for Tim Burrows' Missing Episodes Podcast. Most of the MoCap is done using my dad as the performer, and then he has been replaced using EbSynth. It took quite a lot of re-running the synth with different keyframes to get it all to this point, and it's not perfect. Some of the movement is done in Cinema 4D, to generate parallax (for example in the churchyard) and I also used stoc...

Animating Tele-Snaps Proof of Concept #1

This is my trailer for Tim Burrows' Missing Episode Podcast covering "The Savages", on which I appeared. I used this as a focus of my efforts to create shots relatively faithfully based on Tele-snaps. The speech is done with Wav2Lip and the rest of the movement is done in Cinema 4D.

Jano Ebsynth + Wav2Lip (The Savages)

This was an early test to see if this process might work, and I found it fairly encouraging. The body is animated first (rather too exaggeratedly I know) and then fed to the Wav2Lip software to make him speak. Quick to do.

Hungry Jamie AI Animated (Wheel in Space)

A very speedy way of generating footage, but the trade-off is that it's uncanny. And as with several of these processes, the only animation is in the mouth, so the lack of expression is jarring.

MoCap from Video Test

If ever I want to recreate footage with Unreal Engine, I'd need a motion capture solution. This piece of software is pretty amazing for pulling MoCap data out of video, which means no ping pong ball suits needed! 

Chen and Monk Deepfake (Master Plan)

I did have a longer version of this where the Monk continues speaking and moves his head, but I just couldn't smooth it out well enough to keep it. The shots are not authentic to the camera script but relied on a useful shot of the actors.

Bragan Addresses Colony (Power of the Daleks)

Mostly Deepfake with a bit of jiggery-pokery to create the image on the screen. 

Doctor to Emperor (Evil of the Daleks)

A Deepfake again based on a single shot, and without compositing onto a body, therefore hidden in a TV frame to hide the issues!

Brigadier Deepfake (Web 3)

Tried desperately to hide the shortcomings of this (and failed). The problem I have is that training on multiple clips taken from different scenes tends to produce a lot of flicker. So by restricting the source material to one clip, it makes it a more stable face, but the trade-off is it's less expressive, because there the source clip doesn't have every possible expression.