[WIP] Converting My College Project into Photoreal Turntable

I've been testing AnimateDiff to make a turntable animation of my college project for 3d Modeling which was a Honda Nsx 1990 Model Behance Project Link. My Process was as follows:

• 1)  I downscaled and Exported turntable video in image sequence of 180 frames in 1280x720
• 2) Then used the ControlNet passes export json file and rendered LineArt and Hed (with safe disabled ) passes as they give the most details for hard surfaces.
• 3) In The main workflow, I Chose "Epicrealism NaturalSin" model for Render_1
• 4) I also rendered with another model "Aniverse" which is attached below Render_2

Observations 1 :

It was observed that with 1280x720 only 15 frames were possible for my RTX to render in one batch. In the second batch the background changed violently even with the same seed.
The Car in the source video has only the car isolated with no background details which make it impossible for the ai to guess the same background after 15 frames.

Forcing the prompt, to have that same background somewhat reduced the drastic change. but not efficient way to do.

Observation 2 :

The Wobbling of the wheels, Distortion was observed in the wheel area of the car, The Softedge pass fails to detect minute details of the wheel, So LineArt pass was used with 0.25 weight and Softedge with 0.8.   
Lowering the Resolution increased the wobbling and higher resolution decreased the wobble.

Conclusion:

In the Next test

• Will try to place some objects in parallax or in perspective grid and test if forcing with background lines helps to fix the drastic change or not.
• Will try to weight the lineart more and see if the wobbly gets reduced or not.

To get a full 360 degree turntable rendered you need to lower the resolution and increase the batch  size to maximum. But with increase in batch size the details get reduced as it get blended in between frames. So Its a compromise between details vs Drastic change.
As Lower Batch Size (20-30 Frames) == Highest Details

AnimateDiff has also update with region prompt, I will also test this new update if available inside comfy.

Update:

Part 2 of this Document is Now Available : https://www.patreon.com/posts/92687226