XaiJu
Mickmumpitz
Mickmumpitz

patreon


[NEW WORKFLOW & LORA] AI-Powered 360Β° Scene Generation (Early Access)

Hey Everyone!

I'm currently working on a new AI filmmaking workflow that I think you'll really like! One part of it will be the ability to accurately move the camera around in your scene.

For that, I wanted to see if the Qwen Image model could generate 360-degree equirectangular images, but unfortunately the results weren't great.

So I created a dataset of 20 equirectangular HDRI image pairs of different scenes (CC0 licensed) from Polyhaven.com to train my own Qwen Image Edit LoRA. I trained the LoRA on Runpod using Ostris guide.

πŸ‘‰Runpod for Lora Training https://runpod.io?ref=o2xn5klc*

πŸ‘‰ Guide I followed: https://youtu.be/0IaY8V5hCdU?si=EaeenLVqb0q59UCq

The results with the LoRA aren't perfect yet, but they're MUCH BETTER than the base model!

Just download the workflow & LoRA below and start generating.

  1. Download the models and place them in your ComfyUI folders.

  2. Your Input image

  3. You can change the padding here.

  4. Use a prompt like: "Create a 360 panoramic image of [very short description of surrounding scene]. An HDRI image captured on a 360 degree camera, insta 360, equirectagular projection. Keep the style the same."

  5. Change the seed if you are not happy with the result.

This is still work in progress, but I wanted to share it early so you can test it out. I think it'll be a useful feature for many of you! Let me know if you have any feedback on Discord.

*This is an affiliate link. Use it at no extra cost for you!

UPDATE 251021 - Version 1.1 is now seamless!

I just updated the workflow so that you can finally create seamless 360 degree images! It works automatically by shifting the seam to the middle of the image and then inpainting it. You can change the size, blur of the mask or the seed if you have issues.

[NEW WORKFLOW & LORA] AI-Powered 360Β° Scene Generation  (Early Access)

Comments

Once I downloaded and used the correct model as per Sebastian's comment: " In the workflow the link is wrong - you have to use https://huggingface.co/QuantStack/Qwen-Image-Edit-2509-GGUF/tree/main", the workflow works great with very similar results to the video. I have done a few little test videos now with pretty decent results.

Mark Burns

Long time lurker of your videos. Recent signed up for consistent character workflows (which are great) but I think this is what I'm really excited about. Generated a quick scene with a SDXL model and then used this workflow and bam! It worked really well. I even uploaded it to a 3d scene viewer and it was pretty good for having only spent 5 mins on it. Can't wait to see how this evolves. Nice work!

Brian Gebel


More Creators