NerdyRodent

Reposer+ with bypass & composition

Just for giggles, I wanted to see if I could add composition to Reposer+ and it turns out it does kinda work! It's best if the composition image "makes sense". For example, if the composition image is of an alien landscape, then something similar should be in your prompt like "Alien landscape in the background". Of course, you can use any image and prompt mix to see what weird things happen, or simply bypas...

BrushNet

BrushNet is a great way to do inpainting. Below you can find all the links to the things used in the video, along with each command to copy & paste to get everything set up. Also included are the two different ComfyUI workflows used in the video. One has the standard masking setup, and one with instead uses depth maps as masks. In either workflow you can use your own masks too by connecting the mask node provided instead.

Video guide: 2024-04-06 15:20:01 +0000 UTC View Post

Node templates - April 2024 update

Node templates make life easier. I mean, who wants to pick the colours and layout each time for every node? A whole bunch of nodes have changed since my last release of these time-saving node templates, so this latest release uses all the current release versions :)

AniPortrait

AniPortrait is a fun new python app that can be used to... animate portraits :) A few commands are missing from the current documentation, so here they are in full!

== Setup / Install Command Summary ==

git clone https://github.com/Zejun-Yang/AniPortrait.git

cd AniPortrait

conda create --name anip...

Realiser 6a

A work-in-progress update for Realiser - drop in a cartoon image and make it "real" :)

Now uses the new, new IP Adapter (Plus) and also has a bunch of upscaling in there too.

Enjoy!

IP Composition Adapter

There is a new IP Adapter model available, and this one is for image composition :)

https://huggingface.co/ostris/ip-composition-adapter

Attached are the two workflows used in the video. One has just the ip composition adapter, while the other combines it with Visual Style Prompting to allow both...

Visual Style Prompting

Visual Style prompting allows you to use an image as a style without the need for any additional fine-tuning or training. Just drop and go! One of the best so far, I think :)

You can find the two ComfyUI workflows I use in my Visual Style Prompting video attached here. They're both the same, just one is set up with stable diffusion 1.5 model names and the other is with SDXL ones.

The original repo can be found at 2024-03-17 22:20:01 +0000 UTC View Post

Stable Diffusion - Layer Diffuse ComfyUI

This is the example workflow used in my Layer Diffuse video.

It gives examples of how to use each of the new "Layer Diffuse" nodes for creating transparent images, adding / removing backgrounds/foregrounds, etc.

All sorts of fun to be had now!

Enjoy :)

Gemma 7B in Oobabooga Text Generation WebUI

The Oobabooga Text Generation WebUI is a great way to test a variety of large language models - https://github.com/oobabooga/text-generation-webui

With the recent release of Google's "Gemma" models - https://huggingface.co/google/...

Comparing Models in the Forge X/Y Grid Script

https://youtu.be/3Gl9WK40wkE

(*Image created using Proteus V0.3. Also, 0.4 should hopefully be out soon!)

The Forge WebUI for Stable Diffusion is nicely optimised, meaning SDXL models work great - even on mid or low-end GPUs... but which SDXL model do you use?

In my latest video there are some X/Y grids to help......

Photomaker Plus update

Just a tiny update to the photomaker plus workflow (shii), now that the repo has changed name and comfyui supports photomaker by default! :)

Dontcha just love it when everything changes?

Creative Interpolation + Sparse RGB

Updated version of the creative interpolation with sparse rgb which is much easier to follow and has far fewer noodles :)

Back from Lisbon!

Had loads of fun at the Champalimaud Foundation in Lisbon, with the hackathon code being available at

https://github.com/lunarring

Body2vec + coupling diffusion was great fun, and would make a really great interactive art exhibition, and I'm pretty sure it will happen some time this year ;) It won't be as fast, but I'll probably have a go at doing something close in Comfy soon

PhotoMaker vs IPAdapter comparison

Just a simple workflow using the shii photomaker nodes, along with animatediff plus controlnet, along with ipadapter doing the same thing. This makes it easy to compare the two and see which you prefer :)

PhotoMaker Alternative Workflow (Shiimizu Version)

This workflow is more complex than the Zho-zho-zho version, but it does have the advantage of being more ComfyUI-like and supports most standard nodes :)

https://github.com/shiimizu/ComfyUI-PhotoMaker

It seems the image input for this version doesn't seem to work as yet, so for now you must specify an image directory path.

Personally, I prefer this version even if the image input doesn't q...

Updated PhotoMaker Workflow

I much prefer working with images I can see, rather than directories of images I can't, so here's a little update to the photomaker workflow which takes 4 image inputs! Ideal for 4 images of the same face... or to mix Elon & Trump like in the project page examples ;)

The new version of the custom nodes now also support LoRAs for extra fun!

Uses this implementation:

https://github.com/ZH...

PhotoMaker ComfyUI

Just a basic PhotoMaker workflow for ComfyUI. Works best with multiple images (3-8) in a single directory.

There are loads of PhotoMaker custom nodes, but this workflow uses the ones from  https://github.com/ZHO-ZHO-ZHO/ComfyUI-PhotoMaker

PhotoMaker's ".bin" file can be downloaded from https://huggingface.co...

DeciDiffusion Testing in ComfyUI

A DeciDiffusion model has just gone public so I figured I'd give it a quick test in ComfyUI! Now you can too via this workflow. They use a custom version dpmpp so it isn't exactly the same, though pretty close, I guess? From their release info:

Produces quality images on par with Stable Diffusion v1.5, but 𝟐.𝟔 𝐭𝐢𝐦𝐞𝐬 𝐟𝐚𝐬𝐭𝐞𝐫 𝐢𝐧 𝟒𝟎% 𝐟𝐞𝐰𝐞𝐫 𝐢𝐭𝐞𝐫𝐚𝐭𝐢𝐨𝐧𝐬

Employs a 𝐬𝐦𝐚𝐥𝐥𝐞𝐫 �...

Come say "Hi!" ;)

If you're in or around Lisbon later this month, I'll be presenting at the Champalimaud Foundation for their Workshop on Interactive AI Systems for Digital Therapeutics, along with some other amazing people such as Robin Rombach! There are only a few tickets remaining (for a grand total of FREE ;), with more info available at the following URL:

https://www.fchampa...

AnyText Installers

AnyText uses Stable Diffusion to improve text generation in images. These installers will simplify the installation, though users of Microsoft Windows may require multiple reboots. Best run with the conda of your choice already installed, though the scripts will also try to do this for you.

For Linux, just download and run the script (bash anytext-install-linux,sh)

For Microsoft Windows, download and run the powershell script from your anaconda powershell prompt (.\anytext-windows...

MotionCtrl 0-Click Installer for Linux

MotionCtrl (https://github.com/TencentARC/MotionCtrl) dropped recently and it now also has a gradio interface for easy testing.

As it uses both triton and an early version of xformers, this is best run on Linux.

Just download the script and run as normal (bash motionctrl-install-linux.sh). 0 mouse clicks required ;)

Enjoy and have a great New Year!

AnyDoor Installer for Linux

AnyDoor for MS Windows has an installer available at https://github.com/sdbds/AnyDoor-for-windows - but as I couldn't find one for Linux, I figured I'd just make one :)

The AnyDoor repo is at https://github.com/ali-vilab/AnyDoor

Essentially, you can mask objects and "teleport" them. It can also do a sort of "virtual try o...

ComfyUI Node Templates

Some Node Templates for ComfyUI. These are ones I use quite a lot as they really speed up workflow creation - just like in my recent video on AnimateDiffv3 :)

Krea AI Access For Nerdlings :)

So you like, AI, huh? Would you like access to Krea AI? I've got 5 tickets to give away, and rather than make it complicated, I'll just send the tickets to the first 5 people to respond saying they'd like one :)

Enjoy!

Reposer Plus Bypass Edition - Update

The old IPAdpater is becoming deprecated and Segment Anything now outputs multiple images, so a small update to the previous workflow.

* Now includes a version box ;)

* Use single words in the segment anything box for the best results

* Uses the more current IPAdapter which can also find models in the models directory now :)

Update: All IP Adapter Plus workflows now need the node to be replaced, like in the example Reposer.json

SDXL Turbo 2 SD1.5 LCM Workflow

With a massive 5 steps in total, this workflow generates an image in 1 step using SDXL Turbo, which is then passed to SD1.5 for an additional 4 steps in order to add just a touch more detail.

Tick "Auto Queue" in ComfyUI, and you'll have near realtime generation. For extra fun, change the "Conditioning Blend" mode and strength, to interpolate between your two positive prompts!

SVD Workflows

A couple of basic SVD workflows for you.

One does the standard image2video + upscale + frame interpolation, the other does a sort of text2image2video (+ upscale + frame interpolation).

Can eat a bit of VRAM with the upscaling ;)

AnimateDiff Inpainting with LCM

Just an experimental workflow with AnimateDiff, LCM, IPAdapter, Segment Anything and the Inpainting control net. For whatever reason, inpainting models don't seem to play well with AnimateDiff - hence the use of the controlnet.

Load your image, mask an area (or connect and use the masking prompt), set your area prompt (and optionally image prompt it too using the IPAdapter image) then generate away!

For Stable Diffusion 1.5 only

Realiser Workflow

I was really just messing about, but people in the twitterverse (X-verse?) seemed to like the outputs, so I figured I'd provide the workflow here!

It's really simple to use - basically just drop in your image of choice, and it makes it more "real". Turn a painting or pixel art in a photo. Perhaps give an emoji more of a 3d look and feel. The choice is yours.

Various example outputs on X - https://twitter.com/Nerdy...

Reposer Mod

Just a little mod to the base Reposer workflow to give some slightly better colours. Modifications to upscaling too, once again to balance the final image output. Overall, I think this updated version gives a slightly better output. It also benefits from a higher guidance scale to start with, in order to better match any text prompts.