Flux.1 - Speed Boost + More ControlNets & LoRAs!
Added 2024-09-01 12:51:44 +0000 UTCvideo - https://youtu.be/JqEi82Q9Qfc
Hyper-SD have released some Flux.1 loras for you to reduce the number of steps required in Dev down to just 8! :)
If that isn't enough, Shakker & InstantX have partnered to give us all these goodies:
Controlnet Canny
Controlnet Depth
Controlnet Union
Controlnet Union Pro
LoRA Add Details
LoRA Blend Realistic
They all work with ComfyUI core, so no extra nodes to install (other than image pre-processors to easily create inputs for their controlnets)
(example workflow screenshot)
Update!
Update! Apparently the union modes are like this:
canny = openpose
tile = depth
depth = hed/pidi/scribble
blur = canny/lineart/anime_lineart/mlsd
pose = normal
grayscale = segment
low quality = tile
Makes sense :)
Model File Download links
https://huggingface.co/ByteDance/Hyper-SD
https://huggingface.co/Shakker-Labs
https://huggingface.co/InstantX
Attachment info
1. Flux_Dev_TXT2IMG_LLMParty_HyperSD_v2.json = Flux Dev Text 2 Image with LLM Party & Hyper-SD
2. Flux_Dev_TXT2IMG_LLMParty_HyperSD_MultiLora_v3.json = Same as above, but with a lora loader stack to more easily use multiple loras
3. Flux_Dev_TXT2IMG_HyperSD_ControlNet_v0.json = Flux Dev Text 2 Image with Hyper-SD & Control Net
4. Flux_Dev_TXT2IMG_LLMParty_HyperSD_MultiLora_NoClear_v3.json = Same as 2, but doesn't unload the LLM
Notes
Use ComfyUI manager to install any missing nodes - https://github.com/ltdrdata/ComfyUI-Manager
Use "Update All" in ComfyUI manager to keep up-to-date
Comments
It's been a while, but I'd say it's still back with AnimateDiff & SD1.5 - https://github.com/nerdyrodent/AVeryComfyNerd/blob/main/workflows/SD15/Video_Restyler.png
Nerdy Rodent
2024-09-20 15:31:34 +0000 UTCThanks. What do you think the best Vid2Vid workflow is right now?
Christopher Manzione
2024-09-20 13:31:10 +0000 UTCNothing V2V as yet, though Flux I2V should be next quarter
Nerdy Rodent
2024-09-20 10:02:44 +0000 UTCSomehow fixed this. I have no idea how. Working great! Would love to do Video to Video with this
Christopher Manzione
2024-09-18 21:21:51 +0000 UTCAlso I have the following error on this workflow "The size of tensor a (3072) must match the size of tensor b (38) at non-singleton dimension 3" Any idea why this is happening?
Christopher Manzione
2024-09-18 19:50:37 +0000 UTCHey Nerdy Rodent, Is there a workflow here for Flux that would work for Video to Video?
Christopher Manzione
2024-09-18 19:07:19 +0000 UTCI clone environments in quite a few cases, but especially if I'm upgrading or testing custom nodes (especially ones with strict requirements.txt files). It's much easier to just create a new clone should anything go a bit weird and start downgrading packages, etc :)
Nerdy Rodent
2024-09-10 11:24:17 +0000 UTCYup, should work just fine!
Nerdy Rodent
2024-09-10 11:17:08 +0000 UTCwhoa. you make new environments just for the extensions? i mean i guess it IS a party. :) that's a good idea but I always assumed it wouldnt work.
NetDude
2024-09-08 00:27:53 +0000 UTCDo you recon it would be possible to use xlab's ipadapter and these controlnets in combination with one another?
John Doe
2024-09-06 11:07:33 +0000 UTCLLM party only uses the VRAM while it's loaded, so with OLLAMA_KEEP_ALIVE=0 it essentially uses 0 extra VRAM. It's also easily removable if you don't want any LLM enhacements :)
Nerdy Rodent
2024-09-04 19:52:41 +0000 UTCLLM party doesn't have any fixed version numbers in it's requirements.txt, so it should play well with everything else (I've got over 30 custom nodes installed XD). If you do find you've got other custom nodes impacting your install, just run another environment. (I tend to clone them first when I see custom nodes with hard version numbers in)
Nerdy Rodent
2024-09-04 19:50:44 +0000 UTCLLM-Party node broke my ComfyUI install due to something with torch, It was a hassle to figure out how to restore it, please continue to create workflows without in the future if possible!
ed.enai
2024-09-04 12:59:20 +0000 UTCI find the LLM party addition which has become a staple in your workflows really eats up unecessary ressources while not giving, really, that much of a benefit(in terms of ressource cost) could you think about returning to or maybe making something a little more simplified? Of course it is up to you...I mean, who am I too...really no but who am I....(who am I?)....Thank you for your time!
Asyouare
2024-09-04 09:18:31 +0000 UTCThank you good sir. All is well in the world!
Richard Gates
2024-09-01 18:09:26 +0000 UTCI've deleted the node in this version - Flux_Dev_TXT2IMG_LLMParty_HyperSD_MultiLora_NoClear_v3.json
Nerdy Rodent
2024-09-01 17:45:55 +0000 UTCI tried that and for some reason I get this with it disabled. Error occurred when executing CR Text Concatenate: can only concatenate str (not "NoneType") to str File "D:\AI\pinokio\api\comfy.git\app\execution.py", line 317, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "D:\AI\pinokio\api\comfy.git\app\execution.py", line 192, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "D:\AI\pinokio\api\comfy.git\app\execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "D:\AI\pinokio\api\comfy.git\app\execution.py", line 158, in process_inputs results.append(getattr(obj, func)(**inputs)) File "D:\AI\pinokio\api\comfy.git\app\custom_nodes\ComfyUI_Comfyroll_CustomNodes\nodes\nodes_utils_text.py", line 222, in concat_text return (text1 + separator + text2, )
Richard Gates
2024-09-01 17:24:25 +0000 UTCI've added an option "clear model" node for lower VRAM systems. You can change the settings in that node so that "is_enable" = false. That way the ollama model won't unload each time.
Nerdy Rodent
2024-09-01 17:08:32 +0000 UTCHey Nerdy for the Flux_Dev_TXT2IMG_LLMParty_HyperSD_MultiLora_v3.json workflow, any idea why the LLM Party section would create a prompt very quickly the first time and then take several minutes to do the next one? I have 32GB of system RAM and 16GB of VRAM. I thought one of your previous workflows with the LLM party was faster, but I could be mistaken.
Richard Gates
2024-09-01 16:56:58 +0000 UTCYup! Almost enough for Flux Reposer... just need a face-plus IP adapter!
Nerdy Rodent
2024-09-01 15:45:41 +0000 UTCAdded the video guide link for more info ;)
Nerdy Rodent
2024-09-01 15:44:58 +0000 UTCYeah.... this doesnt work.
KING LIFER
2024-09-01 15:29:14 +0000 UTCOne of my favorite workflows in SDXL is taking the union ControlNet and using IP-Adapters style transfer to effectively restyle or reimagine a base image with a new image. I'm excited to plug these technologies together in Flux!
Andro.Meta
2024-09-01 14:25:53 +0000 UTC
