Run ComfyUI workflows without spending a dime with Gradio on Hugging Face Spaces

Index:

Intro

On this tutorial I’ll present a step-by-step guide on convert a posh ComfyUI workflow to an easy Gradio application, and deploy this application on Hugging Face Spaces ZeroGPU serverless structure, which allows for it to be deployed and run without spending a dime in a serverless manner. On this tutorial, we’re going to work with Nathan Shipley’s Flux[dev] Redux + Flux[dev] Depth ComfyUI workflow, but you’ll be able to follow the tutorial with any workflow that you desire to.

The tl;dr summary of what we’ll cover on this tutorial is:

1. Export your ComfyUI workflow using ComfyUI-to-Python-Extension;
2. Create a Gradio app for the exported Python;
3. Deploy it on Hugging Face Spaces with ZeroGPU;
4. Soon: this whole process will probably be automated;

Prerequisites

• Knowing run ComfyUI: this tutorial requires you to find a way to grab a ComfyUI workflow and run it in your machine, installing missing nodes and finding the missing models (we do plan to automate this step soon though);
• Getting the workflow you desire to to export up and running (if you should learn and not using a workflow in mind, be at liberty to get Nathan Shipley’s Flux[dev] Redux + Flux[dev] Depth ComfyUI workflow up and running);
• A little bit little bit of coding knowledge: but I might encourage beginners to try and follow it, as it could possibly be a very nice introduction to Python, Gradio and Spaces without an excessive amount of prior programming knowledge needed.

(In case you are on the lookout for an end-to-end “workflow-to-app” structure, while not having to setup and run Comfortable or knowing coding, stay tuned on my profile on Hugging Face or Twitter/X as we plan to do that in early 2025!).

1. Exporting your ComfyUI workflow to run on pure Python

ComfyUI is awesome, and because the name indicates, it accommodates a UI. But Comfortable is way greater than a UI, it accommodates its own backend that runs on Python. As we don’t desire to make use of Comfortable’s node-based UI for the needs of this tutorial, we’d like to export the code to be run on pure python.

Thankfully, Peyton DeNiro has created this incredible ComfyUI-to-Python-Extension that exports any Comfortable workflow to a python script, enabling you to run a workflow without firing up the UI.

The simplest technique to install the extension is to (1) seek for ComfyUI to Python Extension within the Custom Nodes Manager Menu of the ComfyUI Manager extension and (2) install it. Then, for the choice to seem, you could have to (3) go on the settings on the underside right of the UI, (4) disable the brand new menu and (5) hit Save as Script. With that, you’ll find yourself with a Python script.

2. Create a Gradio app for the exported Python

Now that we’ve got our Python script, it’s time to create our Gradio app that can orchestrate it. Gradio is a python-native web-UI builder that permits us to create streamline applications. In case you do not have it already, you’ll be able to install it in your Python environment with pip install gradio

Next, we can have to re-arrange our python script a bit to create a UI for it.

Tip: LLMs like ChatGPT, Claude, Qwen, Gemini, LLama 3, etc. know create Gradio apps. Pasting your exported Python script to it and asking it to create a Gradio app should work on a basic level, but you’d probably must correct something with the knowledge you may get on this tutorial. For the aim of this tutorial, we’ll create the applying ourselves.

Open the exported Python script and add an import for Gradio

import os
import random
import sys
from typing import Sequence, Mapping, Any, Union
import torch
+ import gradio as gr

Now, we’d like to think about the UI- which parameters from the complex ComfyUI workflow do we would like to show in our UI? For the Flux[dev] Redux + Flux[dev] Depth ComfyUI workflow, I would love to show: the prompt, the structure image, the style image, the depth strength (for the structure) and the style strength.

For that, a minimal Gradio app can be:

if __name__ == "__main__":
    
    
    
    with gr.Blocks() as app:
        
        gr.Markdown("# FLUX Style Shaping")

        with gr.Row():
            with gr.Column():
                
                prompt_input = gr.Textbox(label="Prompt", placeholder="Enter your prompt here...")
                
                with gr.Row():
                    
                    with gr.Group():
                        structure_image = gr.Image(label="Structure Image", type="filepath")
                        depth_strength = gr.Slider(minimum=0, maximum=50, value=15, label="Depth Strength")
                    
                    with gr.Group():
                        style_image = gr.Image(label="Style Image", type="filepath")
                        style_strength = gr.Slider(minimum=0, maximum=1, value=0.5, label="Style Strength")
                
                
                generate_btn = gr.Button("Generate")
            
            with gr.Column():
                
                output_image = gr.Image(label="Generated Image")

            
            
            generate_btn.click(
                fn=generate_image,
                inputs=[prompt_input, structure_image, style_image, depth_strength, style_strength],
                outputs=[output_image]
            )
        app.launch(share=True)

That is how the app looks once it’s rendered

But should you attempt to run it, it won’t work yet, as now we’d like to establish this generate_image function by altering the def major() function of our exported python

script:

- def major():
+ def generate_image(prompt, structure_image, style_image, depth_strength, style_strength)

And contained in the function, we’d like to search out the hard coded values of the nodes we would like, and replace it with the variables we would love to regulate, equivalent to:

loadimage_429 = loadimage.load_image(
-    image="7038548d-d204-4810-bb74-d1dea277200a.png"
+    image=structure_image
)
# ...
loadimage_440 = loadimage.load_image(
-    image="2013_CKS_01180_0005_000(the_court_of_pir_budaq_shiraz_iran_circa_1455-60074106).jpg"
+    image=style_image
)
# ...
fluxguidance_430 = fluxguidance.append(
-   guidance=15,
+   guidance=depth_strength,
    conditioning=get_value_at_index(cliptextencode_174, 0)
)
# ...
stylemodelapplyadvanced_442 = stylemodelapplyadvanced.apply_stylemodel(
-   strength=0.5,
+   strength=style_strength,
    conditioning=get_value_at_index(instructpixtopixconditioning_431, 0),
    style_model=get_value_at_index(stylemodelloader_441, 0),
    clip_vision_output=get_value_at_index(clipvisionencode_439, 0),
)
# ...
cliptextencode_174 = cliptextencode.encode(
-   text="a woman taking a look at a house on fire",
+   text=prompt,   
    clip=get_value_at_index(cr_clip_input_switch_319, 0),
)

and for our output, we’d like to search out the save image output node, and export its path, equivalent to:

saveimage_327 = saveimage.save_images(
    filename_prefix=get_value_at_index(cr_text_456, 0),
    images=get_value_at_index(vaedecode_321, 0),
)
+ saved_path = f"output/{saveimage_327['ui']['images'][0]['filename']}"
+ return saved_path

Try a video rundown of those modifications:

Now, we ought to be able to run the code! Save your python file as app.py, add it to the foundation of your ComfyUI folder and run it as

python app.py

And identical to that, it’s best to find a way to run your Gradio app on http://0.0.0.0:7860

* Running on local URL:  http://127.0.0.1:7860
* Running on public URL: https://366fdd17b8a9072899.gradio.live

To debug this process, check here the diff between the unique python file exported by ComfyUI-to-Python-Extension and the Gradio app. You possibly can download each at that URL as well to envision and compare along with your own workflow.

That is it, congratulations! You managed to convert your ComfyUI workflow to a Gradio app. You possibly can run it locally and even send the URL to a customer or friend, nevertheless, should you turn off your computer or if 72h pass, the temporary Gradio link will die. For a persistent structure for hosting the app – including allowing people to run it without spending a dime in a serverless manner, you should use Hugging Face Spaces.

3. Preparing it to run Hugging Face Spaces

Now with our Gradio demo working, we may feel tempted to only upload every part to Hugging Face Spaces. Nonetheless, this might require uploading dozens of GB of models to Hugging Face, which isn’t only slow but additionally unnecessary, as all of those models exist already on Hugging Face!

As an alternative, we’ll first install pip install huggingface_hub if we do not have it already, after which we’d like to do the next on the highest of our app.py file:

from huggingface_hub import hf_hub_download

hf_hub_download(repo_id="black-forest-labs/FLUX.1-Redux-dev", filename="flux1-redux-dev.safetensors", local_dir="models/style_models")
hf_hub_download(repo_id="black-forest-labs/FLUX.1-Depth-dev", filename="flux1-depth-dev.safetensors", local_dir="models/diffusion_models")
hf_hub_download(repo_id="Comfortable-Org/sigclip_vision_384", filename="sigclip_vision_patch14_384.safetensors", local_dir="models/clip_vision")
hf_hub_download(repo_id="Kijai/DepthAnythingV2-safetensors", filename="depth_anything_v2_vitl_fp32.safetensors", local_dir="models/depthanything")
hf_hub_download(repo_id="black-forest-labs/FLUX.1-dev", filename="ae.safetensors", local_dir="models/vae/FLUX1")
hf_hub_download(repo_id="comfyanonymous/flux_text_encoders", filename="clip_l.safetensors", local_dir="models/text_encoders")
hf_hub_download(repo_id="comfyanonymous/flux_text_encoders", filename="t5xxl_fp16.safetensors", local_dir="models/text_encoders/t5")

It will map all local models on ComfyUI to their Hugging Face versions. Unfortunately, currently there is no such thing as a technique to automate this process, it’s essential find the models of your workflow on Hugging Face and map it to the identical ComfyUI folders that.

In case you are running models that should not on Hugging Face, it’s essential discover a technique to programmatically download them to the right folder via Python code. It will run just once when the Hugging Face Space starts.

Now, we’ll do one last modification to the app.py file, which is to incorporate the function decoration for ZeroGPU, which can allow us to do inference without spending a dime!

import gradio as gr
from huggingface_hub import hf_hub_download
+ import spaces
# ...
+ @spaces.GPU(duration=60) #modify the duration for the typical it takes on your worflow to run, in seconds
def generate_image(prompt, structure_image, style_image, depth_strength, style_strength):

Check here the diff from the previous Gradio demo with the Spaces prepared changes.

4. Exporting to Spaces and running on ZeroGPU

The code is prepared – you’ll be able to run it locally or in any cloud service of your preference – including a dedicated Hugging Face Spaces GPU. But to run it on a serverless ZeroGPU, follow along below.

Fix requirements

Firstly, it’s essential modify your requirements.txt to incorporate the necessities within the custom_nodes folder. As Hugging Face Spaces require a single requirements.txt file, ensure that so as to add the necessities of the nodes for this workflow to the requirements.txt on the foundation folder.

See the illustration below, the identical process must be repeated for all custom_nodes:

Now we’re ready!

1. Get to https://huggingface.co and create a brand new Space.
2. Set its hardware to ZeroGPU (should you are a Hugging Face PRO subscriber) or set it to CPU basic should you should not a PRO user (you will need an additional step at the top should you should not PRO).
   2.1 (In case you prefer a dedicated GPU that you simply pay for, pick L4, L40S, A100 as a substitute of ZeroGPU, that is a paid option)
3. Click the Files tab, Add File > Upload Files. Drag all of your ComfyUI folder files except the models folder (should you try and upload the models folder, your upload will fail), that is why we’d like part 3.
4. Click the Commit changes to major button on the underside of the page and wait for every part to upload
5. In case you are using gated models, like FLUX, it’s essential include a Hugging Face token to the settings. First, create a token with read access to all of the gated models you would like here, then go to the Settings page of your Space and create a brand new secret named HF_TOKEN with the worth of the token you could have just created.

Move models outside the decorated function (ZeroGPU only)

Your demo should already be working, nevertheless, in the present setup, the models will probably be fully loaded from disk to GPU each time you run it. To utilize the serverless ZeroGPU efficiency, we’ll must move all model declarations outside the decorated function to the worldwide context of Python. Let’s edit the app.py function to do this.

@@ -4,6 +4,7 @@
from typing import Sequence, Mapping, Any, Union
import torch
import gradio as gr
from huggingface_hub import hf_hub_download
+from cozy import model_management
import spaces

hf_hub_download(repo_id="black-forest-labs/FLUX.1-Redux-dev", filename="flux1-redux-dev.safetensors", local_dir="models/style_models")
@@ -109,6 +110,62 @@

from nodes import NODE_CLASS_MAPPINGS

+intconstant = NODE_CLASS_MAPPINGS["INTConstant"]()
+dualcliploader = NODE_CLASS_MAPPINGS["DualCLIPLoader"]()
+dualcliploader_357 = dualcliploader.load_clip(
+    clip_name1="t5/t5xxl_fp16.safetensors",
+    clip_name2="clip_l.safetensors",
+    type="flux",
+)
+cr_clip_input_switch = NODE_CLASS_MAPPINGS["CR Clip Input Switch"]()
+cliptextencode = NODE_CLASS_MAPPINGS["CLIPTextEncode"]()
+loadimage = NODE_CLASS_MAPPINGS["LoadImage"]()
+imageresize = NODE_CLASS_MAPPINGS["ImageResize+"]()
+getimagesizeandcount = NODE_CLASS_MAPPINGS["GetImageSizeAndCount"]()
+vaeloader = NODE_CLASS_MAPPINGS["VAELoader"]()
+vaeloader_359 = vaeloader.load_vae(vae_name="FLUX1/ae.safetensors")
+vaeencode = NODE_CLASS_MAPPINGS["VAEEncode"]()
+unetloader = NODE_CLASS_MAPPINGS["UNETLoader"]()
+unetloader_358 = unetloader.load_unet(
+    unet_name="flux1-depth-dev.safetensors", weight_dtype="default"
+)
+ksamplerselect = NODE_CLASS_MAPPINGS["KSamplerSelect"]()
+randomnoise = NODE_CLASS_MAPPINGS["RandomNoise"]()
+fluxguidance = NODE_CLASS_MAPPINGS["FluxGuidance"]()
+depthanything_v2 = NODE_CLASS_MAPPINGS["DepthAnything_V2"]()
+downloadandloaddepthanythingv2model = NODE_CLASS_MAPPINGS[
+    "DownloadAndLoadDepthAnythingV2Model"
+]()
+downloadandloaddepthanythingv2model_437 = (
+    downloadandloaddepthanythingv2model.loadmodel(
+        model="depth_anything_v2_vitl_fp32.safetensors"
+    )
+)
+instructpixtopixconditioning = NODE_CLASS_MAPPINGS[
+    "InstructPixToPixConditioning"
+]()
+text_multiline_454 = text_multiline.text_multiline(text="FLUX_Redux")
+clipvisionloader = NODE_CLASS_MAPPINGS["CLIPVisionLoader"]()
+clipvisionloader_438 = clipvisionloader.load_clip(
+    clip_name="sigclip_vision_patch14_384.safetensors"
+)
+clipvisionencode = NODE_CLASS_MAPPINGS["CLIPVisionEncode"]()
+stylemodelloader = NODE_CLASS_MAPPINGS["StyleModelLoader"]()
+stylemodelloader_441 = stylemodelloader.load_style_model(
+    style_model_name="flux1-redux-dev.safetensors"
+)
+text_multiline = NODE_CLASS_MAPPINGS["Text Multiline"]()
+emptylatentimage = NODE_CLASS_MAPPINGS["EmptyLatentImage"]()
+cr_conditioning_input_switch = NODE_CLASS_MAPPINGS[
+    "CR Conditioning Input Switch"
+]()
+cr_model_input_switch = NODE_CLASS_MAPPINGS["CR Model Input Switch"]()
+stylemodelapplyadvanced = NODE_CLASS_MAPPINGS["StyleModelApplyAdvanced"]()
+basicguider = NODE_CLASS_MAPPINGS["BasicGuider"]()
+basicscheduler = NODE_CLASS_MAPPINGS["BasicScheduler"]()
+samplercustomadvanced = NODE_CLASS_MAPPINGS["SamplerCustomAdvanced"]()
+vaedecode = NODE_CLASS_MAPPINGS["VAEDecode"]()
+saveimage = NODE_CLASS_MAPPINGS["SaveImage"]()
+imagecrop = NODE_CLASS_MAPPINGS["ImageCrop+"]()

@@ -117,75 +174,6 @@
def generate_image(prompt, structure_image, style_image, depth_strength, style_strength):
    import_custom_nodes()
    with torch.inference_mode():
-        intconstant = NODE_CLASS_MAPPINGS["INTConstant"]()
         intconstant_83 = intconstant.get_value(value=1024)

         intconstant_84 = intconstant.get_value(value=1024)

-        dualcliploader = NODE_CLASS_MAPPINGS["DualCLIPLoader"]()
-        dualcliploader_357 = dualcliploader.load_clip(
-            clip_name1="t5/t5xxl_fp16.safetensors",
-            clip_name2="clip_l.safetensors",
-            type="flux",
-        )
-
-        cr_clip_input_switch = NODE_CLASS_MAPPINGS["CR Clip Input Switch"]()
         cr_clip_input_switch_319 = cr_clip_input_switch.switch(
             Input=1,
             clip1=get_value_at_index(dualcliploader_357, 0),
             clip2=get_value_at_index(dualcliploader_357, 0),
         )

-        cliptextencode = NODE_CLASS_MAPPINGS["CLIPTextEncode"]()
         cliptextencode_174 = cliptextencode.encode(
             text=prompt,
             clip=get_value_at_index(cr_clip_input_switch_319, 0),
         )

         cliptextencode_175 = cliptextencode.encode(
             text="purple", clip=get_value_at_index(cr_clip_input_switch_319, 0)
         )

-        loadimage = NODE_CLASS_MAPPINGS["LoadImage"]()
         loadimage_429 = loadimage.load_image(image=structure_image)

-        imageresize = NODE_CLASS_MAPPINGS["ImageResize+"]()
         imageresize_72 = imageresize.execute(
             width=get_value_at_index(intconstant_83, 0),
             height=get_value_at_index(intconstant_84, 0),
             interpolation="bicubic",
             method="keep proportion",
             condition="at all times",
             multiple_of=16,
             image=get_value_at_index(loadimage_429, 0),
         )

-        getimagesizeandcount = NODE_CLASS_MAPPINGS["GetImageSizeAndCount"]()
         getimagesizeandcount_360 = getimagesizeandcount.getsize(
             image=get_value_at_index(imageresize_72, 0)
         )

-        vaeloader = NODE_CLASS_MAPPINGS["VAELoader"]()
-        vaeloader_359 = vaeloader.load_vae(vae_name="FLUX1/ae.safetensors")

-        vaeencode = NODE_CLASS_MAPPINGS["VAEEncode"]()
         vaeencode_197 = vaeencode.encode(
             pixels=get_value_at_index(getimagesizeandcount_360, 0),
             vae=get_value_at_index(vaeloader_359, 0),
         )

-        unetloader = NODE_CLASS_MAPPINGS["UNETLoader"]()
-        unetloader_358 = unetloader.load_unet(
-            unet_name="flux1-depth-dev.safetensors", weight_dtype="default"
-        )

-        ksamplerselect = NODE_CLASS_MAPPINGS["KSamplerSelect"]()
         ksamplerselect_363 = ksamplerselect.get_sampler(sampler_name="euler")

-        randomnoise = NODE_CLASS_MAPPINGS["RandomNoise"]()
         randomnoise_365 = randomnoise.get_noise(noise_seed=random.randint(1, 2**64))

-        fluxguidance = NODE_CLASS_MAPPINGS["FluxGuidance"]()
         fluxguidance_430 = fluxguidance.append(
             guidance=15, conditioning=get_value_at_index(cliptextencode_174, 0)
         )

-        downloadandloaddepthanythingv2model = NODE_CLASS_MAPPINGS[
-            "DownloadAndLoadDepthAnythingV2Model"
-        ]()
-        downloadandloaddepthanythingv2model_437 = (
-            downloadandloaddepthanythingv2model.loadmodel(
-                model="depth_anything_v2_vitl_fp32.safetensors"
-            )
-        )

-        depthanything_v2 = NODE_CLASS_MAPPINGS["DepthAnything_V2"]()
         depthanything_v2_436 = depthanything_v2.process(
             da_model=get_value_at_index(downloadandloaddepthanythingv2model_437, 0),
             images=get_value_at_index(getimagesizeandcount_360, 0),
         )

-        instructpixtopixconditioning = NODE_CLASS_MAPPINGS[
-            "InstructPixToPixConditioning"
-        ]()
         instructpixtopixconditioning_431 = instructpixtopixconditioning.encode(
             positive=get_value_at_index(fluxguidance_430, 0),
             negative=get_value_at_index(cliptextencode_175, 0),
             vae=get_value_at_index(vaeloader_359, 0),
             pixels=get_value_at_index(depthanything_v2_436, 0),
         )

-        clipvisionloader = NODE_CLASS_MAPPINGS["CLIPVisionLoader"]()
-        clipvisionloader_438 = clipvisionloader.load_clip(
-            clip_name="sigclip_vision_patch14_384.safetensors"
-        )

         loadimage_440 = loadimage.load_image(image=style_image)

-        clipvisionencode = NODE_CLASS_MAPPINGS["CLIPVisionEncode"]()
         clipvisionencode_439 = clipvisionencode.encode(
             crop="center",
             clip_vision=get_value_at_index(clipvisionloader_438, 0),
             image=get_value_at_index(loadimage_440, 0),
         )

-        stylemodelloader = NODE_CLASS_MAPPINGS["StyleModelLoader"]()
-        stylemodelloader_441 = stylemodelloader.load_style_model(
-            style_model_name="flux1-redux-dev.safetensors"
-        )
-
-        text_multiline = NODE_CLASS_MAPPINGS["Text Multiline"]()
         text_multiline_454 = text_multiline.text_multiline(text="FLUX_Redux")

-        emptylatentimage = NODE_CLASS_MAPPINGS["EmptyLatentImage"]()
-        cr_conditioning_input_switch = NODE_CLASS_MAPPINGS[
-            "CR Conditioning Input Switch"
-        ]()
-        cr_model_input_switch = NODE_CLASS_MAPPINGS["CR Model Input Switch"]()
-        stylemodelapplyadvanced = NODE_CLASS_MAPPINGS["StyleModelApplyAdvanced"]()
-        basicguider = NODE_CLASS_MAPPINGS["BasicGuider"]()
-        basicscheduler = NODE_CLASS_MAPPINGS["BasicScheduler"]()
-        samplercustomadvanced = NODE_CLASS_MAPPINGS["SamplerCustomAdvanced"]()
-        vaedecode = NODE_CLASS_MAPPINGS["VAEDecode"]()
-        saveimage = NODE_CLASS_MAPPINGS["SaveImage"]()
-        imagecrop = NODE_CLASS_MAPPINGS["ImageCrop+"]()

         emptylatentimage_10 = emptylatentimage.generate(
             width=get_value_at_index(imageresize_72, 1),
             height=get_value_at_index(imageresize_72, 2),
             batch_size=1,
         )

Moreover, with the intention to pre-load the models we’d like to make use of the ComfyUI load_models_gpu function, which can include, from the above pre-loaded model, all of the models that were loaded ( rule of thumb, is checking which from the above load a *.safetensors file)

from cozy import model_management


model_loaders = [dualcliploader_357, vaeloader_359, unetloader_358, clipvisionloader_438, stylemodelloader_441, downloadandloaddepthanythingv2model_437]


valid_models = [
    getattr(loader[0], 'patcher', loader[0]) 
    for loader in model_loaders
    if not isinstance(loader[0], dict) and not isinstance(getattr(loader[0], 'patcher', None), dict)
]


model_management.load_models_gpu(valid_models)

Check the diff to grasp precisely what changes

In case you should not a PRO subscriber (skip this step should you are)

In case you are not a Hugging Face PRO subscriber, it’s essential apply for a ZeroGPU grant. You possibly can accomplish that easily by happening the Settings page of your Space and submitting a grant request for ZeroGPU. All ZeroGPU grant requests for Spaces with ComfyUI backends will probably be granted 🎉.

The demo is running

The demo we’ve got built with this tutorial is survive Hugging Face Spaces. Come play with it here: https://huggingface.co/spaces/multimodalart/flux-style-shaping

5. Conclusion

😮‍💨, that is all! I realize it is a little bit of work, however the reward is a simple technique to share your workflow with an easy UI and free inference to everyone! As mentioned before, the goal is to automate and streamline this process as much as possible in early 2025! Comfortable holidays 🎅✨