An AI Shopping Assistant with Gradio

Python Developers, want to offer your LLM superpowers? Gradio is the fastest option to do it! With Gradio’s Model Context Protocol (MCP) integration, your LLM can plug directly into the hundreds of AI models and Spaces hosted on the Hugging Face Hub. By pairing the final reasoning capabilities of LLMs with the specialized abilities of models found on Hugging Face, your LLM can transcend simply answering text questions to really solving problems in your each day life.

For Python developers, Gradio makes implementing powerful MCP servers a breeze, offering features like:

• Automatic conversion of python functions into LLM tools: Each API endpoint in your Gradio app is mechanically converted into an MCP tool with a corresponding name, description, and input schema. The docstring of your function is used to generate the outline of the tool and its parameters.
• Real-time progress notifications: Gradio streams progress notifications to your MCP client, allowing you to watch the status in real-time without having to implement this feature yourself.
• Automatic file uploads, including support for public URLs and handling of assorted file types.

Imagine this: you hate shopping since it takes an excessive amount of time, and also you dread trying on clothes yourself. What if an LLM could handle this for you? On this post, we’ll create an LLM-powered AI assistant that may browse online outfitters, find specific garments, after which use a virtual try-on model to indicate you the way those clothes would look on you. See the demo below:

The Goal: Your Personal AI Stylist

To bring our AI shopping assistant to life, we’ll mix three key components:

1. IDM-VTON Diffusion Model: This AI model is chargeable for the virtual try-on functionality. It could possibly edit existing photos to make it appear as if an individual is wearing a unique garment. We’ll be using the Hugging Face Space for IDM-VTON, accessible here.

2. Gradio: Gradio is an open-source Python library that makes it easy to construct AI-powered web applications and, crucially for our project, to create MCP servers. Gradio will act because the bridge, allowing our LLM to call the IDM-VTON model and other tools.

3. Visual Studio Code’s AI Chat Feature: We’ll use VS Code’s built-in AI chat, which supports adding arbitrary MCP servers, to interact with our AI shopping assistant. This may provide a user-friendly interface for issuing commands and viewing the virtual try-on results.

Constructing the Gradio MCP Server

The core of our AI shopping assistant is the Gradio MCP server. This server will expose one primary tool:

1. vton_generation: This function will take a human model image and a garment image as input and use the IDM-VTON model to generate a brand new image of the person wearing the garment.

Here’s the Python code for our Gradio MCP server:

from gradio_client import Client, handle_file
import gradio as gr
import re


client = Client("freddyaboulton/IDM-VTON",
                hf_token="")


def vton_generation(human_model_img: str, garment: str):
    """Use the IDM-VTON model to generate a brand new image of an individual wearing a garment."""
    """
    Args:
        human_model_img: The human model that's modelling the garment.
        garment: The garment to wear.
    """
    output = client.predict(
        dict={"background": handle_file(human_model_img), "layers":[], "composite":None},
        garm_img=handle_file(garment),
        garment_des="",
        is_checked=True,
        is_checked_crop=False,
        denoise_steps=30,
        seed=42,
        api_name="/tryon"
    )

    return output[0]

vton_mcp = gr.Interface(
    vton_generation,
    inputs=[
        gr.Image(type="filepath", label="Human Model Image URL"),
        gr.Image(type="filepath", label="Garment Image URL or File")
    ],
    outputs=gr.Image(type="filepath", label="Generated Image")
)

if __name__ == "__main__":
    vton_mcp.launch(mcp_server=True)

By setting mcp_server=True within the launch() method, Gradio mechanically converts our Python functions into MCP tools that LLMs can understand and use. The docstrings of our functions are used to generate descriptions of the tools and their parameters.

The unique IDM-VTON space was implemented with Gradio 4.x which precedes the automated MCP functionality. So on this demo, we’ll be constructing a Gradio interface that queries the unique space via the Gradio API client.

Finally, run this script with python.

Configuring VS Code

To attach our Gradio MCP server to VS Code’s AI chat, we’ll must edit the mcp.json file. This configuration tells the AI chat where to search out our MCP server and the way to interact with it.

You will discover this file by typing MCP within the command panel and choosing MCP: Open User Configuration. When you open it, be certain the next servers are present:

{
  "servers": {
  "vton": {
    "url": "http://127.0.0.1:7860/gradio_api/mcp/"
  },
  "playwright": {
    "command": "npx",
    "args": [
      "-y",
      "@playwright/mcp@latest"
    ]
   }
  }
}

The playwright MCP server will let our AI assistant browse the net.

Ensure the URL of the vton server matches the url printed to the console within the previous section. To run the playwright MCP server, it’s worthwhile to have node installed.

Putting It All Together

Now we will start interacting with our AI shopping assistant. Open a brand new chat in VS Code, and you may ask the assistant something like “Browse the Uniqlo website for blue t-shirts, and show me what I might appear to be in three of them, using my photo at [your-image-url].”

See the above video for an example!

Conclusion

The mixture of Gradio, MCP, and powerful AI models like IDM-VTON opens up exciting possibilities for creating intelligent and helpful AI assistants. By following the steps outlined on this blog post, you may construct your personal assistant to resolve the issues you care most about!