Paying $0.03–$0.05 per generation on hosted image APIs scales your cloud bill linearly with user growth. Generating 50,000 images a month on managed inference platforms costs up to $2,500. Hosting ComfyUI on a single-tenant bare-metal GPU server—such as an RTX 4090 or RTX A6000 on BytesRack—keeps your infrastructure cost flat at roughly $350/month with zero third-party rate limits.
In this guide, you will deploy ComfyUI headlessly on Ubuntu 24.04, wrap it in an asynchronous FastAPI + WebSocket gateway, and expose a production /v1/generate endpoint secured behind Nginx and Let's Encrypt SSL.
Tutorial Navigation
Prerequisites & Hardware Specs
Step 1: Install ComfyUI & Download Model Weights
Step 2: Run ComfyUI Headless via Systemd
Step 3: Export Your Workflow as workflow_api.json
Step 4: Build the FastAPI + WebSocket Wrapper
Step 5: Deploy with Systemd, Nginx & SSL
Step 6: Test Your Live Endpoint
Frequently Asked Questions & Troubleshooting
Prerequisites & Hardware Specs
Size your bare-metal server around your heaviest production workflow rather than your average test prompt. Model swapping between system RAM and GPU memory introduces multi-second latency spikes.
-
GPU: NVIDIA RTX 3090 or RTX 4090 (24 GB VRAM) for SDXL and Flux.1 Schnell (
fp8); NVIDIA RTX A6000 or L40S (48 GB VRAM) to keep Flux.1fp16and multiple ControlNets resident simultaneously. -
System Memory & Storage: 64 GB RAM minimum (prevents Linux disk swapping during weight staging) and 500 GB NVMe SSD (a single Flux checkpoint exceeds 20 GB, and fast NVMe reads cut cold-start latency).
-
Software Stack: Ubuntu 24.04 LTS, NVIDIA Driver 550+, CUDA 12.4+, Python 3.11+, FastAPI, Uvicorn, and Nginx.
Step 1: Install ComfyUI & Download Model Weights
SSH into your fresh Ubuntu 24.04 server, update system packages, and verify your GPU driver status (nvidia-smi should confirm Driver 550 or higher):
sudo apt update && sudo apt upgrade -y
sudo apt install -y git python3-venv python3-pip build-essential curl
nvidia-smi
Create an isolated, unprivileged comfy system user so third-party custom nodes cannot access root files or environment secrets. Clone ComfyUI and install PyTorch compiled for CUDA 12.4:
sudo useradd -m -d /opt/comfyui -s /bin/bash comfy
sudo -iu comfy
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
python3 -m venv venv && source venv/bin/activate
pip install --upgrade pip
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
Confirm PyTorch detects your CUDA device (True followed by your GPU name):
python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
Pull Flux.1 Schnell (fp8)—which carries an Apache 2.0 license suitable for commercial SaaS APIs—directly onto your server's NVMe storage at datacenter speeds:
pip install -U "huggingface_hub[cli]"
huggingface-cli download Comfy-Org/flux1-schnell flux1-schnell-fp8.safetensors --local-dir models/checkpoints
(Note: FLUX.1 [dev] carries a non-commercial license. Verify model licensing before serving paid API traffic.)
Step 2: Run ComfyUI Headless via Systemd
Never expose ComfyUI's native port (8188) to the public internet—it has no authentication and can read or write local files. Bind it strictly to 127.0.0.1 as a systemd service so it recovers automatically after reboots or out-of-memory kills.
Switch back to your sudo user and create /etc/systemd/system/comfyui.service:
[Unit]
Description=ComfyUI Headless Inference Engine
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=comfy
Group=comfy
WorkingDirectory=/opt/comfyui/ComfyUI
ExecStart=/opt/comfyui/ComfyUI/venv/bin/python main.py --listen 127.0.0.1 --port 8188 --disable-auto-launch
Environment=PYTHONUNBUFFERED=1
Restart=always
RestartSec=5
LimitNOFILE=65535
[Install]
WantedBy=multi-user.target
Enable the service and verify the internal stats endpoint responds:
sudo systemctl daemon-reload
sudo systemctl enable --now comfyui
curl -s http://127.0.0.1:8188/system_stats | head -c 200
Step 3: Export Your Workflow as workflow_api.json
ComfyUI's /prompt endpoint rejects standard UI save files. You must export the stripped-down API JSON format, which maps node IDs directly to class_type and inputs.
-
Forward port
8188locally via SSH so the server port stays private:bashssh -L 8188:localhost:8188 user@your_bytesrack_server_ip -
Open
http://localhost:8188in your local browser and verify your graph generates clean outputs. -
Open Settings (gear icon) → toggle Enable Dev mode Options → click Save (API Format) (or Export (API) under the Workflow menu).
-
Inspect the exported
workflow_api.jsonand note the exact Node IDs for your positive prompt (CLIPTextEncode), negative prompt (CLIPTextEncode), dimensions (EmptyLatentImageorEmptySD3LatentImage), seed (KSamplerorRandomNoise), and output (SaveImage).
Step 4: Build the FastAPI + WebSocket Wrapper
Repeatedly polling /history/{prompt_id} over HTTP wastes CPU cycles and adds latency under concurrent load. Instead, our FastAPI gateway opens a WebSocket (ws://127.0.0.1:8188/ws) before posting to /prompt, listening for the exact executing event where node is null.
Set up an isolated virtual environment for the API layer:
import asyncio
import copy
import json
import os
import random
import secrets
import uuid
from contextlib import asynccontextmanager
from pathlib import Path
import httpx
import websockets
from fastapi import Depends, FastAPI, Header, HTTPException, Response
from pydantic import BaseModel, Field
COMFY_HTTP = "http://127.0.0.1:8188"
COMFY_WS = "ws://127.0.0.1:8188/ws"
MAX_QUEUE = int(os.getenv("MAX_QUEUE_DEPTH", "8"))
TIMEOUT = int(os.getenv("GENERATION_TIMEOUT", "240"))
API_KEYS = [k.strip() for k in os.environ.get("API_KEYS", "").split(",") if k.strip()]
# Map these IDs to your exported workflow_api.json
POSITIVE_NODE, NEGATIVE_NODE, LATENT_NODE, SAMPLER_NODE, OUTPUT_NODE = "6", "7", "5", "3", "9"
BASE_WORKFLOW = json.loads((Path(__file__).parent / "workflow_api.json").read_text())
@asynccontextmanager
async def lifespan(app: FastAPI):
app.state.http = httpx.AsyncClient(timeout=30.0)
yield
await app.state.http.aclose()
app = FastAPI(title="Self-Hosted ComfyUI API", lifespan=lifespan)
async def require_api_key(x_api_key: str = Header(default="")):
if not any(secrets.compare_digest(x_api_key.encode(), k.encode()) for k in API_KEYS):
raise HTTPException(401, "Invalid or missing API key")
class GenerateRequest(BaseModel):
prompt: str = Field(min_length=1, max_length=2000)
negative_prompt: str = Field(default="", max_length=2000)
width: int = Field(default=1024, ge=256, le=2048, multiple_of=8)
height: int = Field(default=1024, ge=256, le=2048, multiple_of=8)
seed: int | None = Field(default=None, ge=0, le=2**63 - 1)
async def run_workflow(client: httpx.AsyncClient, workflow: dict) -> str:
client_id = str(uuid.uuid4())
# Open WebSocket BEFORE submitting prompt to prevent race conditions
async with websockets.connect(f"{COMFY_WS}?clientId={client_id}", max_size=None) as ws:
resp = await client.post(
f"{COMFY_HTTP}/prompt", json={"prompt": workflow, "client_id": client_id}
)
if resp.status_code != 200:
raise HTTPException(502, "ComfyUI rejected workflow")
prompt_id = resp.json()["prompt_id"]
while True:
msg = await ws.recv()
if isinstance(msg, bytes):
continue
event = json.loads(msg)
data = event.get("data", {})
if data.get("prompt_id") == prompt_id:
if event["type"] == "execution_error":
raise HTTPException(500, "ComfyUI graph execution failed")
if event["type"] == "executing" and data.get("node") is None:
return prompt_id
@app.post("/v1/generate", dependencies=[Depends(require_api_key)], response_class=Response)
async def generate(req: GenerateRequest):
client = app.state.http
q = (await client.get(f"{COMFY_HTTP}/queue")).json()
if len(q["queue_running"]) + len(q["queue_pending"]) >= MAX_QUEUE:
raise HTTPException(429, "GPU queue full", headers={"Retry-After": "10"})
wf = copy.deepcopy(BASE_WORKFLOW)
seed = req.seed if req.seed is not None else random.randint(0, 2**63 - 1)
wf[POSITIVE_NODE]["inputs"]["text"] = req.prompt
wf[NEGATIVE_NODE]["inputs"]["text"] = req.negative_prompt
wf[LATENT_NODE]["inputs"]["width"] = req.width
wf[LATENT_NODE]["inputs"]["height"] = req.height
wf[SAMPLER_NODE]["inputs"]["seed"] = seed
try:
prompt_id = await asyncio.wait_for(run_workflow(client, wf), TIMEOUT)
except asyncio.TimeoutError:
raise HTTPException(504, "Generation timed out")
hist = (await client.get(f"{COMFY_HTTP}/history/{prompt_id}")).json()
img_info = hist[prompt_id]["outputs"][OUTPUT_NODE]["images"][0]
img = await client.get(f"{COMFY_HTTP}/view", params=img_info)
return Response(content=img.content, media_type="image/png", headers={"X-Seed": str(seed)})
Production Delivery Tip: Returning raw PNG bytes works well for internal microservices. For public customer-facing APIs, upload img.content to Cloudflare R2 or AWS S3 using boto3 and return a JSON CDN URL instead of raw binary or base64 strings.
Step 5: Deploy with Systemd, Nginx & SSL
Generate a secure API key and store it outside your codebase in /etc/comfy-api.env:
echo "API_KEYS=$(openssl rand -hex 32)" | sudo tee /etc/comfy-api.env
sudo chmod 600 /etc/comfy-api.env
Create /etc/systemd/system/comfy-api.service (keep Uvicorn set to 1 worker, since ComfyUI serializes execution on a single GPU):
[Unit]
Description=ComfyUI FastAPI Gateway
After=network-online.target comfyui.service
Wants=comfyui.service
[Service]
User=comfy
Group=comfy
WorkingDirectory=/opt/comfy-api
EnvironmentFile=/etc/comfy-api.env
ExecStart=/opt/comfy-api/venv/bin/uvicorn main:app --host 127.0.0.1 --port 8000
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
Enable the FastAPI service:
sudo chown -R comfy:comfy /opt/comfy-api
sudo systemctl daemon-reload && sudo systemctl enable --now comfy-api
Configure Nginx Reverse Proxy & Let's Encrypt SSL
Point an A record for api.yourdomain.com to your server's IP, install Nginx and Certbot, and create /etc/nginx/sites-available/comfy-api. Explicitly set proxy_read_timeout 300s; so Nginx's default 60-second timeout does not drop queued or high-resolution generations:
server {
listen 80;
server_name api.yourdomain.com;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_read_timeout 300s;
proxy_send_timeout 300s;
}
}
Activate SSL and lock down UFW firewall rules so ports 8000 and 8188 stay closed externally:
sudo apt install -y nginx certbot python3-certbot-nginx
sudo ln -s /etc/nginx/sites-available/comfy-api /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
sudo certbot --nginx -d api.yourdomain.com
sudo ufw allow OpenSSH && sudo ufw allow 'Nginx Full' && sudo ufw enable
Step 6: Test Your Live Endpoint
Verify authentication, seed reproducibility, and end-to-end latency with cURL:
curl -X POST https://api.yourdomain.com/v1/generate \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_API_KEY" \
-d '{"prompt": "a lighthouse at dusk, volumetric light, 35mm photo", "width": 1024, "height": 1024}' \
-o output.png -w "HTTP %{http_code} in %{time_total}s\n"
On a BytesRack Dedicated RTX 4090 Server (24 GB VRAM, CUDA 12.4), a 1024×1024 4-step Flux.1 Schnell generation finishes in ~2.3 seconds end-to-end once weights are resident in VRAM.
Frequently Asked Questions & Production Troubleshooting
-
Can I expose ComfyUI's built-in API directly without FastAPI?
-
How do I fix CUDA Out of Memory (OOM) errors under load?
-
Why does Nginx return 504 Gateway Time-out on complex workflows?
-
Why do fast API requests occasionally hang until timeout?
-
Why is my NVMe disk filling up over time?
-
How do I scale ComfyUI API across multiple GPUs?
Discover BytesRack Dedicated Server Locations
BytesRack servers are available around the world, providing diverse options for hosting high-performance AI workloads. Each region offers unique advantages, making it easier to choose a location that best suits your specific latency needs.

Media Stream Solutions
Gaming Solutions
E-Commerce Solutions
VPN Server Solutions
GPU Server Solutions
Financial Solutions
Security Solutions