ComfyUI Headless Image Generation (Qwen-Image 2.1 game assets)

Drive the locally installed ComfyUI without its UI: start a headless server, submit an API-format workflow via POST /prompt, poll /history, download results via /view. All commands below run the bundled script scripts/comfy21.py (stdlib only, no deps). Two verified workflow templates live in workflows/ for manual API use. The official Qwen-Image 2.1 prompt-writing guide (model card + ComfyUI docs) is distilled in references/prompt-guide.md — consult it when writing or fixing prompts.

1. Path discovery — NEVER hardcode install or model paths

This machine's ComfyUI layout must be discovered at runtime, in this order:

1. COMFYUI_HOME env var, if set (folder containing ComfyUI/main.py or main.py).
2. %APPDATA%\Comfy Desktop\installations.json — the Desktop app's install registry; each entry's installPath contains the checkout and a bundled .venv python.
3. Shallow scan of every existing drive root and common program directories for directories matching *omfy* (matched folders are scanned one level deeper). No machine-specific paths are assumed anywhere.

Model registration locations (where weights are actually looked up) are never assumed either:

• %APPDATA%\Comfy Desktop\settings.json → modelsDirs (Desktop shared model root).
• %APPDATA%\Comfy Desktop\shared_model_paths.yaml — auto-generated by the Desktop app; registering a model root here is what makes it visible to a Desktop-launched server.
• <code_root>\extra_model_paths.yaml — ComfyUI core reads this file at every startup (upstream mechanism, gitignored). User-registered roots (e.g. a dedicated Qwen-Image-2.1 folder) typically live here, NOT in the Desktop registry.
• When the server is running, GET /models/<folder> is the ground truth of what is registered. python scripts/comfy21.py models prints it.

A headless server does not see the Desktop registries by itself — the script passes --extra-model-paths-config <shared_model_paths.yaml> and --input-directory/--output-directory (derived from the registered models dir) at launch.

2. Weight check and download (Qwen-Image 2.1)

Check what exists before generating:

python scripts\comfy21.py find

Required files (either the int8_convrot quantized set or the bf16 set works; int8 is lighter; weights larger than VRAM still run via ComfyUI's dynamic-VRAM streaming, at a speed cost):

Type	File (pick one per row)	Registered under
diffusion_models	qwen_image_2.1_int8_convrot.safetensors or qwen_image_2.1_bf16.safetensors	diffusion_models/
text_encoders	qwen3vl_8b_int8_convrot.safetensors or qwen3vl_8b_bf16.safetensors	text_encoders/
vae	qwen_image_2.1_vae_bf16.safetensors	vae/

"Registered under" means: inside a directory that one of the registration files above maps to that type — find it with find, do not guess a drive path.

If weights are missing, download from the original HF repo or the mirror, then place each file into the registered dir find reported:

:: original
curl -L -o <registered_dir>\qwen_image_2.1_int8_convrot.safetensors ^
  https://huggingface.co/Comfy-Org/Qwen-Image-2.1/resolve/main/split_files/diffusion_models/qwen_image_2.1_int8_convrot.safetensors

:: mirror (hf-mirror.com) — same URL with the host swapped, or via huggingface-cli:
set HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download Comfy-Org/Qwen-Image-2.1 split_files/text_encoders/qwen3vl_8b_int8_convrot.safetensors --local-dir .

Repo: https://huggingface.co/Comfy-Org/Qwen-Image-2.1 (mirror: https://hf-mirror.com/Comfy-Org/Qwen-Image-2.1), files under split_files/diffusion_models|text_encoders|vae/. Restart the server after adding weights? Not needed for weights placed before launch; a running server lists model files dynamically, but a restart is the safe fallback.

3. Quickstart

python scripts\comfy21.py find                 :: discover + check weights (run this first)
python scripts\comfy21.py serve                :: start headless server on 127.0.0.1:8188 (idempotent)
python scripts\comfy21.py models               :: list registered model files
python scripts\comfy21.py t2i --prompt "a fire sword icon, game item art" --transparent --out assets
python scripts\comfy21.py edit --image .\hero.png --out assets      :: default = background removal
python scripts\comfy21.py edit --image .\hero.png --instruction "Make the sword glow blue" --out assets
python scripts\comfy21.py stop

Outputs land in --out (default ./generated). If the Desktop app is already running, use --port 8189 — two servers cannot share a port.

4. Text-to-image (文生图)

Workflow graph (see workflows/qwen21_t2i_api.json): UNETLoader → TextEncodeQwenImage21(clip, prompt, negative_prompt, resolution) → EmptyLatentImage → KSampler(euler/simple, cfg 1.0, 25 steps, denoise 1.0) → VAEDecode → SaveImage.

• CLIPLoader must use type: qwen_image (not lumina2/other).
• resolution on the encoder node is a total pixel budget (native 1024, up to 2048); aspect ratio preserved. Keep width/height multiples of 32.
• cfg stays 1.0 on the official path — negative_prompt is unused at cfg 1. The CLI exposes the knobs anyway and warns when you pass --negative without --cfg: --negative "blurry, text" --cfg 2.0 activates it (cfg 2 also tracks dense prompts / small text better, per the official tutorial). --sampler / --scheduler default to euler/simple like every official template.
• Official pipeline uses ~40–50 steps with euler; 25 is the template default and is fine for drafts; raise for final quality.

Batch generation (multiple images at once)

Two verified paths — pick by what varies:

• Same prompt, N variations → --n N puts N images in a single run (EmptyLatentImage.batch_size = N; seeds run seed, seed+1, …; the model loads once, so this is the fastest path). Verified: --n 3 at 512px/8 steps → 3 files in one submission. python scripts\comfy21.py t2i --prompt "..." --n 8 --transparent --out assets
• Different prompts → --prompts-file: prompts are separated by blank lines (a prompt may span multiple lines — they are joined with spaces), or use a .json file with an array of strings / {"prompts": [...]} (internal newlines preserved). # comment lines are skipped. Every prompt becomes a queued job; ALL jobs are submitted first so the server keeps the GPU busy, then results are collected one by one. Verified: 2-line block + single line → 2 jobs (not 3).

  python scripts\comfy21.py t2i --prompts-file prompts.txt --transparent --out assets
  

There is no client-side parallelism — ComfyUI serializes on one GPU; the win is that you never wait on submission, and one batch run amortizes model load across N images.

5. Reference-guided generation (参考生图)

Workflow graph (see workflows/qwen21_edit_api.json): same loaders, plus LoadImage, and the reference feeds TextEncodeQwenImage21 together with the VAE; the KSampler's latent_image is the encoder node's third output (["5", 2]) so the canvas follows the reference size.

Three hard rules (each was a real failure during verification):

1. Autogrow images use dotted keys in API JSON: "images.image_1": ["<load_node>", 0]. A flat "image_1" key raises TypeError: unexpected keyword argument; a nested "images": {"image_1": ...} dict is silently dropped — the prompt still executes and the model just ignores your reference. If an edit "ignores the image", check this first.
2. The instruction prompt is an edit directive. Multi-image edits reference slots in text as <image1> … <image10> (e.g. "Put the hat from <image2> on the dog in <image1>").
3. The reference image must sit in the server's registered input directory — the script copies it there automatically; LoadImage only sees that folder.

6. Transparent-background assets (背景透明)

Two verified methods:

A. Straight from text-to-image (preferred, single step). The model card's official RGBA template (verified: alpha min 0, ~50% transparent pixels):

"This is an RGBA image with transparency. A cute cartoon dragon sticker. The image has alpha channel and the background is transparent."

t2i --transparent wraps your prompt in exactly this template. Also verified on the same seed: any prompt containing "alpha channel" triggers transparency on its own, while "transparent background"/"sticker"/Chinese 透明背景 alone produce fully opaque white backgrounds. Full prompt-writing rules: references/prompt-guide.md.

B. Edit-based cutout for existing art. edit with the default instruction "Remove the background, and output a PNG image" turns any image into a subject-only RGBA PNG (verified: ~66% transparent pixels, soft edges).

Always verify the alpha of deliverables (both methods output RGBA, but only a real check proves transparency):

from PIL import Image; import numpy as np
a = np.array(Image.open(p))[:, :, 3]
print(a.min(), (a < 128).mean())   # expect min 0 and a healthy transparent fraction

Mechanism, for debugging: the 2.1 VAE is natively RGBA (64-ch latent = R/G/B/A × 16 ch; comfy/sd.py marks it "Qwen Image 2.1 VAE ... RGBA"), so SaveImage writes alpha directly — no matting node involved anywhere. The transformer has no alpha-specific code; whether alpha is transparent is a trained behavior triggered by the prompt (or carried in by reference images with alpha, via comfy_extras/nodes_qwen.py: "the vae keeps all four").

7. Pitfalls

Symptom	Cause / fix
Value not in list: unet_name ...	weights not registered — run find; check §1 registration files
Edit ignores the reference image	autogrow key must be images.image_1 (dotted), see §5
Headless server can't see Desktop models	launch args must include --extra-model-paths-config shared_model_paths.yaml
LoadImage "No such file"	image not in the registered input dir (--input-directory)
Port conflict with Desktop app	--port 8189
Opaque alpha despite asking for transparency	prompt lacks the alpha phrases — use --transparent (official template, §6A)
Server won't come up	read the log: %TEMP%\comfy21_server.log

8. Official prompt guide

references/prompt-guide.md distills the official guidance with sources: t2i prompt order (subject → composition → style), text-in-quotes rendering rules, the official RGBA template, edit-instruction style (imperative + identity-preservation phrasing), localized edits and region marking by color, multi-reference slot addressing, cfg/steps interplay, and the optional refine_prompt enhancement. Read it before crafting or fixing prompts.