ComfyUI
Tool, image video, by Comfy Org
Assessment
The engine for everything generative that is not text: image, video, lip sync, 3D. Run it as a service with saved workflows and an API, and accept that it takes the cards when it works. A supervisor that swaps it with the LLM is the price of sharing one box.
2026-10-09
Strengths
- Any image or video pipeline as a node graph, saved as JSON and run headless over its API
- The first place new open models land: Wan, LTX, Flux, SDXL, Hunyuan3D
- Multi-GPU splits through custom nodes when a model will not fit one card
Limitations
- It wants the whole GPU: nothing else fits beside a loaded video model
- A graph is as fragile as its custom nodes; updates break workflows
ComfyUI is a node-based front end and runtime for diffusion models: every step of a pipeline, from loading a checkpoint to decoding the final frames, is a node, and a whole workflow is a JSON graph that can be queued over its HTTP API without the browser.[1] That is what turns it from a hobby tool into a service: save the graph, post it with new inputs, poll for the result.
What it is for
Image generation and editing, text-to-video and image-to-video, lip sync and voice-over on video, image-to-3D, upscaling and inpainting. New open models usually get ComfyUI support within days of release, often before anything else.
Sharing a box with a language model
A loaded video model fills a 12 GB card, and a two-card split fills both, so an LLM cannot stay resident beside it. The workable pattern is a supervisor that owns the swap: a job arrives, the LLM is stopped, ComfyUI starts (about a hundred seconds), the job runs, and after an idle timeout the LLM comes back.[2] Everything that calls the LLM then has to treat its absence for minutes as normal and have a tier to fall back to.
Things to know
- Custom nodes are the ecosystem and the fragility; pin versions in a workflow that works.
- Measure VRAM per workflow, not per model: the same checkpoint with a different sampler can differ by gigabytes.
- A manual session in the browser looks the same to a supervisor as a hung job; make the supervisor able to tell them apart.
History
- 2026-10-09: article written and published.
- 2026-10-09: entry created.
Comments
Public comments on each entry are coming. Nothing is collected here yet.