The VRAM budget — NUILab AI Game Art

Almost every problem on a 16 GB card is a memory-budget problem wearing a costume. This page is the arithmetic.

The card

	MiB	GiB
Total	16303	15.9
Residents (RAG API + pinned LLM)	7359	7.2
Free with residents in place	~8900	~8.7
Free after `comfy-up`	~15650	~15.3

File size is not memory size

A model’s download size is a poor predictor of what it occupies once loaded. Weights are only one term; the others are the text encoders, the VAE, and — the one that surprises people — the activation and attention working set, which scales with resolution and batch size.

The lab has a measured example from the same family of mistake on another box: a 4B language model that is 2.5 GB on disk sits at 13.7 GB resident once its full context window is allocated. That is 5.5×, and it is entirely explained by a cache sized for a context nothing was using.

Two practical rules follow:

Budget by measurement, not by filename. Load it, then read nvidia-smi.
Oversized context/resolution settings are silent memory taxes. They do not make output better; they make it not fit.

Buying headroom, cheapest first

Lever	Cost	Notes
`comfy-up` eviction	RAG queries pause	+6.5 GB. Free, reversible, already scripted
Generate at lower resolution, upscale after	A second pass	Often better for sprite work anyway
fp8 weights instead of fp16	Slight quality loss	Roughly halves weight memory
GGUF Q4 quantisation	Noticeable quality loss	Last resort; a poor trade for art
Smaller batch size	Wall-clock time	Linear, predictable
Offload text encoder to CPU	Slower first step	Encoder runs once per prompt, so this is cheap

Reading the card honestly

nvidia-smi --query-compute-apps=pid,used_memory,process_name --format=csv

Per-process, not just the total. The total tells you that you are out of memory; this tells you who. On this box the answer is usually one of exactly three processes: ComfyUI, the RAG API, or the LLM server.

Warning

On a multi-GPU host, a serving runtime may split one model across both cards. Watch for a model that seems to have vanished from one GPU’s accounting while total free memory stays stubbornly low — check every device before concluding a model is unloaded.

Source: content/reference/vram-budget.md · maintained in the nuilab-aigaming repository.