Sharing the GPU

The card is not dedicated to art. It also hosts a read-only replica of the lab’s research-paper RAG and a pinned 4B language model, together holding ~7.3 GB of 16 GB. Two scripts arbitrate.

The design

comfy-up evicts, comfy-down restores. Three properties make this safe to run casually:

It records what it evicted. Before stopping anything, comfy-up writes a state file noting whether each service was actually running. comfy-down restores from that file, so a service you had deliberately stopped beforehand stays stopped. Without this, “restore” silently becomes “start everything”, which is a different and wrong operation.

It waits for the driver. Memory is released asynchronously. The script polls until the card is actually clear rather than racing ComfyUI into a half-freed GPU. Measured: 7359 MiB → 2 MiB in about two seconds, giving ComfyUI 15.3 GiB.

It verifies rather than assumes. comfy-down polls the RAG API’s health endpoint and exits non-zero if it never comes back, so a failed restore is loud instead of a silently dark service.

What is deliberately not stopped

The replica’s database and vector-store containers keep running the entire time. They hold no VRAM, so stopping them would free nothing — and it would break something important:

The primary’s replication push talks to those two containers and the filesystem. It never talks to the query API. So replication continues normally while ComfyUI owns the GPU.

This was checked against the running system rather than reasoned about, and then confirmed by accident under the best possible conditions. A scheduled push fired 25 seconds after an eviction stopped the query API, with ComfyUI holding the card, and completed cleanly:

20260809-190538 ok works=49/49 vectors=2474/2474 pdfs=34

Independently verified afterwards with the primary’s own comparison check: in sync, 49 works and 2474 vectors on both sides.

The practical consequence: leaving ComfyUI up overnight costs you the ability to query the replica. It does not cost you freshness, and it cannot lose data.

The restore is lazy, and that looks like a failure

After comfy-down, VRAM reads about 5200 MiB, not the ~7350 MiB from before. Nothing is wrong. The language model returns to its full ~3.9 GB immediately, but the RAG API loads its embedding and reranking models on demand:

Judge the restore by the health check, not by the memory number.

Why not just leave everything resident?

You can. With the residents in place you have ~8.7 GB, which runs SD 1.5 comfortably and SDXL tightly. Eviction exists for the larger models, which do not fit at all alongside 7.3 GB of tenants. The policy is deliberate: generation is bursty, so claim the card, generate, release it.

Reading the scripts

Both live in ~/scripts on the GPU box and are linked onto PATH as comfy-up and comfy-down, so ssh <box> comfy-up works without a path. See Commands .