Local models run for coding agents with no network
`omarchy agent local use best` runs an open model on this machine for coding agents, picked from a recipe catalog, as a rootless Podman container with no network. The change merged into quattro. Version 4.0.6 is the latest release.
`omarchy agent local use best` runs an open model on this machine for coding agents, picked for the hardware from a curated recipe catalog, as a rootless Podman container with no network access. The change merged into quattro. Version 4.0.6 is the latest release.
Several models can run at once, and every one gets its own line in the agents panel next to the subscriptions. The panel offers Local AI only on machines a recipe fits. Podman and crun are installed the first time a model is set up.
David Heinemeier Hansson measured the first recipes on a Ryzen AI Max+ 395 with 128 GB, at their trained 256K context, with a 222K-token prompt. Both serve Chat Completions, Responses and Anthropic Messages, so other agents can connect too.
| Flash Next on halogen | 27B on Gufo | |
|---|---|---|
| Memory | ~99 GiB | ~55 GiB |
| 32K prompt: prefill / decode | 1,607 / 41.3 tok/s | 329 / 14 tok/s |
| 222K prompt: first token / decode | 2.8 min / 27 tok/s | 27 min / 7.6 tok/s |
David Heinemeier Hansson measured both on a Ryzen AI Max+ 395 with 128 GB, in the pull request.
Every model container runs on an internal Podman network with no egress, a read-only root, and all capabilities dropped. A model whose memory is held by other running models still installs, stopped, and says why it can't start yet.