Gemma 4 in Codex CLI: Running It Locally Works, but It Still Trails the Cloud

Local Gemma 4 can now work in Codex CLI as an agent for everyday coding: reading files, writing patches and running tests. But an experiment on two different setups showed that getting it to run is only half the battle. In reliability, code quality and first-attempt results, cloud-based GPT-5.4 is still clearly ahead.

The author did not want to test abstract "local AI development" but a down-to-earth scenario: can the model replace a cloud API in daily work with Codex CLI? The motivation is easy to understand: token costs, privacy requirements and dependence on external services.

Two configurations were built for the test. The first was a MacBook Pro with an M4 Pro chip and 24 GB of memory, running Gemma 4 26B MoE with Q4_K_M quantization via llama.cpp. The second was a Dell Pro Max GB10 with 128 GB of unified memory and NVIDIA Blackwell, running Gemma 4 31B Dense via Ollama 0.20.5. In both cases the model was connected to Codex CLI as a custom provider in responses API mode.

Standing up the local stack was not easy. On the Mac, Ollama broke on tool calling because of streaming bugs and hung on long prompts, which is critical for Codex CLI: its system prompt alone takes about 27 thousand tokens. The setup that finally worked was llama.cpp with manually tuned flags, web_search disabled and a context of 32,768 tokens.

The GB10 did not work on the first try either: vLLM ran into incompatible PyTorch and CUDA builds for Blackwell, and a hand-built llama.cpp handled some tool types incorrectly. Once again, the most practical solution was not the "ideal" stack but the one that simply worked: Ollama.

The benchmark ran on April 12, 2026, on Codex CLI v0.120.0. Using codex exec --full-auto, all three configurations got the same task: write a Python function parse_csv_summary with error handling, then prepare tests and run them.

Cloud GPT-5.4 with high reasoning effort did best. It produced clean code with type hints and a sensible exception chain, and passed all five tests on the first attempt in 65 seconds.

Local Gemma 4 31B on the GB10 also delivered working code on the first pass, but of lower quality: no type hints and no recognition of boolean values. All five tests still passed right away, and the run took about seven minutes and three tool calls.

The Mac with the 26B MoE was the most problematic. The model left dead code, rewrote the test file several times and made silly typos, such as a broken variable name or an invalid encoding string. The task took 4 minutes 42 seconds but required 10 tool calls and five failed attempts to write the tests.

Most interestingly, in raw speed the Mac unexpectedly beat the more powerful GB10. In llama-bench, the 26B MoE on the Mac produced about 52 tokens per second versus 10 tokens for the 31B Dense on the GB10, while in prompt processing at 8K context the machines were nearly level: 531 versus 548 tokens per second. The explanation lies in the Mixture of Experts architecture: an MoE activates only part of its parameters at each step, so the amount of data pulled from memory per token is much smaller.

But this advantage barely helped in the real task, because most of the time went not on computation but on model errors, repeated tool calls and extra edits along the way.

The main takeaway is twofold. On one hand, Gemma 4 has moved local agentic coding from the "breaks almost every time" category to "you can live with it": the author notes that on tau2-bench, Gemma 3's function calling score was 6.6%, versus 86.4% for Gemma 4 31B. On the other hand, in practical development, first-attempt reliability matters more than a tokens-per-second record.

So local mode already looks realistic for private tasks, quick iterations and work without ongoing API costs, but in complex scenarios cloud models remain stronger. The most sensible conclusion from the test is a hybrid mode: a local model for some tasks, and the cloud as the main tool where the cost of an error outweighs speed or privacy.