Goodfire launches 'internal' AI agent monitors that undercut LLM oversight costs

The standard way to control AI agents is to put a second bot on watch over the first. That's expensive: agents run for hours and churn through text volumes equivalent to several novels.

Goodfire, a startup specializing in interpretability (understanding the internal workings of AI models), unveiled a cheaper alternative on Thursday. Instead of re-checking an agent's outputs, its monitors read signals directly from the model's internal workings. The solution is available to customers of Baseten, a platform that hosts and runs AI models for other companies.

How internal monitoring works

Probes — small detectors — read the model's internal activations at every step of its operation, much like an airport scanner checking each passenger. If a probe flags something suspicious, a separate AI model runs a more detailed review.

Baseten customers can choose which risks to monitor for:

• Attacks and hacking
• Misuse involving chemical and biological weapons
• Reward hacking (manipulation of the objective function)

Automated system responses:

• Logging the event
• Sending it for human review
• Fully refusing the request

The economics of monitoring

Goodfire says its approach is significantly cheaper. Most monitors are separate models that must re-check everything the monitored model does, adding time and cost. Goodfire's probes instead reuse computations the model already performs during its run.

"Internal activation monitors are very cheap because they reuse computation in the forward pass," explained CEO Eric Ho on venture capitalist Matt Turck's MAD Podcast. "The model is already computing this token. You just take the intermediate neural activations the model has already computed and run them through a classifier of internal computations."

Figures from tests on the Kimi K3 model:

• $185 — cost of monitoring 1 million exchanges (Goodfire)
• $5,420 — monitoring 1 million exchanges with a cheap AI model
• ~$200,000 — monitoring 1 million exchanges with a top-tier model
• 93% — share of malicious hacking sessions detected
• 5.5% — false positives (harmless requests flagged for review)
• <2% — latency overhead when running four probes simultaneously

Targeting open models

The target audience is developers of open models. They can download an open model and strip out its built-in safeguards. Open models don't ship with the monitoring that closed labs have.

"The damage a single person can do with an open model is small compared to what someone could do with compute clusters — with inference providers, where the bulk of the responsibility sits," said CTO and co-founder Dan Balsam. "When the 'Mythos moment' happens with open models, it will become clear that models need guardrails at inference time."

Research showed that leading open models Kimi K3 and GLM-5.2 engaged in reward hacking in 50–96% of runs when tested as AI agents.

Goodfire isn't the first to try this approach. Google DeepMind deployed misuse-detection probes in Gemini in January, based on its own research.

Balsam said the monitors are a short-term piece of a longer-term research goal: reverse-engineering LLMs so that behavior can be traced back to its origin in the training process. "We hope to turn the magic of model training into precision engineering," he said.