Anthropic opens a window into Claude's thinking: the Jacobian lens catches hidden concepts
Anthropic has introduced the Jacobian lens, an interpretability tool that opens up the LLM black box. It shows which concepts activate in Claude's intermediate layers when it answers a question. The findings range from the mundane to the unusual: the model builds hidden representations of ideas that do not appear in its ordinary output. This offers a first, naive look at how LLMs "think."
AI-processed from MIT Technology Review; edited by Hamidun News
Anthropic has developed a technique called the Jacobian lens, which offers the clearest view yet of what happens inside Claude when it answers questions or performs tasks.
What it finds
When Claude answers a question, the Jacobian lens shows activations in the intermediate layers. The results range from the everyday (a layer tied to numbers activating while solving a math problem) to the unusual (hidden concepts that do not appear in the explicit output).
How it works
Instead of inspecting weights and activations blindly, the tool uses Jacobian mathematics (a matrix of derivatives) to track how changes in the input affect intermediate representations. This makes it possible to:
- See which abstract concepts the model activates
- Trace the path of logic from input to output
- Understand which faulty chains of reasoning lead to wrong answers
Why it matters for safety
Uncovering hidden concepts matters for AI safety: if we see that a model activates dangerous concepts, even when the final answer is safe, we can fix it.
- Nov 2, 2026Karpathy builds a YouTube clone with 6454 videos from Hacker News recommendations in two hours
- Nov 2, 2026Karpathy used Claude to build HackerTube from 6,454 videos recommended on Hacker News
- Nov 2, 2026Arca gives AI agents finance tasks lasting up to 10 days, but leaves the trades to humans
- Nov 2, 2026Anthropic needs to grow revenue 32% a year to cover $850 billion in contracts
Key Questions Answered
What it finds?
When Claude answers a question, the Jacobian lens shows activations in the intermediate layers. The results range from the everyday (a layer tied to numbers activating while solving a math problem) to the unusual (hidden concepts that do not appear in the explicit output).
How it works?
Instead of inspecting weights and activations blindly, the tool uses Jacobian mathematics (a matrix of derivatives) to track how changes in the input affect intermediate representations. This makes it possible to: See which abstract concepts the model activates Trace the path of logic from input to output Understand which faulty chains of reasoning lead to wrong answers
Why it matters for safety?
Uncovering hidden concepts matters for AI safety: if we see that a model activates dangerous concepts, even when the final answer is safe, we can fix it.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.