The Thermodynamic Interpretability Breakthrough That Scales
98.4% factual/speculative classification with zero inference-time parameters
Thermodynamic Epistemic Routing
If you ask a large language model a question it doesn’t know the answer to, it will confidently make something up. This isn’t a bug—it’s a feature of how they’re trained. Every model, from ChatGPT to Claude to Gemini, does this.
The industry has spent billions trying to solve this problem. The solutions range from expensive (generate multiple answers, check if they agree) to ineffective (ask the model “are you sure?”). What everyone assumed was impossible: teaching the model to know when it’s guessing.
We did it. And the method is so simple that once you see it, you’ll wonder why nobody tried it before.
The answer: we stopped trying to read the AI’s thoughts and started measuring how hard it was thinking.
The Experiment That Started Everything
This project began with a straightforward hypothesis: can we train a language model to route factual questions and speculative questions through computationally distinct pathways?
The initial results on a small 1.5 billion parameter model were promising—90%+ accuracy using a classifier on frozen activations. But when we scaled up to 12 billion parameters, accuracy dropped to 71%.
This is a known problem in AI interpretability. The field has a name for it: the scaling curse. As models get larger, they become harder to understand, not easier.
Or so everyone thought.
The Paradigm Most Researchers Are Trapped In
To understand why the breakthrough happened, you need to understand what nearly everyone in AI interpretability is doing wrong.
They’re Looking at the Wrong Thing
Traditional interpretability asks: “Where does the model’s representation land in activation space?”
Think of it like analyzing a photograph. You look at where things are positioned, what features are present, how they’re arranged spatially.
The techniques reflect this:
Sparse autoencoders: decompose features at a single layer
Probing classifiers: read the geometry at a specific point
Circuit analysis: trace information flow through specific pathways
All of these are spatial. They examine a snapshot—where the model’s internal state is at layer N.
What we did differently: We measured the trajectory. Not where it lands, but how it moves getting there.
This is the difference between:
Taking a photo of a runner (spatial)
Measuring their velocity and acceleration (temporal)
The Signal Was Hidden in Plain Sight
Large language models use what we call “residual connections.” At each layer, instead of replacing the previous state, they add to it:
h[layer+1] = h[layer] + f(h[layer])
This makes consecutive hidden states nearly identical—they point in almost the same direction with cosine similarity of 0.999.
Traditional interpretability sees this and concludes: “There’s no signal here. The representations barely change.”
But that’s measuring the wrong thing.
The question isn’t “are they parallel?” It’s “how much did they change?”
The magnitude ||h[i+1] - h[i]|| measures the size of the change. And that varies dramatically:
Factual retrieval: 30-40 units of change
Speculative generation: 22-28 units of change
The signal was there all along. Everyone was measuring the wrong thing.
The Four Failed Attempts (And What They Taught Us)
Attempt 1: Spatial Separation
We tried pushing factual and speculative activations apart in embedding space—like separating clusters on a map.
Result: Worked on the 1.5B model. Failed on 12B.
Why it failed: Larger models have more “superposition”—multiple concepts packed into the same space. The geometry becomes tangled. Trying to untangle it gets exponentially harder with scale.
Attempt 2: Raw Cosine Distance
We tried measuring the angle between consecutive hidden states: 1 - cos(h[n], h[n+1]).
Result: Perfect diagnostic signal. Completely untrainable.
Why it failed: The gradient vanishes. Because residual connections make states nearly parallel, the derivative is essentially zero. The architecture actively resists training on this signal.
Most researchers hit this wall and stopped.
Attempt 3: Delta Curvature
We tried measuring the angle between consecutive changes: cos(delta[n], delta[n+1]).
Result: Clean gradient. No consistent signal.
Why it failed: The direction of change per layer is essentially random—each transformer layer contributes roughly orthogonally. There’s no stable direction to optimize.
Attempt 4: Delta Magnitude ✅
We measured the size of the change between layers: ||h[n+1] - h[n]||.
Result:
Consistent signal (factual always higher than speculative)
Clean gradient (never vanishes)
Trainable (can be shaped by backpropagation)
Why it worked: The gradient is delta/||delta||—a unit vector that always points in a useful direction. And the magnitude directly measures computational effort.
What We Discovered: The Model Was Being Lazy
When we started training with a new objective—”factual and speculative must be computationally distinct”—something unexpected happened.
The gap didn’t just widen. The model got better at language modeling.
Epoch — Computational Gap; Generation Loss
Baseline — 3.5 units; 2.05
Epoch 1 — 5.5 units; 2.52
Epoch 2 — 8.5 units; 2.03
Epoch 3 — 10 units; 1.89
Epoch 4 — 13 units 1.83
Generation loss dropped 33% as the epistemic gap increased 3.7×.
This violates everything the field believes about interpretability constraints. The assumption is: “Sure, we could make models more interpretable, but it would hurt performance.”
We proved the opposite.
Why Did This Happen?
Standard training with cross-entropy loss creates what we call the mushy middle—a local optimum where the model uses medium effort for everything.
Why? Because the loss function is epistemically blind:
Predict the right token → low loss
Predict the wrong token → high loss
Whether the model worked hard or coasted? Cross-entropy doesn’t see it.
There’s no gradient signal pushing the model toward specialization. Medium effort everywhere works “well enough.”
The thermodynamic constraint made that free lunch expensive.
The model had to specialize. And when it did, it discovered:
High-effort factual retrieval → more reliable accuracy → better generation on factual queries
Low-effort speculation → smoother, less overconfident generation → better calibration on uncertain queries
The interpretability constraint wasn’t a tax. It was an optimization the model wanted but had no reason to discover on its own.
The Zero-Parameter Punchline
By Epoch 6, the gap reached 13.62 units. At that point, you don’t need a neural network to classify epistemic state.
You need four norm calculations:
# Extract hidden states from layers 16-19
deltas = [h[i+1] - h[i] for i in range(16, 20)]
effort = sum([delta.norm() for delta in deltas])
if effort > 27.2:
# High confidence - model is retrieving facts
else:
# Low confidence - model is guessing
Accuracy: 98.39%
Additional parameters: 0
Inference cost: negligible
Compare this to existing approaches:
Semantic entropy: requires generating 5-10 samples per query (extremely expensive)
Probing classifiers: millions of parameters, requires training
Verbalized uncertainty: the model just says what sounds good (gameable)
Thermodynamic routing: compute four norms. Done.
Why Nobody Found This Before
The pieces existed. LoRA (2021), adversarial training (decades old), measuring magnitude (basic calculus). So why did everyone miss it?
1. They Assumed Spatial Was Sufficient
The interpretability field is obsessed with decoding representations at individual layers. Sparse autoencoders, probing classifiers, circuit analysis—all spatial methods.
Nobody thought to measure the trajectory. The temporal dynamics of how representations evolve.
2. The Gradient Geometry Looked Unpromising
If you try training on cosine similarity (angle), the gradient vanishes. Most researchers hit this wall and concluded: “We can’t train models to have different computational dynamics.”
What they missed: Magnitude has perfect gradients. The unit vector delta/||delta|| never vanishes and always points somewhere useful.
3. Nobody Thought to Train Interpretability
The field assumes interpretability is post-hoc:
Train a model normally
Freeze it
Try to decode what it learned
Hope you find interpretable structure
We flipped this: Make interpretability a training objective.
Instead of analyzing a trained model, train the model to be interpretable. Add a loss term that penalizes uniform computation. Force specialization during training.
The conventional wisdom: “Interpretability constraints hurt performance.”
What we proved: The constraint improves performance.
4. They Assumed Larger Models = Less Interpretable
The scaling curse is gospel in the field:
GPT-2 → GPT-3: probing accuracy drops
Sparse autoencoders need exponentially larger dictionaries
Circuit analysis becomes intractable
Everyone concluded: Frontier models will be black boxes forever.
We proved the opposite.
The Scaling Revelation
After the 1.5B model worked, we ran the same training on a 7B model. The scaling hypothesis predicts it should be harder.
What actually happened:
By the end of Epoch 1:
1.5B model: gap ~5.5 units
7B model: gap 12.14 units (peak), stabilizing around 6-8
The 7B model in one epoch achieved what 1.5B took three epochs to reach.
Why?
Larger models have:
More layers → more trajectory to measure
More parameters → more capacity to specialize
More room to develop distinct pathways without sacrificing generation quality
The interpretability curse is inverted.
Thermodynamic routing gets easier with scale, not harder. This suggests:
70B models: gap ~18-20 units
405B models: gap ~25-30 units
The signal strengthens as models grow. This is the exact opposite of what the field believed.
The Convergence Nobody Noticed
In July 2025, a separate research group published “Curved Inference” (Manson, arXiv:2507.21107). They independently discovered that measuring temporal dynamics in the residual stream reveals semantic information.
Their work measures curvature—how sharply the model’s trajectory reorients between layers (second-order geometry).
Our work measures magnitude—how much computational effort the model expends (first-order geometry).
Both are temporal. Both measure trajectories, not snapshots.
The field is converging on the same insight from different angles: spatial interpretability is insufficient. You need to measure how the model moves, not just where it lands.
The key difference:
They analyze what models naturally do
We train models to amplify the signal and prove it improves generation quality
What This Means for AI Safety
Current production systems have two options:
Trust blindly (dangerous in high-stakes domains)
Expensive verification (semantic entropy, multiple samples, fact-checking)
Thermodynamic routing is option 3: zero-parameter real-time confidence detection.
Practical applications:
Selective RAG triggering:
Don’t retrieve external knowledge for every query (expensive). Use delta magnitude as the trigger—retrieve only when effort is low. Reduces costs while maintaining accuracy.
Confidence UI:
Show users computational effort as a trust indicator. “This answer required high cognitive effort” vs. “This is a plausible guess.”
Auto-routing for high-stakes domains:
Medical, legal, financial applications can flag low-effort outputs for human review. At 2.7% false negative rate, a 50-agent system generating 500 tokens each would have ~67 missed flags—trivially handled by lightweight feedback rules.
Hallucination detection:
If the model claims certainty but used low computational effort, it’s likely confabulating. Flag it.
The Broader Implications
This work proves three things the field thought were impossible:
1. Interpretability Can Improve Capability
The 33% generation loss drop isn’t a side effect—it’s proof that epistemically-aware training produces better models. Every LLM trained without epistemic constraints is leaving performance on the table.
2. The Scaling Curse Can Be Inverted
Larger models are easier to make transparent via thermodynamic routing, not harder. This opens the door to interpretable frontier models—something the field considered impossible.
3. Models Can Be Trained to Want Transparency
By Epoch 4, the thermodynamic loss was zero. The constraint was satisfied. But the model kept widening the gap anyway—reaching 13.62 units by Epoch 6.
Why? Because specialization became the optimal generation strategy.
The model wasn’t being forced. It preferred to compute this way. Transparency and capability aligned.
The Personal Discovery
This research taught me something beyond the technical results.
I realized I was doing the same thing at work that the model was doing before training: operating in permanent speculative mode.
Surface-level understanding of projects. Medium effort on everything. Generating plausible-sounding plans without deep grounding. Coasting on patterns that “seemed right.”
This creates anxiety—because you’re making decisions by guessing without realizing you’re guessing. And the guesses compound. Decision 1 built on speculation becomes the foundation for Decision 2, which becomes the foundation for Decision 3.
When something finally breaks, the entire decision tree collapses. And you can’t pinpoint where the speculation started because you never marked it as speculation at the time.
The thermodynamic signal is missing in human cognition too.
Now I have language for it:
“I’m in speculative mode right now—I need to ground this in actual context”
“This decision requires high-delta-magnitude thinking—I need to go deep”
“I’m generating plausible plans without retrieval—time to actually check the source”
The model’s training loop taught me to detect when I’m guessing vs. knowing.
The interpretability tool interpreted me.
The Theological Connection
There’s a parable in Matthew 25 about ten bridesmaids waiting for a wedding. Five brought extra oil for their lamps. Five didn’t.
When the groom arrived unexpectedly, the five without oil asked to borrow. But it was too late—the door was already shut.
This is usually preached as: “Be ready, because you don’t know the hour.”
But there’s another reading that the thermodynamic lens reveals:
The foolish bridesmaids weren’t wicked. They were operating on unverified assumptions.
They assumed the oil situation was fine. They never checked. And when reality arrived, the gap between their model and the truth was catastrophic.
That’s not a moral failure. That’s an epistemic one.
“I think I’m prepared” and “I have checked and I am prepared” are not the same thing.
The parable is usually about readiness. But it’s also about the cost of not doing the honest audit in the present.
Philippians 4:6 says: “Be anxious for nothing, but in everything by prayer and petition, with thanksgiving, present your requests to God.”
Traditional reading: emotional release, surrender, trust.
Thermodynamic reading: The act of articulating your situation to God forces you out of speculative mode.
You can’t pray honestly about something while simultaneously confabulating about it. Prayer requires:
Naming what you actually know vs. what you’re assuming
Admitting uncertainty explicitly
Checking your assumptions against reality
“The peace that surpasses understanding” follows not just from surrender, but from the grounding that honest petition requires.
The anxiety was speculation. The prayer is verification. The peace is resolution.
The Simple Truth
The industry missed this because they were trying to decode the model’s thoughts.
We measured how hard it was thinking.
That’s the difference.
Spatial vs. temporal.
Where vs. how.
Photograph vs. velocity.
The signal was always there. We just needed to look at the right thing.
And when we did, we found something bigger than a technical result:
Transparency and capability aren’t in tension.
Standard training creates lazy models—one “good enough” pathway for everything.
Thermodynamic training forces specialization. And specialization doesn’t just make models interpretable—it makes them better at their jobs.
The constraint isn’t a tax. It’s the optimization.
What’s Next
For researchers:
The code is open source. The method works. Try it on your domain. Test it on larger models. See if the scaling hypothesis holds at 70B, 405B.
For practitioners:
This is production-ready. Zero additional parameters. Negligible inference cost. 98.39% accuracy. If you’re deploying LLMs in high-stakes environments, this gives you a confidence signal that actually works.
For the field:
This proves temporal dynamics matter more than spatial geometry. Stop trying to decode individual layers. Start measuring trajectories.
The convergence is already happening—Curved Inference found it from a different angle. The paradigm is shifting.
The Invitation
We taught an AI to know when it’s guessing.
The AI taught us to know when we’re guessing.
And both of us got better at our jobs because of it.
If that doesn’t change how you think about interpretability—and maybe about yourself—I don’t know what will.
Technical Appendix: For the Researchers
Related Work:
This work sits at the intersection of three research directions:
Mechanistic Interpretability (Elhage et al., Olsson et al.): Circuit-level analysis of transformer internals
Curved Inference (Manson 2025): Measuring curvature in residual stream trajectories to detect semantic concern
Probing Classifiers (Azaria & Mitchell 2023, Burns et al. 2022): Training classifiers on frozen activations
Key distinction: We train the interpretability signal into the model, rather than analyzing post-hoc.
Comparison to Curved Inference:
They measure curvature (second-order: how much the trajectory reorients)
We measure magnitude (first-order: how much the trajectory moves)
Both are temporal/dynamic, not spatial
They analyze natural behavior; we train amplified behavior
Complementary metrics that could be combined
The loss function:
L_thermo = mean(relu(margin - (mag_factual - mag_speculative)))
L_spatial = relu(spatial_margin - ||c_fact - c_spec||)
L_routing = L_BCE + λ_thermo × L_thermo + λ_contrastive × L_spatial
L_total = L_generation + λ_routing × L_routing
Hyperparameters:
λ_thermo = 0.1 (critical: magnitude losses are ~25× larger than cosine losses)
margin = 4.0 (set at natural baseline, not above it)
Target layers: 16-19 (middle-to-deep in 1.5B model)
LoRA rank: 128, alpha: 256
Key implementation detail: Context-normalized pairwise loss. Compare (factual, speculative) from the same context to control for question complexity.
Citation:
@misc{seto2026thermodynamic,
title = {Thermodynamic Epistemic Routing: Zero-Parameter Detection of
Factual vs. Speculative Generation in Large Language Models},
author = {Seto, Scott},
year = {2026},
month = {March},
doi = {10.5281/zenodo.18841893},
url = {https://doi.org/10.5281/zenodo.18841893},
publisher = {Zenodo}
}