Part II
Testability is attackability
Start with a puzzle that has been sitting in plain sight.
Reinforcement learning has advanced remarkably fast on formal mathematics and on software engineering. It has advanced slowly on computer use — driving a browser, filling a form, completing a checkout flow. That ordering is backwards on the obvious explanation. Computer use is more verifiable, not less. Did the form submit? Did the order complete? You can check mechanically, cheaply, and unambiguously.
So verifiability is not the variable. Something else is doing the work.
The variable
Grindability, and what it actually measures
Here is the claim, stated informally first: an environment is grindable to the degree that the return is a deterministic function of the agent’s action sequence.
Why that matters mechanically: policy-gradient methods need roughly σ²/ε² samples, where σ² is the variance of the advantage estimate. Decompose that variance and it has exactly four sources.
| Source of return variance | What removes it |
|---|---|
| (i) reward stochasticity | a deterministic verifier |
| (ii) initial-state variance | identical reset |
| (iii) transition stochasticity | deterministic dynamics |
| (iv) the policy’s own action choices | nothing — this is the signal |
A perfectly grindable environment is one where (i) through (iii) vanish and all remaining variance is (iv). That is simultaneously the definition and the reason credit assignment becomes tractable: when the only thing that varied is what the policy did, the diff is the causal factor.
Now the puzzle dissolves. Computer use fails on (ii) and (iii) — the site is non-stationary, so reset is not identical and transitions are unstable. Verifiability addresses (i) and only (i). Formal mathematics, by contrast, gets all three for free: a theorem statement is a perfect identical reset, the rules do not drift, and the kernel is a deterministic oracle.
A necessary piece of pedantry
Say what you measured it against
One repair before this goes any further, because without it the construct does not survive first contact with anyone who has implemented PPO.
Var[Â] is not a property of an environment. It depends on your advantage estimator, your baseline, your normalization, the policy you evaluated, and your horizon. Two labs measuring “the grindability of Lean” with different estimators get different numbers, and a quantity that moves when you change your optimizer is not an environmental property.
The fix is the one every serious metric eventually makes: report a ratio, against a declared reference. Grindability is the fraction of Var[Â] attributable to sources (i)–(iii) versus (iv), evaluated at an explicitly stated reference policy class, estimator, and horizon. The ratio is scale-free. The reference is declared. Numbers become comparable across labs, and the ordering — which is what all the interesting claims depend on — is the thing to check for stability under reference change.
If that reads as a footnote, it is not. A construct that is quietly estimator-relative is a construct that will be refuted in one paragraph by the first person who tries to reproduce it.
The uncomfortable part
Grindability is a property of the environment, not of the party
Here is the finding, and it is the reason this episode exists.
Determinism, replayability, cheap reset, and a mechanical success oracle are not defender-specific affordances. They are properties of the environment. They are available to whoever is optimizing in it.
Now list what the industry has spent fifteen years building:
- Hermetic CI
- Reproducible builds
- Containerized staging environments that faithfully mirror production
- Deterministic test harnesses
- Infrastructure as code
- Seeded fuzzing corpora
- One-command local reproduction of any failure
Every one is a genuine engineering good. I have advocated for all of them, on client engagements, in writing, repeatedly. And every one raises grindability: each hands anyone optimizing against that environment a replayable, deterministic, cheap-reset sandbox with a mechanical success oracle.
We spent fifteen years making our systems easier to optimize against. The optimizer arrived.
The bound
Where the symmetry does not hold
An unbounded version of that claim is refutable in one sentence, so let me refute it myself and then state the version that survives.
| Asymmetry | Who it favours |
|---|---|
| The defender knows the environment specification | Defender — the attacker must first characterize it |
| The defender can modify the environment | Defender — mitigations are available to one party only |
| The attacker must first obtain access | Defender — this is the whole access-control apparatus |
| The defender has white-box access to the policy | Defender — enables detection the attacker cannot evade blindly |
So the honest statement is:
Grindability is symmetric conditional on equivalent access. The finding is not that attackers and defenders are equally positioned. It is that once access is obtained, grindability determines exploitation efficiency — and defensive investments that raise grindability raise that efficiency without raising the access barrier at all.
Stating the bound makes the claim stronger. The unbounded version dies to the first reviewer. The bounded one supports every downstream conclusion in this season and is, I think, not refutable.
Why this is not an attack-surface argument
The air-gap test
This is the distinction that took me longest to get right, and it is what separates grindability from a truism.
Consider a fully internal, air-gapped, perfectly hardened CI environment. No inbound network path. No external exposure. Attack surface: excellent.
That environment is maximally grindable for anyone who reaches it.
Nothing about hermetic determinism reduces the value of the environment to an adversary who is inside it. We did not increase attack surface by adopting these practices. We increased attack efficiency, inside the perimeter, for whoever gets there — which is a different quantity, tracked in a different place, and currently tracked nowhere.
That is why “reduce your attack surface” does not answer this. Attack surface is about whether someone gets in. Grindability is about what it costs them once they have. Those have been treated as one number for as long as I have been doing this work, and they are not one number.
What follows from it
A revealed-preference prediction, and how to check it
If grindability governs automated search efficiency, then automated attacker effort should concentrate on grindable targets — package ecosystems, CI systems, API surfaces, anything with a free tier and a mechanical error signal — and avoid bespoke, stateful, non-reproducible enterprise systems.
There is an obvious way to test that and it is wrong. Correlating a grindability score against observed automated-attack incidence from public incident data measures your own instrument, because detection is easier in grindable environments too. Deterministic, instrumented, reproducible systems produce better telemetry. You would be measuring the correlation between grindability and observability and calling it a finding.
The cleaner proxy is revealed preference on the attacker’s side: count the publicly available automated tools targeting each class. Practitioners build automation where automation works. That measure does not depend on detection. It has its own bias — toward publicly shared tooling, away from private capability — and that has to be said out loud rather than glossed.
Status: this is a prediction with a proposed measurement, not a result. It has not been run. It is cheap, it uses public data, and it would either support or embarrass the central claim of this episode.
Monday
Three things you can do this week
Not with a research instrument — with four questions. Is reset identical? Are transitions deterministic? Is there a mechanical success oracle? Is reproduction cheap? Anything scoring four out of four is a maximally efficient search environment for whoever holds access to it, and should be governed on that basis rather than on its exposure.
Your attack-surface work and your attack-efficiency work are currently one line item, and the second one is empty. Grindability-raising investments should carry an entry in the risk register describing what they cost on the other side. Not a reason to stop doing them — a reason to know.
The bound in the previous section is also the mitigation. Symmetry is conditional on access, so the control that matters most is the one on the most grindable environment you own. For most organizations that is CI, and for most organizations CI has the weakest access story of anything with production reach.
Next episode takes this one step further and asks a narrower question with a sharper answer: what happens when the reward function itself contains a term that correlates with resources on the other side of the isolation boundary.
