Depth on Demand

The last post ended on a strange gift: a classifier that treats depth as resolution. Its layers are time steps of a learned flow, sixteen steps across a fixed span, and running it at sixty-four just renders the same trajectory finer. Once you see depth that way, an old inefficiency becomes visible. The sixteen steps are uniform. Every input, the easy ones and the awkward ones, the gentle stretches of the flow and the hairpin turns, gets the same sixteen. No numerical analyst would integrate a comet that way; they take small steps at perihelion, where the sun bends the orbit hard, and long lazy ones in the outer dark. Their tool is an error controller, and it needs nothing retrained. Can a trained network just… borrow it?

A controller with no opinions#

The controller in question is the oldest one in the book, step doubling (Hairer, Nørsett, and Wanner, 1993). From the current state, take one step of size $h$. Then take two steps of size $h/2$ from the same place. If the flow is gentle there, the two answers agree and you keep the finer one, banking a cheap step; if the flow is sharp, they disagree, so you halve $h$ and try again. Agree comfortably for a while and $h$ doubles. The renderer speeds up and slows down like a driver reading the road.

Why the disagreement is a legitimate instrument, and not just a vibe, is one line of Richardson’s old argument. A second-order step from a given state carries a local error of roughly $C h^3$, with $C$ set by how violently the flow curves right there. The two half-steps carry $2 \cdot C (h/2)^3 = C h^3 / 4$ between them, a quarter of the error along essentially the same stretch. Subtract the two answers and the true errors mostly cancel out of the difference: what remains is about $\tfrac{3}{4} C h^3$, which is to say the difference between your two attempts measures the error of the worse one to within a constant. The controller never knows the true trajectory. It knows two disagreeing estimates of it, and that is enough, because the disagreement and the error are the same number up to a factor you can absorb into the tolerance.

Nothing in this loop knows about classification, labels, or networks. It needs a flow to integrate and an error to watch. The leapfrog classifier provides both: its hidden state $(q, p)$ moves under a learned potential, its layer is a step, and its trained weights stay exactly as they were. We are not proposing an architecture. We are re-rendering a trained network at inference, with the step budget chosen per input by the input’s own trajectory.

Same verdicts, a fraction of the steps#

The first question is whether the adaptive render answers the same way the network does. The run’s referee is a 256-step uniform render, the flow polished far past anything training used, and the agreement is essentially total: across three datasets, three seeds, and tolerances from 0.3 to 0.003, the adaptive render matches the reference’s verdicts on 99 to 100 percent of held-out inputs. At the loosest tolerance it does so in about 16 accepted steps on average; the reference spends 256 everywhere. Reference-grade answers at one sixteenth of the reference’s resolution, because resolution is only needed where the flow actually bends.

Accuracy tells the same story sideways: it does not move. On moons and rings, every tolerance from 0.3 to 0.003 posts the same 99.9 to 100 percent the fixed-depth net posts. The verdict of the flow converges long, long before the trajectory is polished, which is exactly the depth-as-resolution lesson wearing its inference-time clothes.

There is one honest wrinkle, and it is instructive. On spirals, the hardest dataset, the fully-resolved flow and the sixteen-step training render disagree on a few percent of inputs. The controller sides with the flow, at 99+ percent agreement with the reference, and whether that helps depends on the seed: one seed’s flow beats its own training render on labels, the other two fall behind it, for a three-seed mean of 94.7 against the fixed render’s 97.4. The lesson is not that adaptivity costs accuracy; it is that a network trained at sixteen steps calibrated its decisions to the sixteen-step rendering, and the flow’s fully-resolved answer is a slightly different function. The controller is faithful to the equation. The equation, fully resolved, is not obligated to be faithful to labels it was fitted through a coarser lens.

The dial, and the power law hiding in it#

Because tolerance is a knob, precision becomes a dial on computation that exists at inference time, with no retraining and no architecture change. Crank it and watch what it buys:

The measured cost curve is not just monotone, it is lawful. Accepted steps grow as $\mathrm{tol}^{-1/3}$: tighten the tolerance a hundredfold, from 0.3 to 0.003, and the mean accepted steps rise from about 16 to 68, a factor of 4.3 against the power law’s predicted 4.6, a measured log-log slope of 0.32. That exponent is the leapfrog’s order showing through. A second-order method’s local error scales as $h^3$, so holding error at tol forces $h \propto \mathrm{tol}^{1/3}$, and steps are $T/h$. The previous post found the $h^2$ signature of the integrator inside a trained network’s energy band; here is its sibling, the $\mathrm{tol}^{-1/3}$ signature, inside the network’s compute bill. The architecture keeps honoring the contracts of the integrator it is built from.

Where the effort actually goes#

The controller spends unevenly, which was the point. At any fixed tolerance the most expensive input costs three to four times the cheapest: at tol 0.003 on spirals, the range runs from 111 field evaluations to 282. Depth, at last, is a personal expenditure rather than a flat tax. So the natural next belief, the one we held when designing this experiment, is that the expensive inputs are the hard ones, the ones near the boundary, and that effort could double as a confidence signal.

The run says no, three seeds in a row. The correlation between an input’s work and its difficulty, measured as negative log margin, wobbles between $-0.2$ and $+0.3$ with no consistent sign, on every dataset. Look at the cloud yourself:

And the map of where the flow is expensive settles what effort is actually measuring:

That claim, effort measures the flow rather than the verdict, should itself be measured rather than gestured at, and it can be: the first-order proxy for how hard the flow is working is the force magnitude $\lVert \nabla V \rVert$ along the trajectory the controller actually walked. Correlate each input’s work with the mean force along its accepted path and the mechanism appears at full strength where the margin showed nothing: $r = +0.42$ on moons, $+0.62$ on rings, $+0.41$ on spirals, against the margin’s zero on all three. The controller is charging for force. The picture to keep is the marble on the terrain, one last time: the bill is for the hairpins where the learned potential is steep, the stretches where the marble accelerates hard, and a marble can cross rough country on its way to an easy, decisive rest just as it can glide gently to a contested one. This makes the effort signal a different creature from the fixed-point network’s residual halting, which measured distance-to-convergence and did correlate with the boundary. Two kinds of adaptive depth, two different physical quantities: one measures how far the state still has to settle, the other how violently it moves while getting there. Neither one is the margin, and this run is the measurement that says so for the second kind. Graves (2016) learned a halting signal by optimization; the integrator’s controller comes free, but what it meters is the dynamics, not the decision.

Scope: every number is from scripts/adaptive_depth.py on Kaggle (bundle kgl_blog-adaptive-v2): the leapfrog classifier of the previous post trained at sixteen uniform steps on three 2-D datasets, three seeds, then re-rendered by step doubling at five tolerances, with agreement judged against a 256-step uniform reference and work counted as every field evaluation including rejected probes. Accepted-step counts and the power-law fit are means over seeds; the force correlation replays the bundle’s exported seed-0 weights (scripts/render_adaptive_gifs.py); the panels run the same weights live.

The pattern, four integrators in#

By now the pattern is stable: every structural property of a numerical method survives translation into a network, and pays somewhere unexpected. Euler gave depth a direction, the velocity ledger gave it crossings and a rewind, the symplectic step gave it a conserved number, and error control just gave inference a dial: computation per input, chosen by the input, at a cost that follows the method’s order, from weights that were never told any of this would happen. And numerical analysis is not the only theory this series has been mining. The other one has been circling the transformer’s attention mechanism from the far side, and the two finally collide in the next post.

@misc{bouhsine2026depthondemand,
  author       = {Bouhsine, Taha},
  title        = {Depth on Demand},
  year         = {2026},
  month        = {jul},
  howpublished = {\url{https://tahabouhsine.com/blog/depth-on-demand/}},
  note         = {Blog post, Records of the !mmortal Data Scientist}
}