Records of the !mmortal Data Scientist

series · 9 parts · in progress

The Prototype Network

Replace the activation with a finite, positive-definite kernel and a network becomes a list of prototypes you can read, edit by hand, and finally collapse into a single fixed-point operator.

Start reading → Your Network Is a List of Pictures. You Can Edit It.
  1. 01 Your Network Is a List of Pictures. You Can Edit It. If a neuron is a labelled picture, a classifier is a list of them, and a list is something you edit. Add a class to a trained-free Yat-kernel network by placing twenty pictures, and it recognizes that class at 95% with zero gradient steps. Delete a class by removing its pictures, and it is forgotten exactly, the other classes untouched. Class-incremental learning with no penalty and machine unlearning that is instant and exact, both falling out of the architecture rather than bolted on. JAX companion Editing a Network by Hand, in JAX/Flax NNX
  2. 02 How Much of a Fashion-MNIST Network Can You Build by Hand? Construct the prototype head on random and learned features, then replace the backbone with named edge and corner measurements. On Fashion-MNIST the zero-training pipeline reaches 83.3%, versus 85.7% for the matched trained model. JAX companion Constructing the Fashion-MNIST Network, in JAX/Flax NNX
  3. 03 How Far Down Can You Build? One hand-built feature layer matched a trained backbone at 83.3% on Fashion-MNIST, and real networks are deep. Conveniently, the recipe for a second layer has been on the shelf for half a century: vision science says edges assemble into junctions, continuations, bends and stripes. This post takes the recipe down and follows it, builds layer 2 entirely by hand with every dimension still nameable in one sentence, and measures exactly where construction stops, and why. JAX companion Building the Second Layer by Hand, in JAX/Flax NNX
  4. 04 When 80% Should Mean 80% A network hands you a probability with every answer, and the number is the part you act on. So when this series' bounded, self-explaining kernel network says 80%, is that a measurement or a mood? Five posts of evidence say it should be the honest one. This post puts that reputation through a lie-detector test, reliability diagrams, expected calibration error and temperature scaling against a matched ReLU MLP on Fashion-MNIST, and what the test found is the post. JAX companion Calibrating a Bounded Net, in JAX/Flax NNX
  5. 05 The White-Box Survival Model on Trial Build a survival network from learned prototype patients, derive its exact risk decomposition, and benchmark it on five datasets against Cox, penalized Cox, Random Survival Forest, and ReLU DeepSurv. Calibration, editing, and shift detection are measured separately. JAX companion Running the Survival Trial, in JAX/Flax NNX
  6. 06 Your Network Is a Stack of Layers. It Could Be a Fixed Point. A deep network makes you choose its depth before you have seen the problem, and gives every layer its own weights. Share one Yat-kernel operator across depth and the stack becomes a single equation: the answer is the fixed point reached by iteration. On the measured test trajectories, the solver converges from widely separated starts and the local Jacobian norm stays below one. The same twenty-four prototypes describe every step, reaching 98.2% on two moons from 1,700 shared parameters. JAX companion A Network That Is a Fixed Point, in JAX/Flax NNX
  7. 07 Edit One Operator, Edit Every Depth One post taught and forgot classes by editing rows of a Yat network, with proofs that nothing else moved. Another melted the stack of layers into a single operator iterated to a fixed point. This is the collision. Every one of those editing proofs rested on a pasted row entering the score once, as one term in one sum, and in an equilibrium network there is no once: whatever you paste is applied at every depth and fed back into its own input, and every fixed point is free to drift. So did melting the stack melt the editability? This post pastes, deletes, and measures: every guarantee that survives is either proved inside the recursion or measured against the real run, fixed point by fixed point. JAX companion Editing a Deep Equilibrium Network, in JAX/Flax NNX
  8. 08 One Kernel Family, Fitted Two Ways A dense kernel-ridge solve and a learned-center Yat expansion use the same kernel family but optimize different hypothesis classes. Their predictions correlate at 0.95 on housing; the compressed model then scales through datasets the dense baseline cannot hold. JAX companion Solving It and Descending It, in JAX/Flax NNX
  9. 09 How Many Random Neurons Buy a Trained One? Freeze a bank of randomly initialized Yat units and train only the linear readout. The induced random-feature kernel is a Monte Carlo average; under finite variance its estimation error has the familiar square-root scaling. This post measures that exponent, counts how many frozen units buy each rung of an accuracy ladder, and then unfreezes the centers to measure what feature learning adds. JAX companion Lazy Training a Yat Network in JAX/Flax NNX

This series is still being written. New parts land in the archive and the RSS feed as they publish.