Every post here belongs to a running argument. Each series is a narrative: start at part 1, or jump back in where you left off.
6 parts · ongoing
Where do representations live, and what makes a latent space good? Contrastive learning, the geometry of embedding spaces, and why the usual activation functions work against it.
6 parts
Attention, read as kernel regression: what the softmax is really computing, why that makes it explainable, and what happens when you make the kernel cheap.
4 parts
Once everything is a kernel, what is a weight? An interlude on RKHS foundations: where a weight lives, what it can be, and why the MLP block is a representer theorem.
9 parts · ongoing
Replace the activation with a finite, positive-definite kernel and a network becomes a list of prototypes you can read, edit by hand, and finally collapse into a single fixed-point operator.
4 parts · ongoing
A trained network carries its own kernel, and a kernel can be measured. Decompose it, cut modes out of it, count the concepts it will admit to having, and find out which parts of the design were doing the work all along.
5 parts · ongoing
Numerical analysis as an architecture catalog: skip connections as an Euler step, momentum nets as half of Newton, and conservation laws as testable predictions about hidden states.