Geometry of Representations
8 parts
Where do representations live, and what makes a latent space good? Contrastive learning, the geometry of embedding spaces, and why the usual activation functions work against it.
All 8 parts →Series
Every post here belongs to a running argument. Each series is a narrative: start at part 1, or jump back in where you left off.
Attention Is a Kernel
6 parts
Attention, read as kernel regression: what the softmax is really computing, why that makes it explainable, and what happens when you make the kernel cheap.
- Start here Attention is Explainable Because it is a Kernel
- Newest The Geometry of Attention Is a Choice of Kernel
Weights in Kernel Space
5 parts
Once everything is a kernel, what is a weight? An interlude on RKHS foundations: where a weight lives, what it can be, and why the MLP block is a representer theorem.
- Start here The Readout is a Convex Combination of Prototypes
- Newest Why Regularization Is a Price List
The Prototype Network
12 parts · ongoing
Replace the activation with a finite, positive-definite kernel and a network becomes a list of prototypes you can read, edit by hand, and finally collapse into a single fixed-point operator.
- Start here What a Finite Kernel Buys an MLP
- Newest One Kernel, Fitted Twice
Networks as Integrators
5 parts · ongoing
Numerical analysis as an architecture catalog: skip connections as an Euler step, momentum nets as half of Newton, and conservation laws as testable predictions about hidden states.
- Start here Your Skip Connection Is Half of Newton
- Newest Depth on Demand