Activation Functions

1 long-form post on Activation Functions: machine-learning research by Taha Bouhsine, each built around live, in-browser interactive visualizations.

1 post tagged #activation-functions.

Feb 20, 2026

Activations Are Bad for Geometry

ReLU, GELU, and friends factor into a layer's Jacobian as a diagonal modulation that wrecks the geometry of the data manifold. Why pointwise activations are a representational bug.