#neural-collapse
2 posts tagged #neural-collapse.
-
What Makes a Good Latent Space? The Welch Bound and the Simplex
A visual story about the hidden codebook inside representation learning: why collapse happens, why opposition is a trap, why class means form a simplex, and why the Welch bound from radio engineering tells us the best possible geometry when too many concepts share too few dimensions.
-
Auditing Latent Space Geometry in JAX
A runnable companion to the Welch-bound latent-space post: generate GIFs and implement the JAX metrics that tell you whether embeddings are collapsing, wasting rank, forming a simplex, or pressing against the Welch floor.