Transfer Learning
2 long-form posts on Transfer Learning: machine-learning research by Taha Bouhsine, each built around live, in-browser interactive visualizations.
-
Constructing the Head on Learned Features, in JAX/Flax NNX
A runnable companion: train a small conv backbone in Flax NNX, then on its frozen features build a constructed Yat head with no gradient steps and compare. The constructed head lands within a couple of points of the trained one, and even a random backbone's features sort at 73% while its trained head is at chance.
-
You Only Have to Train the Features
Leave a convolutional network's weights at their random starting values and build a Yat head on its features by hand: the trained head on that random backbone sorts at chance while the constructed one reaches 74%. On a properly trained backbone the constructed head reaches 83.2% against 85.7% for the trained one. The accuracy lives in the representation; the classifier, and its edits, are furniture you place. This maps the boundary between what you must optimize and what you can construct.