Advanced Vision
CNNs and ResNets are the workhorses of image classification, but they aren't the whole story. This track covers two architectures that go beyond the convolutional backbone: the Vision Transformer, which brings attention to images, and the U-Net, which produces a full-resolution output for dense prediction.
What you'll build
- Vision Transformer (ViT) —
split an image into patches, embed them as tokens, and classify with a
transformer encoder. A global receptive field from the very first layer, using
the built-in
nnx.MultiHeadAttention. - U-Net Segmentation — an
encoder-decoder with skip connections that labels every pixel, using
nnx.ConvTransposeto upsample back to full resolution.
Prerequisites
You should have built a CNN and understand transformers (ViT reuses the encoder) and ResNet skip connections (U-Net generalizes them).
Next steps
- Start with the Vision Transformer.
- The U-Net is also the denoiser backbone behind diffusion models.