HomeCourse catalog

Your route through JAX.

Open a lesson. Build something you can explain.

22 phases · 104 lessons available

These are practice drafts. Some courses are still in development; CPU checks and teaching review are separate. See status and contribution priorities.

Start on CPU setup, continue through foundations and training, then choose a deployment or research branch.

Current sectionStart Here · CPU Setup

Course search

Find a lesson or concept

Press Enter to apply · Escape to close

Section 1 of 51 courses

Start Here · CPU Setup & Optional TPU Track

Start with the local CPU environment. Add the optional TPU specialization when you need to qualify accelerator execution.

00Onboarding · Local CPULaptop CPU4 lessons (~3.2h)

Setup & first steps

Available draftStart here
4 available

Revision v0.18.1 ·

Run and inspect saved programs in a CPU environment, then learn how to prepare and verify explicit, source-reviewed TPU targets. TPU jobs are optional and separate from the course’s CPU practice.

Start with each lesson’s recorded CPU baseline. For first physical TPU practice, use only the bounded launcher and precision targets in the downloadable jax-tpu-gcp.zip workspace with its pinned qualification requirements; inspect their receipts for the observed TPU backend. Review each additional program’s source, dependencies, topology, and data ownership before considering a separate target run.

Phase study guide: Can someone else reproduce your first result?

Start after: No previous JAX experience needed. Begin with the guided setup.

Hardware and future work

CPU; the TPU bridge is optional accelerator practice

Bring the phase together · Integration Project

Check the CPU environment and a saved matrix arithmetic identity

Checkpoint: Explain which device ran your code and reproduce the result in a fresh environment.

What to do & where to put your code: Write your implementation in projects/welcome-audit/starter/model.py (copied from the starter scaffold in projects/welcome-audit/starter/) and run projects/welcome-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3, 4

Work in projects/welcome-audit/starter/model.py. Complete stages 1 and 2 for independent calculations and changed-condition diagnosis, then retain the stage 3 evidence with --evidence-dir runs/welcome-audit. Stage 4 integrates the checks. Learner commands explicitly select the starter; --implementation solution is instructor reference execution. This bounded practice does not certify the entire phase or target hardware.

Additional project: Connect arrays, transformations and explicit state

Section 2 of 54 courses

Part I · Shared CPU Foundations

Master arrays, pure functions, jit/grad/vmap transforms, explicit state, and numerical optimization on CPU.

01Core Foundation · Local CPULaptop CPU4 lessons (~3.1h)

Arrays & pure functions

Available draftShared foundations
4 available

Revision v0.18.1 ·

Treat every axis as part of a contract. Work through normalization, broadcasting and immutable updates, then diagnose mistakes that still return plausible numbers.

Draw the input and output shapes before running each change.

Phase study guide: Can you spot a wrong axis even when the program runs?

Start after: 00 Setup & first steps

Hardware and future work

CPU

Bring the phase together · Integration Project

Audit masked batch moments and padding-sensitive denominators

Checkpoint: Predict output shapes and explain why a pure function is easier to transform.

What to do & where to put your code: Write your implementation in projects/arrays-audit/starter/model.py (copied from the starter scaffold in projects/arrays-audit/starter/) and run projects/arrays-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3, 4

Work in projects/arrays-audit/starter/model.py. Complete stages 1 and 2 for independent calculations and changed-condition diagnosis, then retain the stage 3 evidence with --evidence-dir runs/arrays-audit. Stage 4 integrates the checks. Learner commands explicitly select the starter; --implementation solution is instructor reference execution. This bounded practice does not certify the entire phase or target hardware.

Additional project: Connect arrays, transformations and explicit state

02Core Foundation · Local CPULaptop CPU5 lessons (~4.8h)

JAX transformations

Available draftShared foundations
5 available

Revision v0.18.1 ·

Begin with a scalar function, then differentiate it, batch it and compile it. Compare each transformed result with a simple reference so you can explain what changed.

Keep eager and transformed outputs plus a deliberately broken assumption.

Phase study guide: What changes when you differentiate, batch or compile a function?

Start after: 01 Arrays & pure functions

Hardware and future work

CPU

Bring the phase together · Integration Project

Compare scalar, vmap, autodiff, and changed-shape JIT behavior

Checkpoint: Compose grad, vmap, and jit; explain what each transformation changes.

What to do & where to put your code: Write your implementation in projects/transforms-audit/starter/model.py (copied from the starter scaffold in projects/transforms-audit/starter/) and run projects/transforms-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3, 4

Work in projects/transforms-audit/starter/model.py. Complete stages 1 and 2 for independent calculations and changed-condition diagnosis, then retain the stage 3 evidence with --evidence-dir runs/transforms-audit. Stage 4 integrates the checks. Learner commands explicitly select the starter; --implementation solution is instructor reference execution. This bounded practice does not certify the entire phase or target hardware.

Additional project: Connect arrays, transformations and explicit state

03Core Foundation · Local CPULaptop CPU4 lessons (~4.1h)

State, randomness & control flow

Available draftShared foundations
4 available

Revision v0.18.1 ·

Make randomness, parameters and loop state explicit. Learn how to repeat an experiment without accidentally reusing randomness or mutating a caller’s state.

Record the state before and after a transition and explain every changed leaf.

Phase study guide: Have you saved everything needed to determine the next step?

Start after: 02 JAX transformations

Hardware and future work

CPU

Bring the phase together · Integration Project

A deterministic batched simulation

Checkpoint: Replay the same experiment with saved state and keys; explain the update order.

What to do & where to put your code: Write your implementation in projects/foundation-toolkit/my_toolkit.py (copied from the starter scaffold in projects/foundation-toolkit/starter/) and run projects/foundation-toolkit/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 4

Copy projects/foundation-toolkit/starter/toolkit.py to projects/foundation-toolkit/my_toolkit.py and write your code in projects/foundation-toolkit/my_toolkit.py. Run python3 projects/foundation-toolkit/tests/check.py --implementation projects/foundation-toolkit/my_toolkit.py --stage 4 from the top-level folder to verify stage 4.

04Core Foundation · Local CPULaptop CPU12 lessons (~15.5h)

Math & optimization

Available draftShared foundations
12 available

Revision v0.18.1 ·

Connect geometry and probability to the updates in a training loop. Begin with linear algebra, gradient checks and gradient descent, then investigate conditioning, curvature, likelihoods, regularization and optimizer behavior.

For the initial regression project, focus on Linear algebra and loss intuition, Gradient checking and numerical accuracy, Write gradient descent yourself, and Optimize with Optax. The other eight lessons deepen the mathematical explanation; return to them as needed.

Phase study guide: Can you explain an optimizer’s behavior from the objective and data?

Direct links for the first project: Linear algebra and loss intuitionGradient checking and numerical accuracyWrite gradient descent yourselfOptimize with Optax

Start after: 03 State, randomness & control flow

Hardware and future work

CPU

Bring the phase together · Integration Project

A fitted regression model

Checkpoint: Explain prediction geometry and loss assumptions; verify derivatives; diagnose conditioning, curvature and gradient noise; justify an optimizer using held-out evidence.

What to do & where to put your code: Write your implementation in projects/optimizer-audit/my_optimization.py (copied from the starter scaffold in projects/optimizer-audit/starter/) and run projects/optimizer-audit/tests/check.py from the top-level folder to verify each stage.

Math synthesis assessment

Open the staged project · stages 1, 2, 3, 4, 5

Complete the regression audit after the four core lessons. This deeper project follows all twelve optimization lessons and connects curvature, conditioning, gradient noise, optimizer state and regularization. Copy projects/optimizer-audit/starter/optimization.py to projects/optimizer-audit/my_optimization.py and write your code in projects/optimizer-audit/my_optimization.py. Run python3 projects/optimizer-audit/tests/check.py --implementation projects/optimizer-audit/my_optimization.py --stage 1 from the top-level folder to verify stages 1, 2, 3, 4, 5.

Section 3 of 56 courses

Part II · Complete Training Systems

Build neural networks, checkpoint recovery, Transformers, pretraining, post-training, and performance profiling.

05Training Systems · Local CPULaptop CPU5 lessons (~7.6h)

Neural network training

Available draftTraining systems
5 available

Revision v0.18.1 ·

Build the same ideas into small neural networks. Understand hidden layers, parameter state, stable objectives and evaluation before moving to a small image classifier.

Explain a loss curve together with held-out errors; a lower training loss alone is not sufficient.

Phase study guide: Did the network learn the task, or exploit an evaluation mistake?

Start after: 04 Math & optimization

Hardware and future work

CPU first; accelerator for larger experiments

Bring the phase together · Integration Project

Build and audit a neural classifier

Checkpoint: Keep train and evaluation behavior separate and reproduce metrics from a saved configuration.

What to do & where to put your code: Write your implementation in projects/mlp-classifier/my_model.py (copied from the starter scaffold in projects/mlp-classifier/starter/) and run projects/mlp-classifier/tests/check.py from the top-level folder to verify each stage.

Open the staged project

Copy projects/mlp-classifier/starter/model.py to projects/mlp-classifier/my_model.py and write your code in projects/mlp-classifier/my_model.py. Run python3 projects/mlp-classifier/tests/check.py --implementation projects/mlp-classifier/my_model.py --stage 1 from the top-level folder to verify all 3 stages.

Additional project: Reproducible JAX/Flax experiments workshop

06Training Systems · Local CPULaptop CPU6 lessons (~8.6h)

Data & checkpoint recovery

Available draftTraining systems
6 available

Revision v0.18.1 ·

Follow examples from the dataset into a training update, then save enough information to reproduce the next update after interruption. Track experiment configuration, data identity and metric histories with MLflow alongside complete checkpoints.

Compare sample IDs, optimizer memory, random state and losses after restore.

Phase study guide: Can an interrupted run make exactly the same next update?

Start after: 05 Neural network training

Hardware and future work

CPU first; TPU for system measurements

Bring the phase together · Integration Project

Resume optimizer and PRNG state across an interrupted training update

Checkpoint: Compare resumed and uninterrupted runs under a stated determinism tolerance.

What to do & where to put your code: Write your implementation in projects/recovery-audit/starter/model.py (copied from the starter scaffold in projects/recovery-audit/starter/) and run projects/recovery-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3, 4

Work in projects/recovery-audit/starter/model.py. Complete stages 1 and 2 for independent calculations and changed-condition diagnosis, then retain the stage 3 evidence with --evidence-dir runs/recovery-audit. Stage 4 integrates the checks. Learner commands explicitly select the starter; --implementation solution is instructor reference execution. This bounded practice does not certify the entire phase or target hardware.

Additional project: Ship a tracked and containerized model release

Additional project: Resume and profile a sharded training run

Additional project: Reproducible JAX/Flax experiments workshop

07Training Systems · Local CPULaptop CPU4 lessons (~6.5h)

Transformers & language models

In developmentTraining systems
4 available

Revision v0.18.1 ·

Build attention and causal masking from small arrays, then connect tokenization, mixed precision, training and generation.

Inspect what each token can attend to and count only valid targets in the loss.

Phase study guide: Can you prove that a predicted token cannot see its future?

Start after: 06 Data & checkpoint recovery

Hardware and future work

CPU for small blocks; TPU or GPU for training experiments

Bring the phase together · Integration Project

A checkpointed causal language model

Checkpoint: Verify masking, sequence handling, restoration, and generation before comparing speed.

What to do & where to put your code: Write your implementation in projects/text-harness/my_model.py (copied from the starter scaffold in projects/text-harness/starter/) and run projects/text-harness/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3, 4

Begin with token contracts and training. Complete the linked performance and deployment prerequisites before cache, export and profiling checks. Copy projects/text-harness/starter/model.py to projects/text-harness/my_model.py and write your code in projects/text-harness/my_model.py. Run python3 projects/text-harness/tests/check.py --implementation projects/text-harness/my_model.py --stage 1 from the top-level folder to verify stages 1, 2, 3, 4.

Additional project: Trace a tiny bidirectional encoder from pretraining to retrieval

08Training Systems · Local CPULaptop CPU5 lessons (~7.6h)

Performance diagnosis

Available draftTraining systems
5 available

Revision v0.18.1 ·

Measure completed work, inspect a real profiler trace and relate compiler output to memory traffic. Compare rematerialization choices while separating saved residuals, compiler memory estimates and observed latency.

Keep workload shapes, warm-up policy and synchronized timings with your device report.

Phase study guide: What measured evidence identifies the bottleneck?

Start after: 05 Neural network training

Hardware and future work

CPU for timing concepts; TPU or GPU for profiling

Bring the phase together · Integration Project

Measure CPU JIT latency and changed-shape traces with retained JAXPR evidence

Checkpoint: Separate compilation, execution, input stalls, and synchronization before claiming a speedup.

What to do & where to put your code: Write your implementation in projects/performance-audit/starter/model.py (copied from the starter scaffold in projects/performance-audit/starter/) and run projects/performance-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3, 4

Work in projects/performance-audit/starter/model.py. Complete stages 1 and 2 for independent calculations and changed-condition diagnosis, then retain the stage 3 evidence with --evidence-dir runs/performance-audit. Stage 4 integrates the checks. Learner commands explicitly select the starter; --implementation solution is instructor reference execution. This bounded practice does not certify the entire phase or target hardware.

Additional project: Resume and profile a sharded training run

17Training Systems · Local CPULaptop CPU3 lessons (~5.3h)

Self-supervised pretraining: masked and contrastive learning

In developmentTraining systems
3 available

Revision v0.18.1 ·

Train small masked-token, hidden-patch and paired-view models. Audit information leakage, reduction counts and pair identity before interpreting a falling objective.

Predict the mask, denominator or preference direction before training. Keep independent objective checks and changed-condition evidence.

Phase study guide: What learning signal can unlabelled data provide?

Start after: 07 Transformers & language models

Hardware and future work

Executed CPU synthetic objectives; large-scale and human-feedback evaluation remain separate

Bring the phase together · Integration Project

An audited pretraining and adaptation objective toolkit

Checkpoint: Complete stages 1–3 of training-methods with independent loss references, real training curves, changed masks or pairs and explicit synthetic-data limits.

What to do & where to put your code: Write your implementation in projects/training-methods/my_methods.py (copied from the starter scaffold in projects/training-methods/starter/) and run projects/training-methods/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3

Copy projects/training-methods/starter/methods.py to projects/training-methods/my_methods.py and write your code in projects/training-methods/my_methods.py. Run python3 projects/training-methods/tests/check.py --implementation projects/training-methods/my_methods.py --stage 1 from the top-level folder to verify stages 1, 2, 3.

Additional project: Trace a tiny bidirectional encoder from pretraining to retrieval

18Training Systems · Local CPULaptop CPU5 lessons (~8.8h)

Post-training: SFT, LoRA, reward models and RLHF

In developmentTraining systems
5 available

Revision v0.18.1 ·

Connect supervised demonstrations, frozen-base LoRA, reward modeling, sampled PPO and direct preference optimization. Keep the reference, behavior policy, reward model and dataset identities separate.

Predict the mask, denominator or preference direction before training. Keep independent objective checks and changed-condition evidence. SFT, LoRA, reward ranking and DPO can follow the supervised route. Before the PPO lesson, complete the explicitly linked RL rollout, policy and evaluation preparation. DPO branches from reward learning and does not require completing PPO.

Phase study guide: Which parameters and signals change during adaptation?

Start after: 17 Self-supervised pretraining: masked and contrastive learning

Hardware and future work

Executed CPU synthetic objectives; large-scale and human-feedback evaluation remain separate

Bring the phase together · Integration Project

Audit SFT, DPO and sequence PPO

Checkpoint: Complete stages 4–8 of training-methods. Retain shifted token-mask checks, frozen-base/merge evidence, preference oracles, signed PPO clipping and fixed-reference DPO comparisons.

What to do & where to put your code: Write your implementation in projects/posttraining-audit/starter/model.py (copied from the starter scaffold in projects/posttraining-audit/starter/) and run projects/posttraining-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3, 4, 5

Begin in projects/posttraining-audit/starter/model.py for the response-mask and preference-pair checks. Stages 1 and 2 compare independent calculations and changed conditions; stage 3 retains evidence, and stage 4 integrates those checks. Stage 5 uses starter/sequence_ppo.py to collect synthetic responses, fit a critic, distinguish EOS from truncation, freeze rollout snapshots across PPO epochs and retain token-level evidence. Each stage lists its own edit file and command. This local mechanics exercise does not qualify a Transformer, human preferences, the full capstone or target hardware.

Additional project: Audit pretraining and post-training objectives

Section 4 of 54 courses

Part III · Accelerator Scale, Kernels, Deployment & Operations

Partition across device meshes, author Pallas TPU kernels, ship inference services, and operate production workloads.

09Systems Specialization · CPU ReferenceCPU reference; target qualification separate4 lessons (~6.3h)

Distributed training

In developmentSpecializations
4 available

Revision v0.18.1 ·

Partition arrays and training updates, compare all-reduce with reduce-scatter, and restore a complete sharded state across an epoch boundary. Verify CPU placement and arithmetic before separate cluster validation.

Compare against a single-device reference; measure real accelerator performance separately.

Phase study guide: Is the distributed update still optimizing the same objective?

Start after: 06 Data & checkpoint recovery08 Performance diagnosis

Hardware and future work

Logical CPU devices for sharding practice; multi-host work needs a real cluster

Bring the phase together · Integration Project

A resilient sharded training experiment

Checkpoint: Explain partitioning, communication costs, and recovery behavior with measured evidence.

What to do & where to put your code: Write your implementation in projects/sharded-training/my_model.py (copied from the starter scaffold in projects/sharded-training/starter/) and run projects/sharded-training/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3, 4

This connected project also uses performance and distributed lessons. Work through its prerequisites before the full integration check; return here with the completed evidence. Copy projects/sharded-training/starter/model.py to projects/sharded-training/my_model.py and write your code in projects/sharded-training/my_model.py. Run python3 projects/sharded-training/tests/check.py --implementation projects/sharded-training/my_model.py --stage 1 from the top-level folder to verify stages 1, 2, 3, 4.

13Systems Specialization · CPU ReferenceCPU reference; target qualification separate4 lessons (~7.4h)

Pallas kernels

In developmentSpecializations
4 available

Revision v0.18.1 ·

Write actual Pallas grids and buffered pipelines, verify boundaries and precision on CPU, and prepare guarded target measurements before claiming an optimization win.

Begin with a profiled operation and independent oracle. Complete CPU semantics first; actual TPU compilation, correctness and performance require the separate target runner on supported hardware.

Phase study guide: Does a custom kernel earn its complexity?

Start after: 09 Distributed training

Hardware and future work

CPU interpretation/simulation reference; TPU target lab requires separately validated hardware

Bring the phase together · Integration Project

A checked Pallas kernel with an explicit target qualification plan

Checkpoint: Compare against a reference across shapes and justify each optimization with evidence.

What to do & where to put your code: Write your implementation in projects/kernel-audit/my_model.py (copied from the starter scaffold in projects/kernel-audit/starter/) and run projects/kernel-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project

Copy projects/kernel-audit/starter/model.py to projects/kernel-audit/my_model.py and write your code in projects/kernel-audit/my_model.py. Run python3 projects/kernel-audit/tests/check.py --implementation projects/kernel-audit/my_model.py --stage 1 from the top-level folder to verify all 3 stages.

15Systems Specialization · CPU ReferenceCPU reference; target qualification separate8 lessons (~15.7h)

Deployment, interoperability & edge AI

In developmentSpecializations
8 available

Revision v0.18.1 ·

Adapt a trained model, verify framework and export contracts, compare precision policies and measure local inference before planning server or edge capacity. Package a model service in a reproducible container and verify readiness, artifact identity and request contracts.

Preserve checkpoint, data and calibration provenance. Compare held-out inputs before and after conversion; separate local request timing from queue simulation and target-device evidence.

Phase study guide: Does the exported service preserve the model’s complete contract?

Start after: 06 Data & checkpoint recovery08 Performance diagnosis

Hardware and future work

CPU for core labs; optional framework environments and target devices for conversion and edge validation

Bring the phase together · Integration Project

Verify an inference artifact and capacity hypothesis

Checkpoint: Verify an exported prediction contract, report weight/activation precision error, and distinguish CPU request measurements from target-device evidence.

What to do & where to put your code: Write your implementation in projects/deployment-audit/my_model.py (copied from the starter scaffold in projects/deployment-audit/starter/) and run projects/deployment-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project

Copy projects/deployment-audit/starter/model.py to projects/deployment-audit/my_model.py and write your code in projects/deployment-audit/my_model.py. Run python3 projects/deployment-audit/tests/check.py --implementation projects/deployment-audit/my_model.py --stage 1 from the top-level folder to verify all 3 stages.

Additional project: Ship a tracked and containerized model release

Additional project: Convert PyTorch weights and audit Flax architecture parity

16Systems Specialization · CPU ReferenceCPU reference; target qualification separate8 lessons (~11.3h)

Workload operations

In developmentSpecializations
8 available

Revision v0.18.1 ·

Launch and observe real local jobs, detect failures and stalls, restore complete training state, verify artifact quality and rehearse rollback before planning capacity. Connect MLOps data/release gates, LLMOps evaluation/traces and ModelOps ownership/approval to the workload runbook.

Use a bounded failure drill and retain its raw event log. Check state and data identity before resuming; separate measured execution from hypothetical demand and pricing.

Phase study guide: Can you detect, explain and safely recover a failed model release?

Start after: 09 Distributed training

Hardware and future work

Executed local CPU processes; accelerator and cluster transfer require separate validation

Bring the phase together · Integration Project

An observable, restartable local workload

Checkpoint: Diagnose a failed or stalled job, restore it correctly, and justify its resource plan from recorded measurements.

What to do & where to put your code: Write your implementation in projects/workload-operations/my_model.py (copied from the starter scaffold in projects/workload-operations/starter/) and run projects/workload-operations/tests/check.py from the top-level folder to verify each stage.

Open the staged project

Copy projects/workload-operations/starter/model.py to projects/workload-operations/my_model.py and write your code in projects/workload-operations/my_model.py. Run python3 projects/workload-operations/tests/check.py --implementation projects/workload-operations/my_model.py --stage 1 from the top-level folder to verify all 5 stages.

Additional project: Ship a tracked and containerized model release

Section 5 of 54 courses

Part IV · Research & Domain Specializations

Apply differentiable programming to scientific simulations, Bayesian inference, reinforcement learning, and JAX internals.

10Domain Specialization · Local CPULaptop CPU4 lessons (~6.8h)

Scientific computing

In developmentSpecializations
4 available

Revision v0.18.1 ·

Build and differentiate a physical simulator, recover a positive rate from observations, and separate numerical accuracy, identifiability and measured execution cost.

Complete gradients, explicit state and scan first. Start with analytic references, check discretization and sensitivity, then fit parameters and defend held-out predictions.

Phase study guide: Are you fitting the physical model or the solver’s error?

Start after: 04 Math & optimization

Hardware and future work

CPU; target accelerator execution requires separate evidence

Bring the phase together · Integration Project

A differentiable inverse problem

Checkpoint: Recover a known physical parameter and report error, stability, and gradient checks.

What to do & where to put your code: Write your implementation in projects/scientific-inverse/my_model.py (copied from the starter scaffold in projects/scientific-inverse/starter/) and run projects/scientific-inverse/tests/check.py from the top-level folder to verify each stage.

Open the staged project

Copy projects/scientific-inverse/starter/model.py to projects/scientific-inverse/my_model.py and write your code in projects/scientific-inverse/my_model.py. Run python3 projects/scientific-inverse/tests/check.py --implementation projects/scientific-inverse/my_model.py --stage 1 from the top-level folder to verify all 3 stages.

11Domain Specialization · Local CPULaptop CPU4 lessons (~7.8h)

Probabilistic modeling

In developmentSpecializations
4 available

Revision v0.18.1 ·

Specify probability models, derive exact Bayesian regression, implement and diagnose HMC, and compare variational approximations with predictive evidence.

Start with likelihoods, stable losses and explicit random keys. Check a known posterior before trusting a sampler, then separate inference error from model mismatch.

Phase study guide: Which uncertainty does your model actually represent?

Start after: 04 Math & optimization

Hardware and future work

CPU; transparent JAX samplers with an optional NumPyro ecosystem bridge

Bring the phase together · Integration Project

A Bayesian model with diagnostic evidence

Checkpoint: Report sampler diagnostics and distinguish uncertainty from predictive error.

What to do & where to put your code: Write your implementation in projects/bayesian-regression/my_model.py (copied from the starter scaffold in projects/bayesian-regression/starter/) and run projects/bayesian-regression/tests/check.py from the top-level folder to verify each stage.

Open the staged project

Copy projects/bayesian-regression/starter/model.py to projects/bayesian-regression/my_model.py and write your code in projects/bayesian-regression/my_model.py. Run python3 projects/bayesian-regression/tests/check.py --implementation projects/bayesian-regression/my_model.py --stage 1 from the top-level folder to verify all 3 stages.

12Domain Specialization · Local CPULaptop CPU5 lessons (~6.8h)

Reinforcement learning

In developmentSpecializations
5 available

Revision v0.18.1 ·

Build a functional environment, collect randomized trajectories, train policy-gradient and clipped-policy agents, fit a tabular value baseline, and evaluate independent seeds.

Keep terminal, timeout and reset behavior explicit. Verify a small enumerated expectation before trusting a sampled policy gradient. Fit and check the value baseline separately, then distinguish bootstrap permission from GAE trace continuation.

Phase study guide: Is the policy improving under a fixed evaluation protocol?

Start after: 05 Neural network training

Hardware and future work

CPU; target accelerator execution requires separate evidence

Bring the phase together · Integration Project

An evaluated policy in a functional environment

Checkpoint: Compare a trained policy to baselines across independent seeds, then defend a learned value estimate and a hand-computed GAE trace at terminal and collector-cutoff boundaries.

What to do & where to put your code: Write your implementation in projects/policy-evaluation/my_model.py (copied from the starter scaffold in projects/policy-evaluation/starter/) and run projects/policy-evaluation/tests/check.py from the top-level folder to verify each stage.

Open the staged project

Copy projects/policy-evaluation/starter/model.py to projects/policy-evaluation/my_model.py and implement the stage functions there. From the workspace root, run python3 projects/policy-evaluation/tests/check.py --implementation projects/policy-evaluation/my_model.py --stage 1, then advance through --stage 5. Stage 5 is cumulative and adds a separate tabular critic and GAE; it does not replace the actor training targets.

14Domain Specialization · Local CPULaptop CPU4 lessons (~7.4h)

Autodiff & JAX internals

Available draftSpecializations
4 available

Revision v0.18.1 ·

Trace functions into jaxpr, check forward and reverse sensitivities, write exact custom derivatives and compile a small transformation with explicit supported primitives.

Keep an independent derivative reference and test changed values, higher derivatives and unsupported cases before trusting a transformation.

Phase study guide: Which mathematical operation does the transformed program represent?

Start after: 04 Math & optimization

Hardware and future work

CPU

Bring the phase together · Integration Project

Audit derivatives and a tiny transformation

Checkpoint: Explain tracing and verify the derivative rule numerically.

What to do & where to put your code: Write your implementation in projects/derivative-audit/my_model.py (copied from the starter scaffold in projects/derivative-audit/starter/) and run projects/derivative-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project

Copy projects/derivative-audit/starter/model.py to projects/derivative-audit/my_model.py and write your code in projects/derivative-audit/my_model.py. Run python3 projects/derivative-audit/tests/check.py --implementation projects/derivative-audit/my_model.py --stage 1 from the top-level folder to verify all 3 stages.

Optional Cloud TPU specialization · 3 steps · 6 lessons
Optional accelerator branch

Qualify a TPU workflow when you need it

Work through provisioning, checkpointed jobs and TPU system profiling after the laptop CPU foundations. Cloud access is not needed for the core course.

19Cloud TPU Mini-Course · Step 1 of 3Cloud TPU VM + CPU verifierBite-sized · 2 lessons (~45m)

TPU Setup: Provisioning & Runtime Verification

In developmentStart here
2 available

Revision v0.18.1 ·

Step 1 of the 3-part Cloud TPU mini-course. Choose the right TPU slice, provision a Cloud TPU VM (or Colab TPU), bootstrap ~/jax-tpu-lab/.venv with jax[tpu], verify a synchronized pmap runtime receipt, and practice clean teardown.

Complete this short 2-lesson setup course right after 00: Setup & first steps, then continue directly to 20: TPU Workflows and 21: TPU Systems.

Phase study guide: Can you provision a Cloud TPU VM, verify that JAX actually executes on its chips, and prove clean teardown?

Start after: 00 Setup & first steps

Hardware and future work

CPU topology/contract verifier; Cloud TPU VM (v5litepod-4 / v6e-4) or Colab TPU for target runs

Bring the phase together · Integration Project

Inspect CPU device inventory, BF16 storage, and collective participant arithmetic

Checkpoint: Select a single-host TPU topology, provision a Cloud TPU VM (or Colab TPU), bootstrap jax[tpu] in an isolated .venv, verify pmap execution, and confirm clean VM deletion.

What to do & where to put your code: Write your implementation in projects/tpu-runtime-audit/starter/model.py (copied from the starter scaffold in projects/tpu-runtime-audit/starter/) and run projects/tpu-runtime-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3, 4

Work in projects/tpu-runtime-audit/starter/model.py. Complete stages 1 and 2 for independent calculations and changed-condition diagnosis, then retain the stage 3 evidence with --evidence-dir runs/tpu-runtime-audit. Stage 4 integrates the checks. Learner commands explicitly select the starter; --implementation solution is instructor reference execution. This bounded practice does not certify the entire phase or target hardware.

Additional project: Operate and recover a local JAX workload

20Cloud TPU Mini-Course · Step 2 of 3Cloud TPU VM + CPU verifierBite-sized · 2 lessons (~45m)

TPU Workflows: Launching Jobs & Checkpointed Experiments

In developmentStart here
2 available

Revision v0.18.1 ·

Step 2 of the 3-part Cloud TPU mini-course. Rehearse checkpointed jobs on CPU, then use only source-reviewed TPU targets with matching backend and dependency contracts. Separate single-host receipts, bounded multi-host recovery, and broader multi-host training qualification.

Do not run arbitrary lesson scripts on TPU. distributed-02 is CPU-only; inspect every other source before considering a target. Existing receipts qualify only their named programs and recorded shapes, environments, and topologies: the one-device v5e1 launcher and precision target, the four-device single-host collective target, and the sixteen-device/four-worker synthetic checkpoint-recovery target. That multi-host receipt qualifies its recovery path only; it does not establish general multi-host training, worker-loss or elastic recovery, production model quality, or compatibility for other lessons.

Phase study guide: Can you launch a JAX job over SSH, separate XLA warmup from steady-state timing, and resume cleanly after interruption?

Start after: 19 TPU Setup: Provisioning & Runtime Verification

Hardware and future work

CPU workflow planner; target execution only for source-reviewed programs with matching backend, dependencies, topology, and data-ownership contracts

Bring the phase together · Integration Project

Gate simulated multi-worker checkpoint publication on complete receipts

Checkpoint: Classify each exact program before staging. Route CPU-only and unreviewed scripts to CPU or inspection; use a TPU target only when its backend, pinned dependencies, topology, and data ownership are reviewed. Treat --worker=all discovery as topology evidence, not distributed training.

What to do & where to put your code: Write your implementation in projects/tpu-workflow-audit/starter/model.py (copied from the starter scaffold in projects/tpu-workflow-audit/starter/) and run projects/tpu-workflow-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3, 4

Work in projects/tpu-workflow-audit/starter/model.py. Complete stages 1 and 2 for independent calculations and changed-condition diagnosis, then retain the stage 3 evidence with --evidence-dir runs/tpu-workflow-audit. Stage 4 integrates the checks. Learner commands explicitly select the starter; --implementation solution is instructor reference execution. This bounded practice does not certify the entire phase or target hardware.

Additional project: Operate and recover a local JAX workload

21Cloud TPU Mini-Course · Step 3 of 3Cloud TPU VM + CPU verifierBite-sized · 2 lessons (~50m)

TPU Systems: Generations, Memory, Precision & XProf

In developmentStart here
2 available

Revision v0.18.1 ·

Step 3 of the 3-part Cloud TPU mini-course. Compare TPU v5e, v5p, and v6e specifications and ridge points, budget training and KV-cache HBM before provisioning, measure BF16 and INT8 numerical error, and capture warmed XProf traces.

After completing this third TPU mini-course, you can plan a candidate accelerator workload and interpret hardware evidence. Before running a program on TPU, review its exact source, dependencies, backend assertion, topology, and data ownership; then qualify that program and configuration with target-specific evidence. Existing receipts do not authorize other accelerator phases by association.

Phase study guide: How do you size TPU HBM capacity, bound BF16/INT8 precision error, and capture a clean XProf trace?

Start after: 20 TPU Workflows: Launching Jobs & Checkpointed Experiments

Hardware and future work

CPU numerical & profiler harness; Cloud TPU VM for target XProf traces

Bring the phase together · Integration Project

Estimate optimizer-state bytes and calculate supplied-rate scaling efficiency

Checkpoint: Compare published TPU roofline estimates, budget training and KV-cache HBM, compare BF16 result-dtype policies and an INT8 weight-dequantization reference, and retain a warmed profiler trace with its observed backend.

What to do & where to put your code: Write your implementation in projects/tpu-systems-audit/starter/model.py (copied from the starter scaffold in projects/tpu-systems-audit/starter/) and run projects/tpu-systems-audit/tests/check.py from the top-level folder to verify each stage.

Open the staged project · stages 1, 2, 3, 4

Work in projects/tpu-systems-audit/starter/model.py. Complete stages 1 and 2 for independent calculations and changed-condition diagnosis, then retain the stage 3 evidence with --evidence-dir runs/tpu-systems-audit. Stage 4 integrates the checks. Learner commands explicitly select the starter; --implementation solution is instructor reference execution. This bounded practice does not certify the entire phase or target hardware.

Additional project: Operate and recover a local JAX workload

Jump to a course step

104 authored lessons with CPU companions · 31 staged projects · 11 synthesis assessments

Put your learning to the test

After a project, use its synthesis assessment to explain your decisions, check changed inputs and collect evidence for your portfolio.