Course · Probabilistic modelingPhase 11 · Study guide

Which uncertainty does your model actually represent?

Domain Specialization · Local CPULaptop CPU4 lessons (~7.8h)

Revision v0.7.0 ·

Use an exactly solvable Bayesian regression to check inference. Separate uncertainty about weights, uncertainty about a future observation, and mismatch between the assumed likelihood and the data.

Preparation: Math & optimization

Runtime: CPU; transparent JAX samplers with an optional NumPyro ecosystem bridge

Check your starting point

Should an interval for a new noisy observation equal an interval for the latent mean prediction?

Compare your reasoning

No. The observation includes additional likelihood noise. Under the independent Gaussian model, add its variance to the latent predictive variance before taking a square root.

If this is unfamiliar, work through Build a Bayesian regression before attempting the project.

Build in stages

1. Build an exact reference

Keep distribution units and normalization, derive a posterior in a small solvable case and compare latent versus observation uncertainty.

2. Challenge approximate inference

Compare sampler moments with the exact target, inspect multiple-chain diagnostics and measure the covariance lost by a diagonal approximation. Use predictive residuals to challenge the likelihood itself.

Try a changed condition

Make a prediction before opening the reasoning. Keep your answer and the evidence you would collect.

Chains agree closely, but prediction residuals bend systematically with the input. What should be changed first?

Compare an approach

Investigate the model’s mean function and observation assumptions. Agreement between chains concerns exploration of the chosen posterior; it does not certify that the chosen model describes the data.

Diagnose a likely failure

Symptom: Intervals are narrow and diagnostics look reassuring despite poor predictions.

Check next: Check whether you plotted latent or observation uncertainty, and whether the likelihood or mean function misses structure.

Decide what is ready

Use bayesian-regression. Retain analytic posterior checks, sampler/approximation diagnostics and a predictive model-mismatch case, with uncertainty assumptions stated.

Where to put your project code

Write your code in: projects/bayesian-regression/my_model.py (copy projects/bayesian-regression/starter/model.py first: cp projects/bayesian-regression/starter/model.py projects/bayesian-regression/my_model.py).

Verify your implementation: python3 projects/bayesian-regression/tests/check.py --stage all --impl projects/bayesian-regression/my_model.py

Run the staged project checks on your implementation in projects/bayesian-regression/my_model.py and record your transfer experiment results in your portfolio.

Open A Bayesian model with diagnostic evidence

What still needs further work

General hierarchical models and robust modern diagnostic workflows need further examples. The instructional sampler is not a replacement for validated inference libraries.