Phase 16 / Workload operations
ModelOps: own, approve, roll back and retire a model
Revision v0.7.0 · Updated since your last visit. Review the changes; your progress is saved.
Lesson · about 65 minutes · CPU practice
Bind a release decision to immutable evidence, reject changed or expired approvals and preserve active selection after failure.
Phase study guide: preparation, milestones and project evidence
Before you start
Before this: LLMOps: version prompts, evaluate behavior and trace requests
Hardware: CPU
Setup help: Python, notebook, or Cloud TPU VM
Use the tested CPU packages (the recorded environment uses Python 3.14.3). Other Python versions and accelerator environments require separate checks. The optional Cloud TPU specialization explains the target-specific execution boundaries. Logical CPU devices practice partitioning; they do not simulate TPU performance.
Download the repository ZIP or use your existing checkout. Extract it and open a terminal in the folder containing requirements-cpu.txt. The beginner ZIP is separate; these lessons use the full repository folder.
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-cpu.txtpy -3 -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements-cpu.txtAfter activation, run the complete worked example from that same folder:
python phases/16-operations/08-modelops-ownership-approval-and-retirement/code/main.py.\.venv\Scripts\python.exe phases/16-operations/08-modelops-ownership-approval-and-retirement/code/main.pyFor an incremental build, create main.py in your own practice folder, append each build block in order, and run python main.py with this environment active. Keep that file separate from the downloaded reference solution.
Download this lesson’s notebook. Open it with a Python kernel using the course environment. Run cells from the top after restarting the kernel; experiments depend on earlier definitions.
For local Jupyter, first install the course requirements using the Local Python steps, then use your environment’s Python to start Jupyter:
python -m pip install jupyterlab
python -m jupyter lab.\.venv\Scripts\python.exe -m pip install jupyterlab
.\.venv\Scripts\python.exe -m jupyter labSelect the downloaded operations-08.ipynb file in JupyterLab. The notebook contains reference solutions; make your prediction and attempt each practice task before opening them. Restart the kernel before changing logical device setup.
The worked reference for this lesson is CPU-validated. Some references explicitly configure logical CPU devices; setting JAX_PLATFORMS does not adapt them to a TPU. Follow the execution matrix and reviewed multi-host example before attempting a target run.
For accelerator preparation, study TPU setup, observed runtime receipts, checkpointed jobs and precision and retained traces. These target paths remain unqualified on physical TPU hardware until an actual receipt is recorded. Cloud provisioning is optional for CPU learning.
Check: Accepted original / changed artifact / expired approval / wrong target: [True, False, False, False] All failed gates preserved the selected version. This fixture does not authenticate reviewers.
Revisit the guided CPU setup · Open 19: TPU Setup. Package instructions are CPU instructions; Windows and accelerator environments need their own validation.
What you will be able to do
- Inventory the complete release
- Bind review to immutable evidence
- Make failed transitions leave no side effects
- Separate selection, rollout and monitoring
The problem
A model passed evaluation last week. Someone changed its preprocessing or image today and reused the old approval. Let’s bind a review to the exact release bundle, reject stale decisions and preserve the current version when a gate fails.
The idea
Model governance connects ownership, evaluation, approval and lifecycle transitions. An approval should identify exactly what was reviewed, where it may be used and when it remains valid. A label saying approved is not enough.
Approval applies to a specific release
Suppose a model passes review, then its tokenizer or weights change. Reusing the old approval would apply a decision to an object that was never evaluated under that identity. Bind approvals to the relevant model, processor, configuration and evaluation records.
Specify transitions for activation, expiration, rejection and retirement. Invalid transitions should leave the active state unchanged rather than partially applying a release. A retired artifact may remain archived for reproducibility while no longer being eligible for new use.
The acceptance bars are summaries of discrete policy decisions. Read the underlying state and reason for each result. The fixture demonstrates decision logic, not completion of an organization's actual review process.
Inventory the complete release
Record the model artifact, data/evaluation identity, image digest, preprocessing/signature, precision, intended target and responsible owner. The fixture compresses that into a small bundle with model, data and evaluation hashes, image digest, owner, target and passing result. Real release records should also identify training/code versions, documented limitations, dependency/license obligations where applicable, and how to reach the owner. Registry tags help discovery but are not access control.
Bind review to immutable evidence
Hash the whole release bundle, then record who reviewed that hash and when the approval expires. If the model, evaluation, image, target or owner changes, the hash changes and the old approval must not authorize the new bundle. The local function receives a reviewer string as a fixture: it does not authenticate that person. A production workflow needs trusted identity, authorization, protected evidence and an audit log. A checksum proves equality of bytes, not authority or model quality.
Make failed transitions leave no side effects
The example validates the bundle and approval before writing the active pointer. It uses an atomic rename for a single local writer. A failed validation must leave the previous selection byte-for-byte unchanged. With multiple deployment controllers, use a transactional compare-and-swap or a deployment system with concurrency control; atomic file replacement alone does not prevent competing writers from losing updates.
Separate selection, rollout and monitoring
A registry alias chooses a version. A rollout changes serving processes and traffic. Monitor the loaded model/image identity as well as quality and service signals to detect disagreement. Canary and shadow releases answer different questions: a canary serves a limited live population, while shadow traffic runs a candidate without using its answer. Define stop conditions, traffic assignment and comparable metrics before observing a favorable curve.
Rollback needs its own valid target
Keeping a previous pointer is useful but does not guarantee the old artifact still loads, remains approved or matches the current input contract. Revalidate compatibility, artifact availability and approval before rolling back. Practice a rejected rollout and a successful rollback using a known previous bundle. Retire a model by stopping new traffic, removing inappropriate aliases/access and applying the organization’s retention rules to evidence; do not silently delete the only audit trail.
Work through the four decisions
The fixture accepts the original bundle at time , then rejects changed model bytes, an approval expiring at , and a different target. The valid interval is : expiration is exclusive. The image digest in this companion is explicitly a format-valid test value. Replace it with the observed container image identity in the connected project before treating the record as deployment evidence.
Run changed-bundle and expiry checks
# Run run changed-bundle and expiry checks using the course Python environment
python projects/engineering-release/tests/check.py --implementation solution --stage 3Expected:
The gate accepts only the matching, current bundle and every failure preserves the active pointer.
Only the reviewed release passes
Predict before looking: Should changing only the target preserve the approval?
Read the figure
The horizontal categories are four attempted transitions. Height means the local contract accepted the transition; height means it rejected it. Only the original reviewed bundle passes. Changed model content, expiration and target mismatch fail, and the program verifies that none changed the active pointer. Zero bars are observed rejections, not missing measurements.
Connect it to the computation
This is a deterministic policy test, not a model-performance comparison or proof of authenticated approval. Use the real model/image identities and a trusted review process when transferring it to deployment.
Open full-size figureDownload data and provenance
The notebook contains the data experiment, plotting code, and recorded figure output. Rerun it to test your changes.
Where to put & run this code: Work in phases/16-operations/08-modelops-ownership-approval-and-retirement/code/main.py (or downloaded exercises/operations-08.py / exercises/operations-08.ipynb) and execute from the repository root with python3 phases/16-operations/08-modelops-ownership-approval-and-retirement/code/main.py.
1 Prepare the explicit contract
Create main.py and add this setup block. Continue with the next block in the same file.
# Step 1 — Prepare the explicit contract: Record the model artifact, data/evaluation identity, image digest,...
import hashlib, json, math, tempfile
from pathlib import PathRecord the model artifact, data/evaluation identity, image digest, preprocessing/signature, precision, intended target and responsible owner. The fixture compresses that into a small bundle with model, data and evaluation hashes, image digest, owner, target and passing result. Real release records should also identify training/code versions, documented limitations, dependency/license obligations where applicable, and how to reach the owner. Registry tags help discovery but are not access control.
2 Run and inspect the controlled experiment
Append this block, run main.py in the course environment, and retain the actual output.
# Step 2 — Run and inspect the controlled experiment: A registry alias chooses a version.
def digest(value):
return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(',', ':'), allow_nan=False).encode()).hexdigest()
def approval_for(bundle, actor, now, expires):
# Guard input contract (`not actor or expires <= now`) and fail fast if violated.
if not actor or expires <= now:
raise ValueError('invalid reviewer or expiry')
return {'bundle_hash': digest(bundle), 'reviewer': actor, 'approved_at': now, 'expires_at': expires}
def verify_release(bundle, approval, now, target):
required = {'model_hash', 'data_hash', 'evaluation_hash', 'image_digest', 'owner', 'target', 'passed'}
# Guard input contract (`set(bundle) != required or bundle['passed'] is not True or (not bundle['owner']) or (bundle['target'] != target)`) and fail fast if violated.
if set(bundle) != required or bundle['passed'] is not True or not bundle['owner'] or bundle['target'] != target:
raise ValueError('release contract rejected')
for key in ['model_hash', 'data_hash', 'evaluation_hash']:
value = bundle[key]
# Guard input contract (`not isinstance(value, str) or len(value) != 64 or any((c not in '0123456789abcdef' for c in value))`) and fail fast if violated.
if not isinstance(value, str) or len(value) != 64 or any(c not in '0123456789abcdef' for c in value):
raise ValueError('invalid content digest')
image = bundle['image_digest']
# Guard input contract (`not isinstance(image, str) or not image.startswith('sha256:') or len(image) != 71 or any((c not in '0123456789abcdef' for c in image[7:]))`) and fail fast if violated.
if not isinstance(image, str) or not image.startswith('sha256:') or len(image) != 71 or any(c not in '0123456789abcdef' for c in image[7:]):
raise ValueError('resolve a real container digest before approval')
# Guard input contract (`approval.get('bundle_hash') != digest(bundle) or not approval.get('reviewer')`) and fail fast if violated.
if approval.get('bundle_hash') != digest(bundle) or not approval.get('reviewer'):
raise ValueError('approval belongs to another release')
# Guard input contract (`not approval['approved_at'] <= now < approval['expires_at']`) and fail fast if violated.
if not approval['approved_at'] <= now < approval['expires_at']:
raise ValueError('approval is not currently valid')
return True
def activate(store, version, bundle, approval, now, target):
"""A single-writer local fixture; a deployed service needs concurrency control and auth."""
verify_release(bundle, approval, now, target)
store = Path(store)
store.mkdir(parents=True, exist_ok=True)
pointer = store / 'active.json'
previous = json.loads(pointer.read_text())['current'] if pointer.exists() else None
selected = {'current': version, 'previous': previous, 'bundle_hash': digest(bundle)}
temporary = store / 'active.tmp'
temporary.write_text(json.dumps(selected))
temporary.replace(pointer)
return selected
# Compute deterministic cryptographic digest `bundle` for provenance verification.
bundle = dict(model_hash=digest({'weight':2,'bias':1}), data_hash=digest(['fixture-v1']), evaluation_hash=digest({'high_slice_mse':.01}), image_digest='sha256:'+'1'*64, owner='course-team', target='cpu-demo', passed=True)
# This image digest is a format-valid fixture. Real releases must use the observed build digest.
approval = approval_for(bundle,'reviewer-fixture',now=100,expires=200)
checks = [True]
# Create an isolated temporary directory to run and inspect artifacts safely:
with tempfile.TemporaryDirectory(prefix='modelops-') as folder:
active = activate(folder,'model-v1',bundle,approval,150,'cpu-demo')
pointer = Path(folder)/'active.json'
before = pointer.read_bytes()
for candidate,at,target in [(dict(bundle,model_hash=digest({'weight':3})),150,'cpu-demo'), (bundle,200,'cpu-demo'), (bundle,150,'edge-device')]:
# Run the boundary check and catch the expected exception:
try:
activate(folder,'rejected-v2',candidate,approval,at,target)
except ValueError:
checks.append(False)
else:
raise AssertionError('invalid release activated')
assert pointer.read_bytes() == before
assert checks == [True,False,False,False]
print('Accepted original / changed artifact / expired approval / wrong target:',checks)
print('All failed gates preserved the selected version. This fixture does not authenticate reviewers.')Code building blocks in this step
digest(...)— Calldigestwith your updated parameters or inputs from this lesson's workspace.hashlib.sha256(...)— Callhashlib.sha256with your updated parameters or inputs from this lesson's workspace.assert condition— Verify that the observed output shape, status, or numerical value satisfies the contract.
A registry alias chooses a version. A rollout changes serving processes and traffic. Monitor the loaded model/image identity as well as quality and service signals to detect disagreement. Canary and shadow releases answer different questions: a canary serves a limited live population, while shadow traffic runs a candidate without using its answer. Define stop conditions, traffic assignment and comparable metrics before observing a favorable curve.
3 Step 3: Verify invariants on the completed state
Run the final shape and numerical assertions to confirm the state built in Steps 1 and 2.
# Verify that only the reviewed release passed and all rejected transitions preserved active.json:
assert checks == [True, False, False, False]
print("Accepted original / changed artifact / expired approval / wrong target:", checks)Code building blocks in this step
assert condition— Verify that the observed output shape, status, or numerical value satisfies the contract.
Checking these invariants confirms the computation is ready for the full worked experiment.
View the cumulative build in main.py
These blocks append in order to the same file.
# Step 1 — Prepare the explicit contract: Record the model artifact, data/evaluation identity, image digest,...
import hashlib, json, math, tempfile
from pathlib import Path
# Step 2 — Run and inspect the controlled experiment: A registry alias chooses a version.
def digest(value):
return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(',', ':'), allow_nan=False).encode()).hexdigest()
def approval_for(bundle, actor, now, expires):
# Guard input contract (`not actor or expires <= now`) and fail fast if violated.
if not actor or expires <= now:
raise ValueError('invalid reviewer or expiry')
return {'bundle_hash': digest(bundle), 'reviewer': actor, 'approved_at': now, 'expires_at': expires}
def verify_release(bundle, approval, now, target):
required = {'model_hash', 'data_hash', 'evaluation_hash', 'image_digest', 'owner', 'target', 'passed'}
# Guard input contract (`set(bundle) != required or bundle['passed'] is not True or (not bundle['owner']) or (bundle['target'] != target)`) and fail fast if violated.
if set(bundle) != required or bundle['passed'] is not True or not bundle['owner'] or bundle['target'] != target:
raise ValueError('release contract rejected')
for key in ['model_hash', 'data_hash', 'evaluation_hash']:
value = bundle[key]
# Guard input contract (`not isinstance(value, str) or len(value) != 64 or any((c not in '0123456789abcdef' for c in value))`) and fail fast if violated.
if not isinstance(value, str) or len(value) != 64 or any(c not in '0123456789abcdef' for c in value):
raise ValueError('invalid content digest')
image = bundle['image_digest']
# Guard input contract (`not isinstance(image, str) or not image.startswith('sha256:') or len(image) != 71 or any((c not in '0123456789abcdef' for c in image[7:]))`) and fail fast if violated.
if not isinstance(image, str) or not image.startswith('sha256:') or len(image) != 71 or any(c not in '0123456789abcdef' for c in image[7:]):
raise ValueError('resolve a real container digest before approval')
# Guard input contract (`approval.get('bundle_hash') != digest(bundle) or not approval.get('reviewer')`) and fail fast if violated.
if approval.get('bundle_hash') != digest(bundle) or not approval.get('reviewer'):
raise ValueError('approval belongs to another release')
# Guard input contract (`not approval['approved_at'] <= now < approval['expires_at']`) and fail fast if violated.
if not approval['approved_at'] <= now < approval['expires_at']:
raise ValueError('approval is not currently valid')
return True
def activate(store, version, bundle, approval, now, target):
"""A single-writer local fixture; a deployed service needs concurrency control and auth."""
verify_release(bundle, approval, now, target)
store = Path(store)
store.mkdir(parents=True, exist_ok=True)
pointer = store / 'active.json'
previous = json.loads(pointer.read_text())['current'] if pointer.exists() else None
selected = {'current': version, 'previous': previous, 'bundle_hash': digest(bundle)}
temporary = store / 'active.tmp'
temporary.write_text(json.dumps(selected))
temporary.replace(pointer)
return selected
# Compute deterministic cryptographic digest `bundle` for provenance verification.
bundle = dict(model_hash=digest({'weight':2,'bias':1}), data_hash=digest(['fixture-v1']), evaluation_hash=digest({'high_slice_mse':.01}), image_digest='sha256:'+'1'*64, owner='course-team', target='cpu-demo', passed=True)
# This image digest is a format-valid fixture. Real releases must use the observed build digest.
approval = approval_for(bundle,'reviewer-fixture',now=100,expires=200)
checks = [True]
# Create an isolated temporary directory to run and inspect artifacts safely:
with tempfile.TemporaryDirectory(prefix='modelops-') as folder:
active = activate(folder,'model-v1',bundle,approval,150,'cpu-demo')
pointer = Path(folder)/'active.json'
before = pointer.read_bytes()
for candidate,at,target in [(dict(bundle,model_hash=digest({'weight':3})),150,'cpu-demo'), (bundle,200,'cpu-demo'), (bundle,150,'edge-device')]:
# Run the boundary check and catch the expected exception:
try:
activate(folder,'rejected-v2',candidate,approval,at,target)
except ValueError:
checks.append(False)
else:
raise AssertionError('invalid release activated')
assert pointer.read_bytes() == before
assert checks == [True,False,False,False]
print('Accepted original / changed artifact / expired approval / wrong target:',checks)
print('All failed gates preserved the selected version. This fixture does not authenticate reviewers.')
# Verify that only the reviewed release passed and all rejected transitions preserved active.json:
assert checks == [True, False, False, False]
print("Accepted original / changed artifact / expired approval / wrong target:", checks)Run the example
python3 phases/16-operations/08-modelops-ownership-approval-and-retirement/code/main.pyAbout this command
This shortcut uses an activated macOS/Linux course environment in the repository folder. For Windows, use the explicit interpreter command in “Setup help” above. The script includes worked examples and reference solutions.
Inspect the complete assembled example
# Step 1 — Prepare the explicit contract: Record the model artifact, data/evaluation identity, image digest,...
import hashlib, json, math, tempfile
from pathlib import Path
# Step 2 — Run and inspect the controlled experiment: A registry alias chooses a version.
def digest(value):
return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(',', ':'), allow_nan=False).encode()).hexdigest()
def approval_for(bundle, actor, now, expires):
# Guard input contract (`not actor or expires <= now`) and fail fast if violated.
if not actor or expires <= now:
raise ValueError('invalid reviewer or expiry')
return {'bundle_hash': digest(bundle), 'reviewer': actor, 'approved_at': now, 'expires_at': expires}
def verify_release(bundle, approval, now, target):
required = {'model_hash', 'data_hash', 'evaluation_hash', 'image_digest', 'owner', 'target', 'passed'}
# Guard input contract (`set(bundle) != required or bundle['passed'] is not True or (not bundle['owner']) or (bundle['target'] != target)`) and fail fast if violated.
if set(bundle) != required or bundle['passed'] is not True or not bundle['owner'] or bundle['target'] != target:
raise ValueError('release contract rejected')
for key in ['model_hash', 'data_hash', 'evaluation_hash']:
value = bundle[key]
# Guard input contract (`not isinstance(value, str) or len(value) != 64 or any((c not in '0123456789abcdef' for c in value))`) and fail fast if violated.
if not isinstance(value, str) or len(value) != 64 or any(c not in '0123456789abcdef' for c in value):
raise ValueError('invalid content digest')
image = bundle['image_digest']
# Guard input contract (`not isinstance(image, str) or not image.startswith('sha256:') or len(image) != 71 or any((c not in '0123456789abcdef' for c in image[7:]))`) and fail fast if violated.
if not isinstance(image, str) or not image.startswith('sha256:') or len(image) != 71 or any(c not in '0123456789abcdef' for c in image[7:]):
raise ValueError('resolve a real container digest before approval')
# Guard input contract (`approval.get('bundle_hash') != digest(bundle) or not approval.get('reviewer')`) and fail fast if violated.
if approval.get('bundle_hash') != digest(bundle) or not approval.get('reviewer'):
raise ValueError('approval belongs to another release')
# Guard input contract (`not approval['approved_at'] <= now < approval['expires_at']`) and fail fast if violated.
if not approval['approved_at'] <= now < approval['expires_at']:
raise ValueError('approval is not currently valid')
return True
def activate(store, version, bundle, approval, now, target):
"""A single-writer local fixture; a deployed service needs concurrency control and auth."""
verify_release(bundle, approval, now, target)
store = Path(store)
store.mkdir(parents=True, exist_ok=True)
pointer = store / 'active.json'
previous = json.loads(pointer.read_text())['current'] if pointer.exists() else None
selected = {'current': version, 'previous': previous, 'bundle_hash': digest(bundle)}
temporary = store / 'active.tmp'
temporary.write_text(json.dumps(selected))
temporary.replace(pointer)
return selected
# Compute deterministic cryptographic digest `bundle` for provenance verification.
bundle = dict(model_hash=digest({'weight':2,'bias':1}), data_hash=digest(['fixture-v1']), evaluation_hash=digest({'high_slice_mse':.01}), image_digest='sha256:'+'1'*64, owner='course-team', target='cpu-demo', passed=True)
# This image digest is a format-valid fixture. Real releases must use the observed build digest.
approval = approval_for(bundle,'reviewer-fixture',now=100,expires=200)
checks = [True]
# Create an isolated temporary directory to run and inspect artifacts safely:
with tempfile.TemporaryDirectory(prefix='modelops-') as folder:
active = activate(folder,'model-v1',bundle,approval,150,'cpu-demo')
pointer = Path(folder)/'active.json'
before = pointer.read_bytes()
for candidate,at,target in [(dict(bundle,model_hash=digest({'weight':3})),150,'cpu-demo'), (bundle,200,'cpu-demo'), (bundle,150,'edge-device')]:
# Run the boundary check and catch the expected exception:
try:
activate(folder,'rejected-v2',candidate,approval,at,target)
except ValueError:
checks.append(False)
else:
raise AssertionError('invalid release activated')
assert pointer.read_bytes() == before
assert checks == [True,False,False,False]
print('Accepted original / changed artifact / expired approval / wrong target:',checks)
print('All failed gates preserved the selected version. This fixture does not authenticate reviewers.')Expected result: Accepted original / changed artifact / expired approval / wrong target: [True, False, False, False] All failed gates preserved the selected version. This fixture does not authenticate reviewers.
Recorded reference execution
Reference output captured from executing the complete lesson script (code/main.py) and its assertions in the course CPU environment.
Accepted original / changed artifact / expired approval / wrong target: [True, False, False, False]
All failed gates preserved the selected version. This fixture does not authenticate reviewers.
Accepted original / changed artifact / expired approval / wrong target: [True, False, False, False]
Accepted original / changed artifact / expired approval / wrong target: [True, False, False, False]
All failed gates preserved the selected version. This fixture does not authenticate reviewers.
Expiry is exclusive: 199 accepted, 200 rejected.
Changed evaluation invalidates the old approval.
Ownership change requires renewed review.
PASS: operations-08
Download execution record · The notebook includes executed cells and figure outputs. Compare your run with these values; timings depend on the machine.
Test the exact expiry boundary
# Experiment — Test the exact expiry boundary: Boundary conventions must be explicit.
assert verify_release(bundle,approval,199,'cpu-demo')
# Run the boundary check and catch the expected exception:
try:
verify_release(bundle,approval,200,'cpu-demo')
except ValueError:
print('Expiry is exclusive: 199 accepted, 200 rejected.')
else:
raise AssertionError('expired approval accepted')Expected: Time 199 is accepted; 200 is rejected.
Boundary conventions must be explicit. Production clocks and identity systems need their own operational guarantees.
Make it yours
Where to put your code: Append your solution at the bottom of phases/16-operations/08-modelops-ownership-approval-and-retirement/code/main.py (or inside the Make it yours cell of exercises/operations-08.ipynb) and run python3 phases/16-operations/08-modelops-ownership-approval-and-retirement/code/main.py to verify your prediction before opening the reference solution.
Change the evaluation hash while keeping the model unchanged, and prove the old approval no longer applies.
Show a writing guide and starter scaffold
How to write this exercise code: step-by-step recipe & starter scaffold
1. Key functions & syntax to use:
digest(...)— Calldigestwith your updated parameters or inputs from this lesson's workspace.verify_release(...)— Callverify_releasewith your updated parameters or inputs from this lesson's workspace.
2. Step-by-step implementation plan:
- Run the boundary check and catch the expected exception:
# Exercise solution: Change the evaluation hash while keeping the model unchanged, and...
rechecked = dict(...) # TODO: compute rechecked
# Run the boundary check and catch the expected exception:
try:
verify_release(rechecked,approval,150,'cpu-demo')
except ValueError:
print('Changed evaluation invalidates the old approval.')
else:
raise AssertionError('stale approval accepted')
Reveal the reference solution
# Exercise solution: Change the evaluation hash while keeping the model unchanged, and...
rechecked=dict(bundle,evaluation_hash=digest({'high_slice_mse':9.}))
# Run the boundary check and catch the expected exception:
try:
verify_release(rechecked,approval,150,'cpu-demo')
except ValueError:
print('Changed evaluation invalidates the old approval.')
else:
raise AssertionError('stale approval accepted')Review a changed owner
transfer
Change the responsible owner and check whether the old bundle approval can be reused.
Show a writing guide and starter scaffold
How to write this practice solution: step-by-step recipe & starter scaffold
1. Key functions & syntax to use:
owner(...)— Callownerwith your updated parameters or inputs from this lesson's workspace.digest(...)— Calldigestwith your updated parameters or inputs from this lesson's workspace.assert condition— Verify that the observed output shape, status, or numerical value satisfies the contract.
2. Step-by-step implementation plan:
- Review a changed owner (transfer): Ownership is an operational responsibility, so this policy...
- Run the boundary check and catch the expected exception:
# Review a changed owner (transfer): Ownership is an operational responsibility, so this policy...
changed_owner = dict(...) # TODO: compute changed_owner
assert digest(changed_owner) # TODO: complete assertion check
# Run the boundary check and catch the expected exception:
try:
verify_release(changed_owner,approval,150,'cpu-demo')
except ValueError:
print('Ownership change requires renewed review.')
else:
raise AssertionError('changed owner accepted')
Show a hint
The owner is part of the reviewed release identity.
Reveal reference solution and reasoning
# Review a changed owner (transfer): Ownership is an operational responsibility, so this policy...
changed_owner=dict(bundle,owner='different-team')
assert digest(changed_owner)!=approval['bundle_hash']
# Run the boundary check and catch the expected exception:
try:
verify_release(changed_owner,approval,150,'cpu-demo')
except ValueError:
print('Ownership change requires renewed review.')
else:
raise AssertionError('changed owner accepted')Ownership is an operational responsibility, so this policy requires review after it changes. Organizations may choose different policies, but the rule must be explicit.
Diagnose the result
Compare the whole bundle hash and target first, then time boundaries and reviewer authority. Never bypass a rejected rollout by editing a passed flag or pointing the alias manually. Keep the rejected transition in the incident record.
Carry forward
- Inventory the complete release
- Separate selection, rollout and monitoring
- Work through the four decisions
Keep your evidence
Keep the reviewed bundle hash, owner and target, expiry boundary, failed-transition pointer equality and changed-owner rejection. Explain why the local reviewer field is not authenticated approval.
Save your modified code, its output, and an explanation of your prediction. Checkpoint answers and exercise evidence are tracked separately.
Saved in this browser. Export or back up your portfolio anytime in My learning.
Verified on local CPU. Stage to a Cloud TPU VM via Cloud TPU setup to record accelerator execution receipts (TODO: add automated Cloud TPU CI execution receipts).
Primary references
- MLflow experiment tracking ↗
- MLflow model registry workflows ↗
- Kubernetes deployment rollout and rollback ↗
Phase 16 Integration Project
Bring this phase together: An observable, restartable local workload
Checkpoint to prove: Diagnose a failed or stalled job, restore it correctly, and justify its resource plan from recorded measurements.
Copy projects/workload-operations/starter/model.py to projects/workload-operations/my_model.py and write your code in projects/workload-operations/my_model.py. Run python3 projects/workload-operations/tests/check.py --implementation projects/workload-operations/my_model.py --stage 1 from the top-level folder to verify all 5 stages.