Systems Specialization · CPU ReferenceCPU reference; target qualification separateLesson 2 of 8 in Phase 15

Phase 15 / Deployment, interoperability & edge AI

Move models between JAX, Keras, TensorFlow and PyTorch

Revision v0.18.1 ·

Lesson · about 110 minutes · CPU practice

Verify parameter layouts and inference semantics before switching frameworks.

Phase study guide: preparation, milestones and project evidence

Before you start

Before this: Adapt a pretrained model and choose a post-training objective
Hardware: CPU; optional framework and edge-device labs have separate requirements

Setup help: Python, notebook, or Cloud TPU VM

Use the tested CPU packages (the recorded environment uses Python 3.14.3). Other Python versions require separate checks. Hardware receipts apply only to their recorded source, environment, hardware, and cases; they do not qualify every lesson or TPU configuration. See the TPU performance and hardware evidence guide. The optional Cloud TPU specialization explains target-specific execution boundaries. Logical CPU devices practice partitioning; they do not simulate TPU performance.

Download the repository ZIP or use your existing checkout. Extract it and open a terminal in the folder containing requirements-cpu.txt. The beginner ZIP is separate; these lessons use the full repository folder.

macOS / Linux terminal · repository folder
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-cpu.txt

PowerShell · repository folder
py -3 -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements-cpu.txt

After activation, run the complete worked example from that same folder:

macOS / Linux terminal · activated environment
python phases/15-deployment/01-keras-and-pytorch-bridges-to-explicit-jax/code/main.py

PowerShell command · repository folder
.\.venv\Scripts\python.exe phases/15-deployment/01-keras-and-pytorch-bridges-to-explicit-jax/code/main.py

For an incremental build, create main.py in your own practice folder, append each build block in order, and run python main.py with this environment active. Keep that file separate from the downloaded reference solution.

Check: Two rows match; the first is [3.1,−0.45][3.1, -0.45] within absolute tolerance 10−610^{-6}.

Revisit the guided CPU setup · Open 19: TPU Setup. Package instructions are CPU instructions; Windows and accelerator environments need their own validation.

Stage 1 of 5 · Understand the MechanismCore mental model, equations & visual walkthrough

What you will be able to do

  • Map Dense and Linear weight axes and verify a known output.
  • Separate architecture, weights, non-trainable state, optimizer state and preprocessing.
  • Run a backend-neutral Keras model and explain the boundaries of tensor exchange and export.
  • Carry an explicit attention tensor layout and processor contract into a bounded Transformer conversion rather than assuming a dense transpose is sufficient.

The problem

You have a dense layer in PyTorch and the same model idea in Keras. Can you reuse the weights in JAX? We’ll start with one small layer, work out a prediction by hand, and check the weight axes. From there, you’ll learn which parts of a model can transfer directly and which need separate checks.

The idea

Cross-framework conversion must preserve the meaning of inputs, parameters and outputs. A layout transpose can be necessary, but architecture, preprocessing and arithmetic conventions must agree too. Start by writing the function each framework computes.

Map the function, not just the weight file

For a dense layer, one framework may store weights as input-by-output while another stores output-by-input. The transpose can align the intended multiplication. If both dimensions happen to match, an incorrect mapping may still have a legal shape.

Use unequal dimensions and distinguishable values, then compare intermediate results on the same input. Check bias placement, activation and normalization conventions instead of inferring equivalence from a successful load.

The transpose heatmaps show storage correspondence. They do not demonstrate conversion of an arbitrary Keras, TensorFlow or PyTorch architecture. A supported converter needs an explicit model-family contract, including tokenizer or image/audio processor identity where relevant.

A parameter file is not a model

Our inputs have shape [batch,3][\mathrm{batch}, 3]. Keras Dense uses a [3,2][3, 2] kernel; PyTorch Linear stores [2,3][2, 3] weights and multiplies by their transpose. Bias has shape [2][2]. For x=[1,2,−1]x=[1, 2, -1], the first output is 1+1+1+0.1=3.11+1+1+0.1=3.1. Verify a hand-derived row before comparing whole arrays. Convolution kernels have additional layout conventions; do not apply this dense transpose rule to them.

The executed two-row batch returns [[3.1,−0.45],[−4.9,4.55]][[3.1,-0.45],[-4.9,4.55]] in both matched implementations. For the first row, the output arithmetic is 1+1+1+0.1=3.11+1+1+0.1=3.1; for the second output it is −2+2−0.25−0.2=−0.45-2+2-0.25-0.2=-0.45. If the outputs differ while shapes match, check feature order before changing tolerances: the fixture deliberately swaps first and last features and observes (−0.9,4.05)(-0.9,4.05).

Use Keras as a shared model vocabulary

Keras 33 can execute models on JAX, TensorFlow or PyTorch. Select KERAS_BACKEND before importing keras, in a fresh process. Use keras.layers and keras.ops in portable model code; a call to tf.* inside a custom layer is a TensorFlow dependency. This does not port arbitrary Python side effects or framework-specific training loops. A .keras archive preserves Keras configuration and weights; an inference export has a different job.

Explicit JAX state is a design decision

Pass parameters and non-trainable state explicitly; return updated state and keep random keys visible. A matched inference result does not show that dropout randomness, BatchNorm moving statistics or optimizer slots were transferred. Start in inference mode with training=False. Compare gradients and an update separately before claiming training equivalence. Reinitializing the optimizer is a new training run, not an exact resume.

Choose the bridge for the destination

NumPy exchange makes a clear host copy and usually breaks autodiff across the boundary. DLPack exchanges supported tensor buffers between frameworks; sharing memory is not model conversion or cross-framework autodiff, and external mutation can violate JAX assumptions. JAX export serializes a staged computation for compatible consumers. jax2tf is a separate, version-sensitive TensorFlow bridge. Keras export can target supported inference formats; inspect the destination operators and signatures. ONNX is a model interchange format, while ONNX Runtime execution providers choose supported execution paths. PyTorch/XLA runs PyTorch through XLA; it does not turn a PyTorch model into an explicit JAX function.

Run the optional real-framework lab

The file labs/keras_backends.py beside this lesson builds the same Dense model, fixes its weights, compares known outputs, and inspects mixed precision policy. From the extracted course workspace root, use an isolated environment with Keras and the backend you want. Run the same file in a new process for each backend; --backend selects it before importing Keras. Tested locally with Keras 3.14.1 on the installed JAX and PyTorch backends. The same companion also passed on TensorFlow 2.21.0 with Keras 3.15.1 in the separate edge-lab environment. These optional backend checks are recorded separately from the core CPU receipt.

Run instructions: Optional Keras environment — macOS/Linux, workspace root
sh
# Run optional keras environment — macos/linux, workspace root using the course Python environment
python3 -m venv .venv-interop
.venv-interop/bin/python -m pip install "keras==3.14.1" "jax==0.9.2" "torch==2.11.0"

Expected:

An isolated environment with the tested Keras/JAX/PyTorch versions. Use .venv-interop/bin/python instead of python3 in the commands below; on Windows use .venv-interop\Scripts\python.exe. TensorFlow is a separate optional backend: use the edge lab environment instructions and record its installed versions.

Run instructions: Optional lab — run from the course workspace root after installing Keras and the selected backend
sh
# Run optional lab — run from the course workspace root after installing keras and the selected backend using the course Python environment
python3 phases/15-deployment/01-keras-and-pytorch-bridges-to-explicit-jax/labs/keras_backends.py --backend jax
python3 phases/15-deployment/01-keras-and-pytorch-bridges-to-explicit-jax/labs/keras_backends.py --backend torch
python3 phases/15-deployment/01-keras-and-pytorch-bridges-to-explicit-jax/labs/keras_backends.py --backend tensorflow

Expected:

Each installed backend prints PASS and measured precision errors. A missing backend must be installed in its isolated environment before running that command.

Write the same dense layer in both layouts

We’ll call the input matrix XX, the Keras-style kernel WW, and the PyTorch-style weight WtorchW_{\mathrm{torch}}. The math agrees when W=WtorchTW=W_{\mathrm{torch}}^\mathsf{T}. The transpose changes how weights are arranged, not what the layer means. Bias bb is added to every output row. Check the known first row before trying a larger model.

Y=XW+b=XWtorchT+bY=XW+b=XW_{\mathrm{torch}}^\mathsf{T}+b

A valid shape can still mean the wrong feature

Imagine that the source reads temperature, pressure and humidity in that order. The target reads humidity, pressure and temperature. Both receive three numbers, so a shape assertion passes. But the first coefficient now multiplies the wrong measurement. Write down feature names and units next to each input column before mapping a weight tensor.

For the worked first row, the two scores are 1+1+1+0.1=3.11+1+1+0.1=3.1 and −2+2−0.25−0.2=−0.45-2+2-0.25-0.2=-0.45. Swapping the first and last features gives −0.9-0.9 and 4.054.05. These large differences are not floating-point noise. Follow the first mismatch: raw values, normalized values, first layer, activation, then output labels. This order prevents you from adjusting a numerical tolerance to conceal a different function.

Use a small bridge before a complete model

Keep the dense example as a test you can reason through by hand. Then add one operation at a time: a nonlinearity, normalization, and a second layer. Record an intermediate tensor at each boundary. For a convolution, declare both image layout and kernel layout; for attention, declare token order, head order, masks, and how query, key and value weights are packed.

The optional Keras lab runs real backends in separate processes. The default companion below uses NumPy and JAX to expose the mapping arithmetic, so its success alone is verified separately from TensorFlow, PyTorch, or a complete pretrained checkpoint. Carry a table of source names, target names, shapes, axis permutations and test inputs into the model-specific conversion lesson.

Attention requires a model-family contract

The dense transpose rule does not describe an attention checkpoint. Query, key and value may be separate or fused; fused rows can be stored as Q-then-K-then-V or interleaved by head. Declare the source key, target key, source shape, target shape, axis transform and reshape order. Also bind tokenization or preprocessing identity, position IDs, mask meaning, normalization epsilon, and whether the language-model head shares the embedding table.

The deployment-08 optional lab gives one saved PyTorch-to-NNX example for the course's single pre-normalized decoder block. It is a bounded fixture with an explicit map, asymmetric probes, intermediate comparisons, semantic negative controls, VJP and process reload. It does not imply that arbitrary saved checkpoints can be converted by applying a general transpose rule.

Read the figure

The upper heatmap uses input feature as row and output feature as column; the lower uses output as row and input as column. Cell units are weights, and the same six numbers move by transpose: weight −1-1 stays on the connection from input 2 to output 0. The shapes change from 3×23\times2 to 2×32\times3, but the function should not. Use the known batch output [[3.1,−0.45],[−4.9,4.55]][[3.1,-0.45],[-4.9,4.55]] as the next numerical check.

Connect it to the computation

The first layout computes XW+bXW+b; the transposed storage computes XW~T+bX\widetilde W^T+b, with W~=WT\widetilde W=W^T. In the executed fixture both produce scores (3.1,−0.45)(3.1,-0.45) and (−4.9,4.55)(-4.9,4.55). A feature-order swap instead gives (−0.9,4.05)(-0.9,4.05), despite valid dimensions. After checking the transpose, inspect feature names, bias, activation and preprocessing; the dense result does not qualify conversion of a full framework model.

Figure data and execution details

Recorded environment: JAX 0.9.2 · 1 CPU device

The notebook contains the data experiment, plotting code, and recorded figure output. Rerun it to test your changes.

Stage 2 of 5 · Build & Run the ReferenceStep-by-step code construction & verified execution output

Your build: Create a separate main.py in your practice folder. Append each block in order and run python main.py in the lesson environment. The downloaded script is the complete reference, so keep it separate from your attempt.

1 Write the source function and its known input

Create main.py in your lesson workspace and run it with the active course Python environment. Keep feature order fixed while you name the input, kernel and bias.

main.py · append build step 1
# Step 1 — Write the source function and its known input
import numpy as np
import jax
import jax.numpy as jnp
# Same dense layer, different parameter layouts.
x = np.array([[1., 2., -1.], [-2., 0., 3.]], np.float32)
keras_kernel = np.array([[1., -2.], [.5, 1.], [-1., .25]], np.float32)
bias = np.array([.1, -.2], np.float32)

np.testing.assert_allclose(x[0] @ keras_kernel + bias, [3.1, -.45], atol=1e-6)

The arrays define a batch of shape (2,3)(2,3), a kernel of shape (3,2)(3,2), and a bias of shape (2,)(2,). The first known output is [3.1,−0.45][3.1,-0.45].

Explain the APIs in this step
  • np.testing.assert_allclose(actual, desired, rtol=..., atol=...): Checks a host-side numerical expectation and raises AssertionError when values differ beyond tolerance.

2 Map the stored tensor into explicit JAX parameters

Append this block to the same main.py and rerun the whole file. Transpose the stored PyTorch-style weights back into input-by-output order, then define the pure prediction function.

main.py · append build step 2
# Step 2 — Map the stored tensor into explicit JAX parameters: The zero-input check returns the bias twice.
torch_weight = keras_kernel.T.copy()  # torch Linear: output, input
params = {"kernel": jnp.asarray(torch_weight.T), "bias": jnp.asarray(bias)}
def predict(params, inputs):
    return inputs @ params["kernel"] + params["bias"]

np.testing.assert_allclose(predict(params, jnp.zeros((2,3))), np.tile(bias,(2,1)), atol=1e-6)

The zero-input check returns the bias twice. If it does not, inspect bias broadcasting before comparing full predictions.

Explain the APIs in this step
  • np.testing.assert_allclose(actual, desired, rtol=..., atol=...): Checks a host-side numerical expectation and raises AssertionError when values differ beyond tolerance.
  • jnp.zeros(shape, dtype=...): Creates a JAX array of the requested shape filled with zeros.

3 Verify values before extending the architecture

Append this block to the same main.py and rerun the whole file. Run both independent checks and inspect the two output rows.

main.py · append build step 3
# Step 3 — Verify values before extending the architecture
# Wrap with `jax.jit` (`actual`) so XLA traces and compiles the function.
actual = np.asarray(jax.jit(predict)(params, jnp.asarray(x)))
# A hand-computed first row detects a shared layout mistake.
np.testing.assert_allclose(actual[0], [3.1, -.45], atol=1e-6)
np.testing.assert_allclose(actual, x @ keras_kernel + bias, atol=1e-6)
print("Matched dense outputs:", actual)

The first assertion checks hand arithmetic; the second compares the entire batch with the source-layout calculation. Continue with the feature-order and second-layer experiments below.

Explain the APIs in this step
  • jax.jit(fn) / @jax.jit: Traces a function for its input signatures and compiles it; changed shapes, dtypes, or static arguments can trigger another compilation.
  • np.testing.assert_allclose(actual, desired, rtol=..., atol=...): Checks a host-side numerical expectation and raises AssertionError when values differ beyond tolerance.
View the cumulative build in main.py

These blocks append in order to your separate practice file.

main.py · cumulative Python build
# Step 1 — Write the source function and its known input
import numpy as np
import jax
import jax.numpy as jnp
# Same dense layer, different parameter layouts.
x = np.array([[1., 2., -1.], [-2., 0., 3.]], np.float32)
keras_kernel = np.array([[1., -2.], [.5, 1.], [-1., .25]], np.float32)
bias = np.array([.1, -.2], np.float32)

np.testing.assert_allclose(x[0] @ keras_kernel + bias, [3.1, -.45], atol=1e-6)

# Step 2 — Map the stored tensor into explicit JAX parameters: The zero-input check returns the bias twice.
torch_weight = keras_kernel.T.copy()  # torch Linear: output, input
params = {"kernel": jnp.asarray(torch_weight.T), "bias": jnp.asarray(bias)}
def predict(params, inputs):
    return inputs @ params["kernel"] + params["bias"]

np.testing.assert_allclose(predict(params, jnp.zeros((2,3))), np.tile(bias,(2,1)), atol=1e-6)

# Step 3 — Verify values before extending the architecture
# Wrap with `jax.jit` (`actual`) so XLA traces and compiles the function.
actual = np.asarray(jax.jit(predict)(params, jnp.asarray(x)))
# A hand-computed first row detects a shared layout mistake.
np.testing.assert_allclose(actual[0], [3.1, -.45], atol=1e-6)
np.testing.assert_allclose(actual, x @ keras_kernel + bias, atol=1e-6)
print("Matched dense outputs:", actual)

Run the example

python3 phases/15-deployment/01-keras-and-pytorch-bridges-to-explicit-jax/code/main.py
About this command

This shortcut uses an activated macOS/Linux course environment in the repository folder. For Windows, use the explicit interpreter command in “Setup help” above. The script includes worked examples and reference solutions.

Inspect the complete assembled example
main.py · complete Python example
# Move models between JAX, Keras, TensorFlow and PyTorch
import numpy as np
import jax
import jax.numpy as jnp
# Same dense layer, different parameter layouts.
x = np.array([[1., 2., -1.], [-2., 0., 3.]], np.float32)
keras_kernel = np.array([[1., -2.], [.5, 1.], [-1., .25]], np.float32)
bias = np.array([.1, -.2], np.float32)
torch_weight = keras_kernel.T.copy()  # torch Linear: output, input
params = {"kernel": jnp.asarray(torch_weight.T), "bias": jnp.asarray(bias)}
def predict(params, inputs):
    return inputs @ params["kernel"] + params["bias"]
# Wrap with `jax.jit` (`actual`) so XLA traces and compiles the function.
actual = np.asarray(jax.jit(predict)(params, jnp.asarray(x)))
# A hand-computed first row detects a shared layout mistake.
np.testing.assert_allclose(actual[0], [3.1, -.45], atol=1e-6)
np.testing.assert_allclose(actual, x @ keras_kernel + bias, atol=1e-6)
print("Matched dense outputs:", actual)

Expected result: Two rows match; the first is [3.1,−0.45][3.1, -0.45] within absolute tolerance 10−610^{-6}.

Recorded output and runtime details

Reference output captured from executing the complete lesson script (code/main.py) and its assertions in the course CPU environment.

2026-10-27T17:09:09.907505+00:00 · Python 3.14.3 · JAX 0.9.2 · 1 CPU device

Matched dense outputs: [[ 3.1  -0.45]
 [-4.9   4.55]]
Matched dense outputs: [[ 3.1  -0.45]
 [-4.9   4.55]]
Equal shapes, unequal outputs
Feature order failed, then matched after aligning kernel rows.
Changed batch verified
Two-layer reference scores: [ 6.5  -4.25]
PASS: deployment-01

Download execution record · The notebook includes executed cells and figure outputs. Compare your run with these values; timings depend on the machine.

Stage 3 of 5 · Controlled ExperimentsPredict before running & test boundary conditions

A square matrix hides a transpose

A square matrix hides a transpose · Python experiment
# Experiment — A square matrix hides a transpose
square = np.array([[1., 2.], [3., 4.]], np.float32)
z = np.array([[2., -1.]], np.float32)
# Check tensor shape invariant: `(z @ square).shape == (z @ square.T).shape`
assert (z @ square).shape == (z @ square.T).shape
assert not np.allclose(z @ square, z @ square.T)
print("Equal shapes, unequal outputs")

Expected: Both outputs have the same shape but different values.

Check asymmetric values and independently known outputs, not just tensor dimensions.

Reorder features without changing tensor dimensions

Reorder features without changing tensor dimensions · Python experiment
# Experiment — Reorder features without changing tensor dimensions: The repair changes the pairing of features and coefficients.
permutation = [2, 1, 0]
reordered = x[:, permutation]
# Transfer the device result to the host for an independent NumPy comparison.
wrong_order = np.asarray(predict(params, reordered))
np.testing.assert_allclose(wrong_order[0], [-.9, 4.05], atol=1e-6)
assert not np.allclose(wrong_order, actual)
repaired_params = {'kernel': params['kernel'][permutation, :], 'bias': params['bias']}
np.testing.assert_allclose(predict(repaired_params, reordered), actual, atol=1e-6)
print('Feature order failed, then matched after aligning kernel rows.')

Expected: The wrong first row is [−0.9,4.05][-0.9,4.05]; permuting the kernel rows restores both original observations.

The repair changes the pairing of features and coefficients. It does not change the model represented by those pairings. Repeat with asymmetric feature values; repeated values can hide a wrong order.

Stage 4 of 5 · Hands-On Exercise & CheckpointModify the code yourself, solve transfer tasks & test your understanding
Hands-On Exercise · Modify & Verify Run in the declared lesson environment

Make it yours

Where to put your code: Modify your own main.py, or use the Make it yours cell in your downloaded notebook. Run your attempt in the declared environment and verify your prediction before opening the reference solution.

Add a batch with four rows and preserve the same parameters. Verify each row independently using a feature-by-feature sum.

Show a writing guide and starter scaffold

How to write this exercise code

Learner-only inputs, outputs, checks and hints

Inputs

  • A dense layer weight/bias pair, the source preprocessing rule, and one or more raw input rows.

Outputs

  • JAX-compatible predictions for each input row plus a per-feature independent arithmetic check.

Invariants to check

  • Weight layout, input normalization, bias, activation, and batch axis are all explicit at the framework boundary.

Hints

  • Use an asymmetric weight matrix and inputs so a transpose or scale mismatch is visible.
  • Compare the same raw values through both paths before testing a new batch.
Your attempt · complete the TODOs
import numpy as np
import jax
import jax.numpy as jnp

def map_dense_inference(weights, bias, inputs, input_transform):
    """JAX-compatible predictions for each input row plus a per-feature independent arithmetic check."""
    # TODO: Keep raw input normalization and dense weight layout explicit, then compare each output row with a NumPy reference.
    raise NotImplementedError
Reveal the reference solution
Reference solution
# Exercise solution: Add a batch with four rows and preserve the same parameters.
changed = np.arange(12, dtype=np.float32).reshape(4, 3) / 4
expected = np.stack([sum(row[i] * keras_kernel[i] for i in range(3)) + bias for row in changed])
np.testing.assert_allclose(predict(params, changed), expected, atol=2e-6)
print("Changed batch verified")

Detect a preprocessing mismatch

Transfer

Suppose the source model expects inputs divided by 255255, while the target receives raw pixels. Demonstrate why correct weight transfer still fails.

Show a writing guide and starter scaffold

How to write this practice solution

Learner-only inputs, outputs, checks and hints

Inputs

  • Suppose the source model expects inputs divided by 255255, while the target receives raw pixels. Demonstrate why correct weight transfer still fails.

Outputs

  • A pair of prediction paths using identical transferred weights.

Invariants to check

  • Changing only input normalization changes the model input seen by the dense layer.

Hints

  • Compute both predictions from the same raw input: once with the source normalization and once without it. Matching weights cannot cancel an input-scale mismatch.
  • Transform the raw pixel row before inference and compare it with the target path.
Your attempt · complete the TODOs
import numpy as np
import jax
import jax.numpy as jnp

def practice_detect_a_preprocessing_mismatch(source_model, target_model, raw_pixels, source_scale):
    """Complete the detect a preprocessing mismatch check."""
    # TODO: ('Apply source scaling and target raw-input paths to the same pixels and compare their dense outputs.', 'Two predictions and a mismatch attributable to preprocessing while weights remain fixed.', 'The only changed factor is input normalization; transferred parameters are identical.')
    raise NotImplementedError
Show a hint

Compute both predictions from the same raw input: once with the source normalization and once without it. Matching weights cannot cancel an input-scale mismatch.

Reveal reference solution and reasoning
Detect a preprocessing mismatch · Python solution
# Detect a preprocessing mismatch (Transfer): Input normalization belongs in the deployment contract.
pixels = np.array([[255., 128., 0.]], np.float32)
source = predict(params, pixels / 255.)
wrong = predict(params, pixels)
assert not np.allclose(source, wrong)
np.testing.assert_allclose(source, predict(params, pixels / 255.), atol=1e-6)

Input normalization belongs in the deployment contract. Matching weights cannot compensate for a different input distribution.

Map a second layer and locate an omitted activation

Transfer / diagnosis

Extend the dense mapping with ReLU and a second dense layer. Use independently computed final scores to catch a converter that forgets ReLU. Then show why an all-positive probe would be a weak test.

Show a writing guide and starter scaffold

How to write this practice solution

Learner-only inputs, outputs, checks and hints

Inputs

  • Extend the dense mapping with ReLU and a second dense layer. Use independently computed final scores to catch a converter that forgets ReLU. Then show why an all-positive probe would be a weak test.

Outputs

  • Independent scores for a two-layer dense mapping with an intermediate activation.

Invariants to check

  • A negative hidden preactivation exercises the ReLU boundary in the parity check.

Hints

  • Compute both first-layer rows by hand, replace negative entries by zero, and apply the new output weights.
  • Compare the first layer, activation, and second layer in sequence.
Your attempt · complete the TODOs
import numpy as np
import jax
import jax.numpy as jnp

def practice_map_a_second_layer_and_locate_an_omitted_activation(weight1, bias1, weight2, bias2, inputs):
    """Complete the map a second layer and locate an omitted activation check."""
    # TODO: ('Implement both dense layers with the intermediate ReLU and compare against independent feature-by-feature scores on a probe containing a negative hidden activation.', 'Independent two-layer scores and a converter failure when the activation is omitted.', 'The comparison includes a probe where ReLU changes the intermediate representation.')
    raise NotImplementedError
Show a hint

Compute both first-layer rows by hand, replace negative entries by zero, and apply the new output weights.

Reveal reference solution and reasoning
Map a second layer and locate an omitted activation · Python solution
# Map a second layer and locate an omitted activation (Transfer / diagnosis)
second_kernel = np.array([[2.], [-1.]], np.float32)
second_bias = np.array([.3], np.float32)
# Transfer the device result to the host for an independent NumPy comparison.
hidden = np.maximum(np.asarray(predict(params, x)), 0.)
bridged_scores = hidden @ second_kernel + second_bias
np.testing.assert_allclose(bridged_scores[:, 0], [6.5, -4.25], atol=2e-6)
# Transfer the device result to the host for an independent NumPy comparison.
omitted_relu = np.asarray(predict(params, x)) @ second_kernel + second_bias
assert not np.allclose(omitted_relu, bridged_scores)
positive_probe = np.array([[1., 2.]], np.float32)
np.testing.assert_array_equal(np.maximum(positive_probe, 0), positive_probe)
print('Two-layer reference scores:', bridged_scores[:, 0])

The first dense layer agrees, but the activation is the first divergent boundary. The final reference scores are 6.56.5 and −4.25-4.25. A probe containing only positive hidden values would make an omitted ReLU invisible.

Check your understanding

The Dense predictions match after transposing weights. What has been established?

Choose one answer

Diagnose the result

If output shapes fail, write the contraction axes. If shapes match but values drift, check layout, preprocessing, bias, inference mode and dtype in that order. A missing TensorFlow import in the optional lab means that backend is not installed; it is not a failure of the JAX core exercise.

Stage 5 of 5 · Keep Your EvidenceRecord your verification receipt & carry takeaways forward

Keep your evidence

Download evidence template

Keep the layout map, two changed-input comparisons, runtime versions and optional backend-lab output. Identify which model state and preprocessing are not covered by a dense-layer match.

Save your modified code, its output, and an explanation of your prediction. Checkpoint answers and exercise evidence are tracked separately.

Saved in this browser. Export or back up your portfolio anytime in My learning.

Save work for this lesson

Reference execution: CPU validation only. This worked reference is validated on CPU. Accelerator compatibility requires a separate source review and target execution receipt. Use the execution matrix to review device selection and data ownership before adapting a separate target program. Execution matrix and target qualification.

Primary references