# Choose weight, activation and accumulation precision

Phase 15: Deployment, interoperability & edge AI · about 110 minutes · CPU

## What you will be able to do

- Compare FP32, FP16 and BF16 rounding without confusing storage with accumulation.
- Implement per-output-channel symmetric INT8 and INT4 weight quantization.
- Calibrate activations and diagnose clipping on unseen ranges.
- Explain FP64, FP8, weight-only, full integer and mixed precision policies.

## The problem

Someone asks you for an “8-bit model.” A helpful first question is: 8-bit weights, activations, or both? We’ll separate those choices, compare them with a float32 reference, and watch what happens when an input falls outside the calibrated range. You’ll leave with a precision policy you can explain, not just a dtype label.

## The idea

A precision policy specifies how weights, activations, accumulations and outputs are represented. Quantization adds scales, zero-points and rounding/clipping rules. Track those units through the computation instead of describing the whole model with one dtype label.

## Follow the units through integer arithmetic

For an illustrative scale $s=0.5$ and zero-point $z=3$, the real value $1$ maps to integer $q=5$, since $q=\operatorname{round}(1/0.5)+3$. Reconstructing gives $0.5(5-3)=1$. Real zero maps to the zero-point, not necessarily integer zero.

In a dot product, centered integer products accumulate in an appropriately wide type. The accumulator's scale comes from the input and weight scales. Bias must use compatible units before the output is rescaled or requantized.

The error plots compare specific representation policies. Read weight, activation and accumulation choices separately. Per-channel scales also require a declared axis. Numerical error, downstream quality and target-runtime speed need separate evidence.

### Integer values need scales and zero-points

**Predict:** What happens if you ignore a nonzero zero-point when interpreting quantized values?

![Integer values need scales and zero-points](../figures/deployment-05-mechanism.svg)

*Architecture and dataflow mechanism diagram.*

Follow affine integer codes into centered products, wide accumulation and compatible bias units, then output rescaling. Exact accumulator dtype and per-channel scale axes belong to the declared runtime policy. These arrows describe arithmetic dependencies, not a specific hardware kernel.

### Pause and reason

What happens if you ignore a nonzero zero-point when interpreting quantized values?

<details><summary>Compare your reasoning</summary>

You introduce an offset error before multiplication or reconstruction. Correct integer storage alone is insufficient; arithmetic must honor the affine mapping.

</details>

## Write the policy before choosing a dtype

Use separate fields for stored weights, input/intermediate activations, multiply operands, accumulator, output and optimizer/master weights. FP32 is a useful baseline. FP64 can help sensitive numerical work; in JAX enable jax_enable_x64 before creating arrays and account for target support and cost. FP16 has a narrower exponent range than BF16, while BF16 gives up fraction precision for range. Keep reductions, normalization statistics and sensitive losses in FP32 when needed. The example rounds operands to each format, then explicitly returns them to FP32 for multiplication; it isolates representation error rather than proving low-precision kernel behavior.

## Mixed precision during training

A Keras mixed_float16 or mixed_bfloat16 policy generally keeps variables in FP32 while using lower-precision computations. Set the policy before constructing layers and inspect variable_dtype and compute_dtype. FP16 training may need loss scaling to avoid gradient underflow; unscale before clipping and updating, reject nonfinite updates, and keep optimizer/master state at suitable precision. BF16 has wider range but still incurs rounding. In explicit JAX code, cast intentionally at operation boundaries and test the full update. Merely changing the parameter storage dtype is not a complete mixed precision implementation.

## Quantize weights with scale metadata

Let $b$ be the number of bits and $q_{\max}=2^{b-1}-1$ the largest positive code. For each weight column, choose scale $s=\max(|w|)/q_{\max}$. Dividing a weight by $s$ moves it onto the integer grid; rounding chooses the nearest code, and clipping keeps the code in range. Reconstruct the approximate weight with $\widehat w=sq$.

Our Dense kernel’s output channels lie along axis $1$, so each column gets its own scale. This preserves resolution when columns have different ranges. For a zero column, use scale $1$ and codes $0$. Without clipping, rounding changes a value by at most $s/2$. Smaller quantization groups can improve local resolution, but each group also needs scale metadata.

## Weight-only and full integer are different deployments

W4A16 and W8A16 usually describe compressed weights with floating activations; a runtime may dequantize before multiplication or use a specialized mixed kernel. Our 4-bit codes live in int8 arrays and are not nibble-packed, so they do not realize 4-bit memory savings. W8A8 quantizes activations too. An affine activation reconstructs as $a=s(q-z)$, where $q$ is its integer code, $s$ is its scale, and $z$ is the zero point; calibration estimates its range from representative training/calibration data, never the held-out evaluation set. Static quantization fixes ranges; dynamic quantization estimates activation ranges at runtime. Accumulation commonly uses INT32 for integer kernels; bias and requantization must use compatible scales.

## FP8 and other low-bit formats require a target contract

FP8 E4M3 and E5M2 differ in range and precision; their names are not interchangeable. Scaling granularity, saturation behavior and kernel support matter. Do not treat a successful CPU dtype cast as evidence of FP8 acceleration. INT4, UINT8, INT16 activation schemes and newer sub-byte floating formats also depend on runtime/operator support. For ONNX Runtime or LiteRT, inspect the format, operator, quantization scheme and execution provider together. If a supported PTQ policy loses too much task accuracy, evaluate quantization-aware training rather than silently relaxing acceptance criteria.

## Evaluate quality separately from size and speed

Compare the original and converted models on held-out examples, including low-margin decisions and uncommon ranges. Record absolute/relative error with a sensible near-zero floor, task metrics, clipping rates and acceptance thresholds chosen in advance. Estimate weight storage as `ceil(parameter_count*bits/8)` plus metadata, but measure the real artifact and peak runtime memory. Activation buffers, KV caches, workspace and fallback copies can dominate. Benchmark on the intended device after verifying which kernels actually execute.

## Follow one value onto the quantization grid

A scale $s$ tells us how far apart the representable values are. For $s=0.1$, a weight of $0.26$ rounds to integer code $3$ and reconstructs as $0.3$. The error is $0.04$, less than half a grid step. If a value lies outside the supported range, clipping can make the error much larger. Our per-channel code computes a separate scale for each output column and treats a zero column specially.

$$
\begin{aligned}q_{\max}&=2^{b-1}-1\\q&=\operatorname{clip}\!\left(\operatorname{round}(w/s),-q_{\max},q_{\max}\right)\\\widehat w&=sq,\qquad |w-\widehat w|\le s/2\quad\text{without clipping}\end{aligned}
$$

## Follow an affine output through every numeric unit

Let an activation code be $q_x$, its zero point $z_x$, and its positive scale $s_x$. The represented value is $s_x(q_x-z_x)$. Subtract the zero point after widening the codes: raw INT8 arithmetic can overflow before accumulation. With symmetric weights, each output channel has weight zero point zero and scale $s_{w,j}$. Its accumulator therefore has units $s_xs_{w,j}$, so round the bias into those same units before adding it. In the worked experiment, corrected products plus bias give accumulators $5$ and $15$; their real outputs are $0.625$ and $0.9375$. Requantization maps these values onto the output grid. A small output scale offers fine spacing but can also make the representable range too narrow. The second plot separates forgetting the input zero point from clipping at the output; these are different bugs with different repairs.

$$
a_j=\sum_i(q_{x,i}-z_x)q_{w,ij}+\operatorname{round}\!\left(\frac{b_j}{s_xs_{w,j}}\right),\qquad q_{y,j}=\operatorname{clip}\!\left(\operatorname{round}\!\left(\frac{s_xs_{w,j}}{s_y}a_j\right)+z_y,-128,127\right)
$$

## Distinguish storage savings from an error budget

For a dense matrix with $K$ input features and $N$ output features, FP32 weights occupy $4KN$ bytes before container metadata. Ideally packed four-bit weights occupy $\lceil KN/2\rceil$ bytes, but each output channel may also need a floating-point scale. If scales are FP32, add $4N$ bytes. Biases, alignment and runtime workspace are additional terms.

For this tiny $(3,2)$ matrix, FP32 needs $24$ weight bytes. A packed four-bit representation plus two FP32 scales would need $3+8=11$ bytes before other metadata. The teaching implementation stores its codes in int8 arrays, so it uses $6+8=14$ bytes instead. Report the representation you actually measured. Neither byte count says which kernel will execute or how long a request will take.

## Follow rounding error through the next multiplication

A weight value rounded to its nearest grid point has error at most half a scale when it is within the represented range. To see why that does not directly bound the output by half a scale, write the output error as a sum of input-weight error products. Large inputs can amplify small weight errors; cancellation may reduce the observed error but should not be assumed in the bound.

For input $[2,-1,0.5]$ and weight errors $[0.1,-0.1,0.1]$, the signed output error is $0.35$. The sum of absolute products is also $0.35$, so the bound can be attained. Activation clipping is different: a value outside the represented range can lose much more than half a scale. Keep a clipping count alongside your rounding and task-quality measurements.

$$
|\Delta y_j|=\left|\sum_i x_i\Delta W_{ij}\right|\leq\sum_i |x_i|\,|\Delta W_{ij}|
$$

## Write the full-precision reference

Create main.py in your lesson workspace and run it with the active course Python environment. Keep the input and kernel in FP32 and establish an independently computed dot product.

```python
# Step 1 — Write the full-precision reference: The top-left output is -1.437.
import numpy as np
import jax.numpy as jnp
x = np.array([[.3, -1.2, 2.1], [1.1, .2, -.8]], np.float32)
w = np.array([[.11, -1.8], [.7, .2], [-.3, 2.4]], np.float32)
# Perform matrix / vector contraction (`@`) to compute `reference`.
reference = x @ w

np.testing.assert_allclose(reference[0,0], -1.437, atol=1e-6)
```

The top-left output is $-1.437$. This anchors the error comparison to a named value before introducing a lower-precision policy.

## Round inputs and weights, then accumulate explicitly

Append this block to the same main.py and rerun the whole file. Compare FP32, FP16 and BF16 representations while returning to FP32 before multiplication.

```python
# Step 2 — Round inputs and weights, then accumulate explicitly: The printed errors isolate the consequence of representational...
def mixed_forward(x, w, dtype):
    # Demonstrate rounding inputs/weights with explicit float32 accumulation.
    a = jnp.asarray(x, dtype).astype(jnp.float32)
    # Create device-backed JAX array `b`.
    b = jnp.asarray(w, dtype).astype(jnp.float32)
    return np.asarray(a @ b)
for dtype in (jnp.float32, jnp.float16, jnp.bfloat16):
    y = mixed_forward(x, w, dtype)
    # Aggregate array values to compute `error`.
    error = float(np.max(np.abs(y-reference)))
    # Confirm that all computed values remain finite (no NaN or Inf).
    assert np.isfinite(y).all() and error < .05
    print(str(dtype), "max absolute error", error)
```

The printed errors isolate the consequence of representational rounding in this example. This does not benchmark a native low-precision matrix kernel.

## Construct signed integer codes and channel scales

Append this block to the same main.py and rerun the whole file. Quantize per output channel and inspect the all-zero-channel case.

```python
# Step 3 — Construct signed integer codes and channel scales: An all-zero channel uses scale 1 and zero codes, so reconstruction...
def quantize_weights(weights, bits):
    # Guard input contract (`bits not in (4, 8)`) and fail fast if violated.
    if bits not in (4, 8):
        raise ValueError("use signed 4 or 8 bit symmetric quantization")
    limit = 2 ** (bits - 1) - 1
    # Reduce along axis=0 to compute `maxima`.
    maxima = np.max(np.abs(weights), axis=0, keepdims=True)
    scale = np.where(maxima == 0, 1., maxima / limit).astype(np.float32)
    # Combine or mask array elements to form `q`.
    q = np.clip(np.rint(weights / scale), -limit, limit).astype(np.int8)
    return q, scale

# Allocate initialized array `(zero_codes, zero_scales)` with the specified shape and dtype.
zero_codes, zero_scales = quantize_weights(np.zeros((3,2),np.float32), 8)
# Allocate initialized array `` with the specified shape and dtype.
np.testing.assert_array_equal(zero_codes, np.zeros((3,2),np.int8))
# Allocate initialized array `` with the specified shape and dtype.
np.testing.assert_array_equal(zero_scales, np.ones((1,2),np.float32))
```

An all-zero channel uses scale $1$ and zero codes, so reconstruction stays zero without division by zero.

## Check reconstructed outputs against a bound

Append this block to the same main.py and rerun the whole file. Reconstruct the weights, then compare observed output errors with the independent absolute-product bound.

```python
# Step 4 — Check reconstructed outputs against a bound: Every error must fit its stated bound.
for bits in (8, 4):
    q, scale = quantize_weights(w, bits)
    reconstructed = q.astype(np.float32) * scale
    assert np.all(np.abs(reconstructed-w) <= scale / 2 + 1e-6)
    # Independent error bound: |x deltaW| <= |x| |deltaW|.
    error = np.abs(x @ reconstructed-reference)
    bound = np.abs(x) @ np.broadcast_to(scale / 2, w.shape)
    assert np.all(error <= bound + 1e-6)
    print("W%dA32 simulated output error" % bits, float(error.max()))
```

Every error must fit its stated bound. Move on to activation clipping and the affine integer experiment before calling the policy W8A8.

## Run the example

```python
# Choose weight, activation and accumulation precision: A precision policy specifies how weights, activations,...
import numpy as np
import jax.numpy as jnp
x = np.array([[.3, -1.2, 2.1], [1.1, .2, -.8]], np.float32)
w = np.array([[.11, -1.8], [.7, .2], [-.3, 2.4]], np.float32)
# Perform matrix / vector contraction (`@`) to compute `reference`.
reference = x @ w
def mixed_forward(x, w, dtype):
    # Demonstrate rounding inputs/weights with explicit float32 accumulation.
    a = jnp.asarray(x, dtype).astype(jnp.float32)
    # Create device-backed JAX array `b`.
    b = jnp.asarray(w, dtype).astype(jnp.float32)
    return np.asarray(a @ b)
for dtype in (jnp.float32, jnp.float16, jnp.bfloat16):
    y = mixed_forward(x, w, dtype)
    # Aggregate array values to compute `error`.
    error = float(np.max(np.abs(y-reference)))
    # Confirm that all computed values remain finite (no NaN or Inf).
    assert np.isfinite(y).all() and error < .05
    print(str(dtype), "max absolute error", error)

def quantize_weights(weights, bits):
    # Guard input contract (`bits not in (4, 8)`) and fail fast if violated.
    if bits not in (4, 8):
        raise ValueError("use signed 4 or 8 bit symmetric quantization")
    limit = 2 ** (bits - 1) - 1
    # Reduce along axis=0 to compute `maxima`.
    maxima = np.max(np.abs(weights), axis=0, keepdims=True)
    scale = np.where(maxima == 0, 1., maxima / limit).astype(np.float32)
    # Combine or mask array elements to form `q`.
    q = np.clip(np.rint(weights / scale), -limit, limit).astype(np.int8)
    return q, scale
for bits in (8, 4):
    q, scale = quantize_weights(w, bits)
    reconstructed = q.astype(np.float32) * scale
    assert np.all(np.abs(reconstructed-w) <= scale / 2 + 1e-6)
    # Independent error bound: |x deltaW| <= |x| |deltaW|.
    error = np.abs(x @ reconstructed-reference)
    bound = np.abs(x) @ np.broadcast_to(scale / 2, w.shape)
    assert np.all(error <= bound + 1e-6)
    print("W%dA32 simulated output error" % bits, float(error.max()))
```

Expected: FP32, FP16 and BF16 comparisons are finite with error below $0.05$ for this fixture. INT8/INT4 reconstruction errors obey a half-scale bound; these are simulated weight-only computations.

## Precision choices trade representation for output error

**Predict:** Must two low-precision formats have the same error?

![Precision choices trade representation for output error](../figures/deployment-05.svg)

**Recorded CPU computation**

### Read the figure

The horizontal categories are precision policies; the vertical axis is maximum absolute output error against the FP32 reference on the same small input. Lower bars mean closer numerical agreement, not faster execution. The FP32 baseline is zero by construction.

The errors are approximately $0.00105$ for FP16 rounding, $0.00617$ for BF16 rounding, $0.00803$ for W8 A32, and $0.146$ for W4 A32. The four-bit weight simulation has about eighteen times the output error of the eight-bit simulation in this example. These are output units, not percentages or classification error rates.

The lower panel uses a separate exactly representable affine fixture, with output channel on the horizontal axis and real output value on the vertical axis. Correct values are $0.625$ and $0.9375$. Omitting the input zero point lowers them to $-0.125$ and $0.5625$; this is systematic offset error, not random rounding. The narrow output grid makes both bars equal $0.52734375$, the largest representable output under that policy. Equal bars here reveal saturation, not equal underlying predictions.

### Connect it to the computation

For the floating-point policies, both inputs and weights are rounded, then converted to FP32 for accumulation. FP16 has finer precision than BF16 around these particular values; BF16’s wider exponent range is not being tested here. The weight-only policies keep activations in FP32 and reconstruct quantized weights before multiplication.

Four-bit quantization uses fewer representable weight levels, so its larger rounding steps can create larger output discrepancies. The code uses per-output-column scales and checks an independent error bound. This is simulated quantization using ordinary CPU arrays, not a measurement of packed four-bit storage or native low-bit kernels. Accuracy, memory, and latency must each be evaluated on the intended workload and runtime.

Use the lower panel to decide what to inspect: zero-point subtraction and bias units for an offset mismatch; output range and clipping counts for saturation. The upper panel compares precision policies on the original fixture, while the lower panel isolates operator bookkeeping on a different fixture. Neither measures latency.

```python
# Compute figure data for: Precision choices trade representation for output error
names = []
errors = []
for name, dtype in [('FP32', jnp.float32), ('FP16 → FP32', jnp.float16), ('BF16 → FP32', jnp.bfloat16)]:
    names.append(name)
    errors.append(float(np.max(np.abs(mixed_forward(x, w, dtype) - reference))))
for bits_plot in (8, 4):
    q_plot, s_plot = quantize_weights(w, bits_plot)
    names.append('W' + str(bits_plot) + ' A32')
    errors.append(float(np.max(np.abs(x @ (q_plot.astype(np.float32) * s_plot) - reference))))
visual_data = {'kind': 'bar', 'labels': names, 'ylabel': 'maximum absolute output error', 'series': [{'label': 'CPU reference error', 'y': errors}]}
# Deliberately exact grid values make an independent hand calculation possible.
a_sx, a_zx = .25, -3
# Convert `a_sw` to a host NumPy array for inspection or verification.
a_sw = np.array([.5, .25], np.float64)
# Convert `a_qx` to a host NumPy array for inspection or verification.
a_qx = np.array([[-3, 1, 5]], np.int8)
# Convert `a_qw` to a host NumPy array for inspection or verification.
a_qw = np.array([[2, -1], [-1, 2], [1, 1]], np.int8)
# Convert `a_bias` to a host NumPy array for inspection or verification.
a_bias = np.array([.125, -.0625], np.float64)
a_bias_codes = np.rint(a_bias / (a_sx * a_sw)).astype(np.int64)
# Check the wide result before representing it with an INT32 accumulator.
a_wide = (a_qx.astype(np.int64) - a_zx) @ a_qw.astype(np.int64) + a_bias_codes
assert np.all((a_wide >= np.iinfo(np.int32).min) & (a_wide <= np.iinfo(np.int32).max))
a_acc = a_wide.astype(np.int32)
# Execute `np.testing.assert_array_equal(a_acc, [[5, 15]])`
np.testing.assert_array_equal(a_acc, [[5, 15]])
a_real = a_acc.astype(np.float64) * a_sx * a_sw
# Independent real-valued input/weights, written without decoding the codes.
a_reference = np.array([[0., 1., 2.]]) @ np.array([[1., -.25], [-.5, .5], [.5, .25]]) + a_bias
# Execute `np.testing.assert_array_equal(a_real, [[.625, .9375]])`
np.testing.assert_array_equal(a_real, [[.625, .9375]])
np.testing.assert_allclose(a_real, a_reference, atol=1e-12, rtol=0)
def output_grid(real, scale, zero_point):
    # Guard input contract (`not np.isfinite(scale) or scale <= 0`) and fail fast if violated.
    if not np.isfinite(scale) or scale <= 0:
        raise ValueError('positive finite output scale required')
    # Guard input contract (`type(zero_point) is not int or not -128 <= zero_point <= 127`) and fail fast if violated.
    if type(zero_point) is not int or not -128 <= zero_point <= 127:
        raise ValueError('integer INT8 zero point required')
    unbounded = np.rint(real / scale) + zero_point
    clipped = (unbounded < -128) | (unbounded > 127)
    # Combine or mask array elements to form `codes`.
    codes = np.clip(unbounded, -128, 127).astype(np.int8)
    return codes, (codes.astype(np.float64) - zero_point) * scale, clipped

a_codes, a_restored, a_clipped = output_grid(a_real, .0625, -8)
# Execute `np.testing.assert_array_equal(a_codes, [[2, 7]])`
np.testing.assert_array_equal(a_codes, [[2, 7]])
assert not a_clipped.any()
np.testing.assert_allclose(a_restored, a_reference, atol=1e-12, rtol=0)
a_wrong = (a_qx.astype(np.int64) @ a_qw.astype(np.int64) + a_bias_codes) * a_sx * a_sw
# Execute `np.testing.assert_array_equal(a_wrong, [[-.125, .5625]])`
np.testing.assert_array_equal(a_wrong, [[-.125, .5625]])
_, a_saturated, a_small_clipped = output_grid(a_real, 1./256, -8)
assert a_small_clipped.all()
# Execute `np.testing.assert_array_equal(a_saturated, [[135./256, 135./`
np.testing.assert_array_equal(a_saturated, [[135./256, 135./256]])
print('Affine accumulators:', a_acc.tolist(), 'output codes:', a_codes.tolist())
print('Correct / missing zero point / clipped:', a_restored.tolist(), a_wrong.tolist(), a_saturated.tolist())

first_panel = visual_data
# Combine or mask array elements to form `visual_data`.
visual_data = {'panels': [first_panel, {'kind': 'bar', 'labels': ['output 0', 'output 1'], 'ylabel': 'output value', 'series': [
    {'label': 'correct affine / float reference', 'y': a_restored[0].tolist()},
    {'label': 'input zero point omitted', 'y': a_wrong[0].tolist()},
    {'label': 'narrow output grid clips', 'y': a_saturated[0].tolist()}]}]}
```

## Recorded reference execution

CPU run: 2026-10-20T10:50:41.243511+00:00. JAX 0.9.2.

```text
<class 'jax.numpy.float32'> max absolute error 0.0
<class 'jax.numpy.float16'> max absolute error 0.0010547637939453125
<class 'jax.numpy.bfloat16'> max absolute error 0.006171703338623047
W8A32 simulated output error 0.008031368255615234
W4A32 simulated output error 0.14571428298950195
<class 'jax.numpy.float32'> max absolute error 0.0
<class 'jax.numpy.float16'> max absolute error 0.0010547637939453125
<class 'jax.numpy.bfloat16'> max absolute error 0.006171703338623047
W8A32 simulated output error 0.008031368255615234
W4A32 simulated output error 0.14571428298950195
Affine accumulators: [[5, 15]] output codes: [[2, 7]]
Correct / missing zero point / clipped: [[0.625, 0.9375]] [[-0.125, 0.5625]] [[0.52734375, 0.52734375]]
Clipped values: 1 reconstructed: [0.19685039 0.8976378  1.        ]
Affine accumulators: [[5, 15]] output codes: [[2, 7]]
Correct / missing zero point / clipped: [[0.625, 0.9375]] [[-0.125, 0.5625]] [[0.52734375, 0.52734375]]
Small-channel max error; shared / per-channel: 0.03 7.874053e-05
Zero channel handled
W8A8 fixture error 0.00742650032043457
Code-origin invariance and bias-only zero input verified.
Weight-plus-scale subtotals: 24 14 11
PASS: deployment-05

```

## Calibrate an activation range, then shift it

**Predict before running:** What happens to a value of $4$ if calibration only covered $[-1, 1]$?

```python
# Experiment — Calibrate an activation range, then shift it: Do not recalibrate on the test set to hide this error.
calibration = np.linspace(-1., 1., 101, dtype=np.float32)
# Aggregate array values to compute `activation_scale`.
activation_scale = float(np.max(np.abs(calibration))) / 127
held_out = np.array([.2, .9, 4.], np.float32)
# Combine or mask array elements to form `activation_codes`.
activation_codes = np.clip(np.rint(held_out / activation_scale), -127, 127).astype(np.int8)
restored_activations = activation_codes.astype(np.float32) * activation_scale
clipped = np.count_nonzero(np.abs(held_out) > 127 * activation_scale)
assert clipped == 1 and abs(float(restored_activations[-1])-4.) > 2.9
print("Clipped values:", clipped, "reconstructed:", restored_activations)
```

**Expected:** One value clips and reconstructs near $1$ instead of $4$.

Do not recalibrate on the test set to hide this error. Collect representative calibration data or choose a policy that meets the task requirements.

## Include zero points, bias and the output grid

**Predict before running:** Can finer output spacing make the answer worse? Predict the two outputs when the output scale changes from $0.0625$ to $1/256$, with zero point $-8$ held fixed.

```python
# Deliberately exact grid values make an independent hand calculation possible.
a_sx, a_zx = .25, -3
a_sw = np.array([.5, .25], np.float64)
a_qx = np.array([[-3, 1, 5]], np.int8)
a_qw = np.array([[2, -1], [-1, 2], [1, 1]], np.int8)
a_bias = np.array([.125, -.0625], np.float64)
a_bias_codes = np.rint(a_bias / (a_sx * a_sw)).astype(np.int64)
# Check the wide result before representing it with an INT32 accumulator.
a_wide = (a_qx.astype(np.int64) - a_zx) @ a_qw.astype(np.int64) + a_bias_codes
assert np.all((a_wide >= np.iinfo(np.int32).min) & (a_wide <= np.iinfo(np.int32).max))
a_acc = a_wide.astype(np.int32)
# Execute `np.testing.assert_array_equal(a_acc, [[5, 15]])`
np.testing.assert_array_equal(a_acc, [[5, 15]])
a_real = a_acc.astype(np.float64) * a_sx * a_sw
# Independent real-valued input/weights, written without decoding the codes.
a_reference = np.array([[0., 1., 2.]]) @ np.array([[1., -.25], [-.5, .5], [.5, .25]]) + a_bias
# Execute `np.testing.assert_array_equal(a_real, [[.625, .9375]])`
np.testing.assert_array_equal(a_real, [[.625, .9375]])
np.testing.assert_allclose(a_real, a_reference, atol=1e-12, rtol=0)
def output_grid(real, scale, zero_point):
    # Guard input contract (`not np.isfinite(scale) or scale <= 0`) and fail fast if violated.
    if not np.isfinite(scale) or scale <= 0:
        raise ValueError('positive finite output scale required')
    # Guard input contract (`type(zero_point) is not int or not -128 <= zero_point <= 127`) and fail fast if violated.
    if type(zero_point) is not int or not -128 <= zero_point <= 127:
        raise ValueError('integer INT8 zero point required')
    unbounded = np.rint(real / scale) + zero_point
    clipped = (unbounded < -128) | (unbounded > 127)
    # Combine or mask array elements to form `codes`.
    codes = np.clip(unbounded, -128, 127).astype(np.int8)
    return codes, (codes.astype(np.float64) - zero_point) * scale, clipped

a_codes, a_restored, a_clipped = output_grid(a_real, .0625, -8)
# Execute `np.testing.assert_array_equal(a_codes, [[2, 7]])`
np.testing.assert_array_equal(a_codes, [[2, 7]])
assert not a_clipped.any()
np.testing.assert_allclose(a_restored, a_reference, atol=1e-12, rtol=0)
a_wrong = (a_qx.astype(np.int64) @ a_qw.astype(np.int64) + a_bias_codes) * a_sx * a_sw
# Execute `np.testing.assert_array_equal(a_wrong, [[-.125, .5625]])`
np.testing.assert_array_equal(a_wrong, [[-.125, .5625]])
_, a_saturated, a_small_clipped = output_grid(a_real, 1./256, -8)
assert a_small_clipped.all()
# Execute `np.testing.assert_array_equal(a_saturated, [[135./256, 135./`
np.testing.assert_array_equal(a_saturated, [[135./256, 135./256]])
print('Affine accumulators:', a_acc.tolist(), 'output codes:', a_codes.tolist())
print('Correct / missing zero point / clipped:', a_restored.tolist(), a_wrong.tolist(), a_saturated.tolist())
```

**Expected:** Accumulators [[5, 15]], output codes [[2, 7]], correct outputs [[0.625, 0.9375]]. Omitting the input zero point gives [[-0.125, 0.5625]]. The narrow output grid clips both outputs to [[0.52734375, 0.52734375]].

The first grid represents both values exactly. With scale $1/256$, the highest output is $(127+8)/256=0.52734375$, so both values saturate. A half-step rounding bound applies only without clipping. This is a NumPy arithmetic audit, not a native INT8 kernel or bit-exact LiteRT implementation: real runtimes specify operator layouts, multiplier approximations and rounding rules separately.

## Give small and large output channels different grids

**Predict before running:** If one output channel has weights near $0.01$ and another near $10$, what happens to the small channel under one shared INT8 scale?

```python
# Experiment — Give small and large output channels different grids: The global maximum determines a coarse grid for every channel.
heterogeneous = np.array([[.01, 10.], [-.02, -5.], [.03, 2.]], np.float32)
channel_codes, channel_scales = quantize_weights(heterogeneous, 8)
per_channel = channel_codes.astype(np.float32) * channel_scales
# Aggregate array values to compute `shared_scale`.
shared_scale = np.max(np.abs(heterogeneous)) / 127
shared = np.clip(np.rint(heterogeneous/shared_scale), -127, 127).astype(np.int8).astype(np.float32)*shared_scale
assert np.all(shared[:,0] == 0)
assert np.max(np.abs(per_channel[:,0]-heterogeneous[:,0])) < .001
np.testing.assert_allclose(shared[:,0], [0.,0.,0.], atol=0)
print('Small-channel max error; shared / per-channel:', np.max(np.abs(shared[:,0]-heterogeneous[:,0])), np.max(np.abs(per_channel[:,0]-heterogeneous[:,0])))
```

**Expected:** The shared scale rounds the entire small channel to zero. Its error is $0.03$; the per-channel error stays below $0.001$.

The global maximum determines a coarse grid for every channel. A per-channel grid protects the small channel here, at the cost of additional scale metadata and a runtime contract that must support the chosen axis.

## Make it yours

Add a zero-valued output channel and verify that its quantization scale is finite and its reconstruction is exactly zero.

### How to write this exercise — Step-by-step recipe & starter scaffold

**Key functions & syntax to use:**
- `jnp.isfinite(x)` — Returns a boolean mask verifying that no element is `NaN` or `Inf`.

**Step-by-step implementation plan:**
1. Confirm that all computed values remain finite (no NaN or Inf).
2. Execute `np.testing.assert_array_equal(q[:, -1], 0)`
3. Execute `np.testing.assert_array_equal((q * scale)[:, -1], 0)`

**Starter code scaffold (fill in the TODOs):**

```python
# Exercise solution: Add a zero-valued output channel and verify that its quantization...
with_zero = np.column_stack(...)  # TODO: compute with_zero
q, scale = quantize_weights(...)  # TODO: compute q, scale
# Confirm that all computed values remain finite (no NaN or Inf).
assert np.isfinite(scale).all()  # TODO: complete assertion check
# Execute `np.testing.assert_array_equal(q[:, -1], 0)`
np.testing.assert_array_equal(q[:, -1], 0)
# Execute `np.testing.assert_array_equal((q * scale)[:, -1], 0)`
np.testing.assert_array_equal((q * scale)[:, -1], 0)
print("Zero channel handled")
```

<details><summary>Reference solution</summary>

```python
# Exercise solution: Add a zero-valued output channel and verify that its quantization...
with_zero = np.column_stack((w, np.zeros(3, np.float32)))
q, scale = quantize_weights(with_zero, 8)
# Confirm that all computed values remain finite (no NaN or Inf).
assert np.isfinite(scale).all()
# Execute `np.testing.assert_array_equal(q[:, -1], 0)`
np.testing.assert_array_equal(q[:, -1], 0)
# Execute `np.testing.assert_array_equal((q * scale)[:, -1], 0)`
np.testing.assert_array_equal((q * scale)[:, -1], 0)
print("Zero channel handled")
```

</details>

## Compute W8A8 with a wide accumulator

**Transfer**

Use calibrated symmetric scales for $x$ and per-channel scales for $w$. Multiply integer codes with INT32 accumulation, rescale, and compare to the float baseline. Do not multiply int8 arrays directly.

<details><summary>Hint</summary>

Cast both integer-code arrays before multiplication. The real accumulator unit is the activation scale multiplied by each output channel’s weight scale; compare with the unchanged floating-point reference.

</details>

### How to write: Compute W8A8 with a wide accumulator — Step-by-step recipe & starter scaffold

**Key functions & syntax to use:**
- `jnp.allclose(actual, expected, rtol=..., atol=...)` — Checks that two arrays match elementwise within floating-point tolerance.

**Step-by-step implementation plan:**
1. Aggregate array values to compute `sx`.
2. Combine or mask array elements to form `qx`.
3. Perform matrix / vector contraction (`@`) to compute `accumulator`.

**Starter code scaffold (fill in the TODOs):**

```python
# Compute W8A8 with a wide accumulator (Transfer): This dense fixture uses zero points of zero and no bias.
# Aggregate array values to compute `sx`.
sx = float(...)  # TODO: compute sx
# Combine or mask array elements to form `qx`.
qx = np.clip(...)  # TODO: compute qx
qw, sw = quantize_weights(...)  # TODO: compute qw, sw
# Perform matrix / vector contraction (`@`) to compute `accumulator`.
accumulator = qx.astype(...)  # TODO: compute accumulator
y_integer = accumulator.astype(...)  # TODO: compute y_integer
assert accumulator.dtype  # TODO: complete assertion check
np.testing.assert_allclose(y_integer, reference, atol = ...  # TODO: compute np.testing.assert_allclose(y_integer, reference, atol
print("W8A8 fixture error", float(np.max(np.abs(y_integer-reference))))
```

<details><summary>Reference solution and reasoning</summary>

```python
# Compute W8A8 with a wide accumulator (Transfer): This dense fixture uses zero points of zero and no bias.
# Aggregate array values to compute `sx`.
sx = float(np.max(np.abs(x))) / 127
# Combine or mask array elements to form `qx`.
qx = np.clip(np.rint(x / sx), -127, 127).astype(np.int8)
qw, sw = quantize_weights(w, 8)
# Perform matrix / vector contraction (`@`) to compute `accumulator`.
accumulator = qx.astype(np.int32) @ qw.astype(np.int32)
y_integer = accumulator.astype(np.float32) * sx * sw
assert accumulator.dtype == np.int32
np.testing.assert_allclose(y_integer, reference, atol=.06, rtol=0)
print("W8A8 fixture error", float(np.max(np.abs(y_integer-reference))))
```

This dense fixture uses zero points of zero and no bias. A real affine operator must subtract zero points and handle bias and output requantization. Wide accumulation avoids int8 overflow; sufficiently long reductions can still overflow INT32.

</details>

## Change the code origin without changing the values

**Transfer**

Change the activation zero point from $-3$ to $17$, shift the input codes accordingly, and prove that predictions stay unchanged. Then feed codes representing real zero and identify the expected bias-only result.

<details><summary>Hint</summary>

Shift the zero point and codes by the same amount before subtracting; verify representability before casting back to INT8.

</details>

### How to write: Change the code origin without changing the values — Step-by-step recipe & starter scaffold

**Key functions & syntax to use:**
- `values(...)` — Call `values` with your updated parameters or inputs from this lesson's workspace.
- `a_qx.astype(...)` — Call `a_qx.astype` with your updated parameters or inputs from this lesson's workspace.
- `assert condition` — Verify that the observed output shape, status, or numerical value satisfies the contract.

**Step-by-step implementation plan:**
1. Execute `np.testing.assert_array_equal(shifted_acc, a_acc)`
2. Execute `np.testing.assert_array_equal(zero_result, a_bias[None, :])`

**Starter code scaffold (fill in the TODOs):**

```python
# Change the code origin without changing the values (Transfer): Integer codes are coordinates on a grid.
shifted_zx = ...  # TODO: compute shifted_zx
shifted_codes_wide = a_qx.astype(...)  # TODO: compute shifted_codes_wide
assert np.all((-128  # TODO: complete assertion check
shifted_codes = shifted_codes_wide.astype(...)  # TODO: compute shifted_codes
shifted_acc = ...  # TODO: compute shifted_acc
# Execute `np.testing.assert_array_equal(shifted_acc, a_acc)`
np.testing.assert_array_equal(shifted_acc, a_acc)
zero_codes = np.full(...)  # TODO: compute zero_codes
zero_result = ...  # TODO: compute zero_result
# Execute `np.testing.assert_array_equal(zero_result, a_bias[None, :])`
np.testing.assert_array_equal(zero_result, a_bias[None, :])
for bad_scale in (0., -1., float('nan')):
    try:
        output_grid(a_real, bad_scale, -8)
    except ValueError:
        pass
    else:
        raise AssertionError('invalid scale accepted')
print('Code-origin invariance and bias-only zero input verified.')
```

<details><summary>Reference solution and reasoning</summary>

```python
# Change the code origin without changing the values (Transfer): Integer codes are coordinates on a grid.
shifted_zx = 17
shifted_codes_wide = a_qx.astype(np.int64) + (shifted_zx - a_zx)
assert np.all((-128 <= shifted_codes_wide) & (shifted_codes_wide <= 127))
shifted_codes = shifted_codes_wide.astype(np.int8)
shifted_acc = (shifted_codes.astype(np.int64) - shifted_zx) @ a_qw.astype(np.int64) + a_bias_codes
# Execute `np.testing.assert_array_equal(shifted_acc, a_acc)`
np.testing.assert_array_equal(shifted_acc, a_acc)
zero_codes = np.full((1, 3), shifted_zx, np.int8)
zero_result = ((zero_codes.astype(np.int64) - shifted_zx) @ a_qw.astype(np.int64) + a_bias_codes) * a_sx * a_sw
# Execute `np.testing.assert_array_equal(zero_result, a_bias[None, :])`
np.testing.assert_array_equal(zero_result, a_bias[None, :])
for bad_scale in (0., -1., float('nan')):
    try:
        output_grid(a_real, bad_scale, -8)
    except ValueError:
        pass
    else:
        raise AssertionError('invalid scale accepted')
print('Code-origin invariance and bias-only zero input verified.')
```

Integer codes are coordinates on a grid. Changing the coordinate origin consistently preserves represented values. The zero-input check exposes a missing zero-point correction or bias in the wrong accumulator units.

</details>

## Account for actual low-bit storage

**Transfer / diagnosis**

Calculate FP32 storage, ideal packed four-bit storage, and the actual int8-backed four-bit simulation for the fixture. Include per-output-channel FP32 scales; keep biases and runtime workspace explicitly outside this subtotal.

<details><summary>Hint</summary>

Use nbytes for actual arrays. Packing two signed four-bit codes into one byte is not implemented by astype(int8).

</details>

### How to write: Account for actual low-bit storage — Step-by-step recipe & starter scaffold

**Key functions & syntax to use:**
- `storage(...)` — Call `storage` with your updated parameters or inputs from this lesson's workspace.
- `quantize_weights(...)` — Call `quantize_weights` with your updated parameters or inputs from this lesson's workspace.
- `assert condition` — Verify that the observed output shape, status, or numerical value satisfies the contract.

**Step-by-step implementation plan:**
1. Set up the changed input or configuration parameters requested in the prompt.
2. Run the target function or transformation and assert that the output matches your prediction.

**Starter code scaffold (fill in the TODOs):**

```python
# Account for actual low-bit storage (Transfer / diagnosis): The simulation uses 14 bytes for its codes and scales, not...
q4, s4 = quantize_weights(...)  # TODO: compute q4, s4
float_weight_bytes = ...  # TODO: compute float_weight_bytes
simulated_weight_and_scale_bytes = ...  # TODO: compute simulated_weight_and_scale_bytes
ideal_packed_weight_and_scale_bytes = ...  # TODO: compute ideal_packed_weight_and_scale_bytes
assert (float_weight_bytes, simulated_weight_and_scale_bytes, ideal_packed_weight_and_scale_bytes)  # TODO: complete assertion check
print('Weight-plus-scale subtotals:', float_weight_bytes, simulated_weight_and_scale_bytes, ideal_packed_weight_and_scale_bytes)
```

<details><summary>Reference solution and reasoning</summary>

```python
# Account for actual low-bit storage (Transfer / diagnosis): The simulation uses 14 bytes for its codes and scales, not...
q4, s4 = quantize_weights(w, 4)
float_weight_bytes = w.nbytes
simulated_weight_and_scale_bytes = q4.nbytes + s4.nbytes
ideal_packed_weight_and_scale_bytes = (w.size + 1)//2 + s4.nbytes
assert (float_weight_bytes, simulated_weight_and_scale_bytes, ideal_packed_weight_and_scale_bytes) == (24,14,11)
print('Weight-plus-scale subtotals:', float_weight_bytes, simulated_weight_and_scale_bytes, ideal_packed_weight_and_scale_bytes)
```

The simulation uses $14$ bytes for its codes and scales, not the ideally packed $11$. Tiny tensors emphasize metadata overhead; large models also need block/group metadata, alignment, biases and working memory.

</details>

## Check your understanding

What must a deployment policy specify beyond “INT8 weights”?

1. Activation, accumulator and output formats, scales, calibration and supported kernels.
2. Only the filename extension.
3. Nothing; INT8 guarantees acceleration.

<details><summary>Answer and explanation</summary>

Activation, accumulator and output formats, scales, calibration and supported kernels.

Bit width alone is incomplete. Arithmetic, scale metadata, calibration and runtime support determine both behavior and practical benefit.

</details>

## Diagnose the result

NaNs in FP16 suggest checking range and sensitive reductions. Finite but degraded predictions suggest rounding or clipping; inspect tensor ranges and per-channel errors. No memory improvement from the INT4 exercise is expected because its codes use int8 storage. No latency gain after conversion calls for profiling fallback operators and data transfers on the target.

## Keep your evidence

Keep a precision table listing weight, activation, accumulator and output dtypes; max error and clipping counts on changed inputs; calibration provenance; and storage estimates that include scales and packing.

Save your predictions, modified code, terminal outputs, and reasoning in your engineering portfolio.

## Primary references

- [Keras mixed precision policies](https://keras.io/api/mixed_precision/)
- [JAX dtype defaults and X64](https://docs.jax.dev/en/latest/default_dtypes.html)
- [ONNX Runtime quantization](https://onnxruntime.ai/docs/performance/model-optimizations/quantization.html)
- [LiteRT integer conversion](https://developers.google.com/edge/litert/conversion/tensorflow/quantization/post_training_integer_quant)
- [LiteRT INT8 operator, zero-point and bias contracts](https://developers.google.com/edge/litert/conversion/tensorflow/quantization/quantization_spec)