Floats spend bits on dynamic range. Integers don’t have any. That sounds like a downgrade, and for general-purpose computing it would be. For ML inference — where weights and activations have already been normalized to roughly known ranges, and the bandwidth wall is the wall you’re hitting — it’s a feature. Integers fit four times as densely in the same SIMD register, move at four times the rate through cache, and burn a fraction of the energy per operation. The whole “quantize a neural network” industry exists because the math will tolerate the precision cost in exchange for the throughput. This section is the bridge: integer representations on their own, plus the trivial-sounding “scale × integer” trick that lets integers represent reals.

Integer types and ranges

Six integer types you’ll meet in ML code, with the ranges you can hold:

All signed integers use two’s complement representation Standard signed-integer encoding: the most significant bit is the sign (1 = negative), and a negative value is stored as the bitwise complement of its magnitude plus one. Lets the CPU add signed numbers using the same circuit as unsigned addition. Range is asymmetric: −2ⁿ⁻¹ to 2ⁿ⁻¹ − 1. — the asymmetry (int8 goes to −128 but only +127) is a consequence. The math just works: addition, subtraction, and multiplication use the same circuits for signed and unsigned operands; only comparisons and the high bits of multiplication differ. Most importantly for ML, the cost of an integer op is the same as the cost of a smaller-range integer op — int8 + int8 is no faster than int8 + int8, but it lets you fit four additions into the silicon footprint of one int32 + int32.

Fixed-point — the simplest way to represent a real

You have integers. You want to represent reals. The simplest possible scheme:

Pick a scale (a float, chosen once for a whole tensor or group). Store one integer per value. To recover the real: multiply. That’s fixed-point core term A real number stored as an integer plus a single scale factor: real = scale × integer. The 'fixed' refers to the implicit decimal point being at a fixed position (vs. floating point where the exponent moves it). Special case of affine quantization with zero point = 0. . It’s older than IEEE-754 by a few decades — every embedded system that didn’t have an FPU did its arithmetic this way — and it’s the substrate underneath modern quantization forward ref The process of approximating a continuous-valued tensor with a discrete (usually integer) representation. The simplest case is uniform fixed-point with one scale per tensor; production schemes add zero-points (asymmetric), per-channel scales, and clipping ranges. (§3.3 adds one more parameter — the zero-point — and you have everything modern frameworks use).

Two consequences fall out:

1. Every representable value is exactly scale apart. The integers are 1 apart, so their real-valued images are scale apart. Anywhere on the number line — near zero, far from zero, same gap.
2. The representable range is exactly [−128 × scale, 127 × scale] for signed int8. Outside that, you have to clamp (“saturate”), which loses information silently. Choosing a scale is choosing this range.

The viz below makes the contrast with float visible. Slide the int8 range — what you can represent grows wider, the gap between adjacent values grows with it, but the uniformity is preserved. Float, by contrast, packs its representable points logarithmically: half of them are between -1 and +1, the rest exponentially spread out.

Why integers run faster

Three reasons, in increasing depth:

1. SIMD lane density. A 256-bit AVX2 register fits 8 float32 lanes — or 32 int8 lanes. Four times the work per instruction at the same register width. The 512-bit AVX-512 ratio is 16 vs 64. NEON gives you 4 vs 16. The ratio is fixed by physics: a lane needs space for its bits, and bits are bits.
2. Memory bandwidth. A 1 MB cache holds 256 K floats — or 1 M int8 values. Four times the working set in the same footprint. Bandwidth from main memory likewise pushes 4× as many int8 values per second. For memory-bound kernels (and every neural network inference kernel above ~50M parameters is memory-bound on consumer hardware), this 4× shows up directly in wall time.
3. Energy per op. Integer arithmetic skips the exponent unit and the renormalisation step that floats need. On big GPUs and inference-specialised silicon (TPU, Apple Neural Engine, Habana Gaudi) the energy ratio is roughly 4–10× in favour of int8. For batched inference at scale that’s the difference between profitable and unprofitable.

The accumulator problem (revisited)

If you multiply two int8 values, the result can need up to 16 bits (since 127 × 127 ≈ 16 K and -128 × -128 = 16 K, both well outside the int8 range). If you then sum many such products in a dot product, the accumulator needs to handle the sum. For a dot product of length 1024 of int8 × int8 products, the sum can reach ~1024 × 16 K ≈ 1.6 × 10⁷ — fits in int32 (range ±2 × 10⁹) with room to spare, but not in int16 (range ±32 K).

This is the same argument as §1’s accumulation problem, in integer form: the accumulator needs more bits than the operands. Production int8 dot product kernels uniformly accumulate into int32 for this reason. The fused intrinsics (_mm_maddubs_epi16 widens to int16; _mm256_dpbusd_epi32 widens further to int32; NEON’s SDOT directly into int32) are designed precisely to chain operand-width multiplies into accumulator-width sums, in hardware.

We’ll see all of this in §3 when we build the kernel.

The fixed-point formula in code

Plain C, no SIMD yet — just the conversion in both directions:

/* Encode a real value into a scaled int8. */
int8_t real_to_int8(float x, float scale) {
    float r = x / scale;                    /* unscale */
    int rounded = (int)lrintf(r);           /* nearest-even rounding */
    if (rounded < -128) return -128;        /* clamp to int8 range */
    if (rounded >  127) return  127;
    return (int8_t)rounded;
}

/* Decode an int8 back to a real. */
float int8_to_real(int8_t q, float scale) {
    return scale * (float)q;
}

Two notes worth keeping:

• Round, don’t truncate. Truncating (cast to int) biases negative values toward zero and positive values away. lrintf rounds to nearest-even, the IEEE-754-blessed rounding mode, which is unbiased.
• Clamp, don’t wrap. Casting a 200.0f to int8_t in C is undefined behavior (overflow during the float-to-int cast); even if it gave you the wrapped value (-56), that’s a catastrophic error in the dot product output. Production quantizers always clamp.

These two lines of policy are the whole protocol of affine quantization forward ref The next-up generalization of fixed-point: real ≈ scale × (quantized − zero_point). The zero_point lets the integer range be centered anywhere, not just at zero. §3.3 derives why this matters for asymmetric distributions like ReLU outputs. — the §3.3 generalization just adds a zero_point to recenter the integer range when the data isn’t centered at zero.

END OF CH.3 §2 — Integers and fixed-point.
Built: SpacingCompare viz contrasting float32’s log-uniform spacing with int8 fixed-point’s uniform spacing; three recall items spanning the structural contrast, SIMD throughput math, and absolute-vs-relative error.
Coming next: §3.3 — Affine quantization and the int8 dot product kernel. The substrate of every quantized inference stack.