reimplementing inference

reimplementing a model's inference means reimplementing its exact preprocessing, feature hashing, and quantization decisions — not its architecture diagram. the weights are the part that gets attention; the feature extraction is the part that drifts, and it drifts quietly, because wrong features still produce confident output. worked example throughout: spacez, a zig port of spaCy's en_core_web_sm NER pipeline.

features are part of the model

spaCy's small english NER model is ~1.5M float32s. before any of them run, the pipeline turns a document into four hashed features per token — a normalized form, the first character, the last three characters, and a shape string — plus a tokenization that splits don't into do and n't.

every one of those is a decision the reference implementation made, and the weights were trained against those decisions. get the shape rule wrong and the embedding table is indexed at a row that means something else. the network has no way to signal this. it produces a prediction, in range, plausible.

a reimplementation of en_core_web_sm passing its whole fixture suite — 10 NER cases, 15 tokenizer cases — disagreed with spaCy on 13.5% of real social-media posts. none of the 25 fixtures contained a character above ASCII, and the four most common divergences all did — the character-defined features had been implemented over byte slices, which is exactly right for ASCII and wrong past it (see languages/ziglang/text).

measure the feature, not the outcome

the tempting metric is the model's output: how often do the two implementations produce the same entities. it is the wrong instrument for finding a preprocessing bug, because it moves for many reasons at once and in both directions. two rounds were spent watching that number improve, regress, and improve again while the actual defect stayed uncharacterized.

the useful measurement is the feature itself. dump the reference's value for every distinct token in a real corpus, dump yours, and diff:

norm agreement: 6634/6658 = 99.6%   mismatches: 24

that number names the remaining work: twenty-four tokens, all clitics. "83% of spans agree" names nothing — it moves when the tokenizer changes, when the label set changes, and when the weights are fine and the shape rule is not.

read how the reference keys its tables

lookup tables are where assumptions hide. spaCy's norm table is keyed by the 64-bit hash of the lowercased token, not by the token — vocab.strings[text] on the way in, so a table dumped as text recovers only the entries whose strings happened to be interned. 3,510 entries, 248 of them resolvable.

the same hash function was already in the port, cross-checked against spaCy in an existing script. keying the table the way the reference keys it recovered all 3,510 and needed no strings at all.

this was found the slow way — scraping a corpus for observed pairs — after guessing wrong about a symbol constant (NORM is 67, not 69). the source is MIT licensed and one file.

values scoped to a rule

spaCy norms n't to not. it does that because the rule that split don't attached that norm to that piece, not because those three characters always mean negation. folding such values into a general text→value table generalizes them past their scope:

that trade — 24 conservative misses for 16 confident errors — measured better on the downstream metric and is worse. carrying the value on the token the rule produced fixes it, and the downstream metric agrees once the mechanism is right.

what to build first

a fidelity harness, before any fixing. reference implementation on one side, port on the other, a corpus from the domain the port actually serves, and a diff at the level of the thing you are changing. it turns "seems better" into a number, and it is the only way to notice that a change helped the metric while being wrong.

sources

• spacez — a zig port of spaCy's en_core_web_sm NER pipeline; feature parity work in be0f81f and 8cc1a7a (2026-08-18), taking span agreement from 79.3% to 84.5% on 1,415 posts
• coral — the consumer whose junk entities started the investigation