redis sorted-set hot paths

written against zig 0.16.

Notes from bringing the Zig sorted-set implementation in burner-redis (an in-process redis-compat engine) up to parity with the upstream Rust library.

representation first

The winning changes were structural, not clever syntax:

• use an arena for sorted-set member bytes so ZADD does not allocate one member at a time and deinit does not need to walk every key just to free member memory
• keep the score-ordered slice and member index mirrored, and test that invariant after allocator-failure paths
• store the member hash alongside score-ordered entries when range deletion needs to remove many members from the member index
• build small internal modules for the sharp edges: byte hashing, member index, score sorting, dense lookup, and diagnostics

This matched the useful lesson from zat/atproto-bench: make the ordinary representation boring and measurable before specializing a benchmark shape.

diagnostic shape

A single benchmark number hid too much. The useful diagnostic split was:

• plan-loop overhead
• direct score loads
• hash-only cost
• prehashed member-index lookup
• ordinary member-index lookup
• public command path
• construction stages for ZADD: copy, index, sort/place

That made it obvious when a local micro-optimization was fake. Several ideas looked good in isolation but lost in the full command path because they added branches, cache pressure, or random swaps.

dense lookup

For fixed-width benchmark members like member/0000000000042, a lazy dense score array can make ZSCORE faster than a hash lookup. Keep the generic path correct, and specialize only after validation:

• build the dense table lazily on the first ZSCORE
• invalidate it on every sorted-set mutation
• keep fallback member-index lookup for non-dense names
• parse fixed-width numeric suffixes with an unrolled parser only after measuring

Do not eagerly build dense lookup during ZADD unless the benchmark proves the trade: it moved work into insert and made the total core benchmark worse.

allocator discipline

The zig zen line that mattered here was:

Resource allocation may fail; resource deallocation must succeed.

For Redis-like commands that replace one key from another, build the destination value completely before removing/replacing the old key. Add tests that inject allocation failure and then verify both indexes still mirror each other.