compression
compression
written against zig 0.16.
std.compress.zstd (decode)
zig std can decompress zstandard — no vendored libzstd needed for plain-frame decode (this is what let zat consume jss archive blocks with zero new dependencies):
var in: std.Io.Reader = .fixed(compressed);
var d: std.compress.zstd.Decompress = .init(&in, &.{}, .{});
_ = try d.reader.streamRemaining(&allocating_writer.writer);measure before shipping it on a hot path: on real jss archive blocks (atproto-bench jss lane, 58 frames / 61 MB decompressed) std's decoder took ~410 ms where libzstd took 22 ms — ~20× — and the gap survived every buffering strategy (allocating writer, reused window + discard, preallocated). zat ended up vendoring facebook/zstd v1.5.7 decode-only (10 C files, ~150-line build.zig addition, cross-compiles) and got 2× ahead of the go reference. std zstd is fine when the wire is the bottleneck.
sharp edges, verified in the 0.16 source:
- decode only. there is no compressor in std.
verify_checksum: truepanics — "not implemented yet" inDecompress.zig. frame content CRCs are silently skipped, so integrity must come from an outer layer (zat uses the jss segment xxh3 checksum;std.hash.XxHash3has streaming init/update/final).- no dictionary support (
DictionaryIdFlagUnsupported). fine for plain frames; not for dictionary-compressed streams like jetstream subscribe-v2 wire compression. - with a non-empty buffer,
initasserts capacity ≥options.window_len + zstd.block_size_max; the empty-buffer + writer form (above) is the simple path for buffered whole-frame decode.
sources
- zat — vendored zstd v1.5.7 decode-only, jss archive block decode
Did you enjoy this article?
Recommend it — Standard Reader surfaces well-loved writing to more readers across the network.