watermark semantics

a log-structured system accumulates cursors: how far readers may read, how far data is durable, how far it may be evicted, how far a replica has caught up, how far compaction has run. these usually coincide, they are all integers over the same sequence space, and there is always a moment where reusing an existing one for a second purpose looks free. that convenience is the smell — this is the one-value-one-meaning thesis of change-timestamps, at the cursor layer.

visibility, durability, retention, and replication want separate cursors. they coincide often enough that a single variable passes review, and diverge under exactly the conditions that matter: large batches, slow disks, an overloaded replica, a stalled compaction pass.

the cost of a gate scales with the quantum of the thing it waits on. gating visibility on durability is invisible when flushes are frequent; at production block sizes the identical code is a latency floor. the gate's author never observes it because the development-loop quantum is small — which is the normal case, not an unlucky one. when adding a gate, ask what the waited-on quantum is in production, not in the test.

in a parity port, divergence is a bug even when it looks like prudence. a deliberate-sounding comment ("only now may subscribers observe…") makes a divergence look designed, which delays diagnosis more than an obvious mistake would. what upstream does is the specification; extra safety is still a diff and needs a written justification against upstream's actual code.

measured example (stream): a durable watermark — pinning rows against eviction until the archive fsyncs — was also used to hide rows from readers. nothing was lost and nothing errored, but live delivery quantized to the archive's block-seal cadence: 13-second bursts of ~450 events separated by silence, flapping every short-window rate meter downstream between zero and burst-peak. the upstream being ported read from the tip cursor, not the durable one.

related

• change-timestamps — the shared thesis, and the same failure at the column layer
• serving-from-snapshots
• operations/bounded-scans — a watermark used as a work bound, and why it does not bound reads
• operations/observability-of-absence

sources

• stream — the 13-second-burst live delivery bug, fixed in cf461ef (2026-08-07)