embedded redis (the burner-redis pattern)

run a redis-compatible engine in-process, with no external server. the use case: self-hosted servers that want redis semantics (streams, atomic EVAL, consumer groups) without telling users "go set up a separate redis."

the contract

an embedded redis engine is "redis-compatible" if:

• the data model matches (string / hash / set / zset / stream / list with the same semantics)
• the EVAL scripts you'd write against real redis run unchanged
• the wire-shape replies (RESP-shaped) match for the same inputs

what you skip: the actual TCP server, replication, AOF/RDB persistence (replace with whatever serialization makes sense), cluster, ACLs. for a backing-store-for-one-process role, none of those matter.

why an in-process engine instead of "fake it with a HashMap"

three reasons:

1. EVAL. the moment you want atomic multi-key operations, you need a Lua interpreter and a single mutex around the whole store. the scripts encode the semantics — duplicating the docket scheduler/promote/ack logic in zig directly means maintaining two implementations.
2. streams. consumer groups + PEL + XAUTOCLAIM are non-trivial state machines. once you've written them once for real redis, redoing the in-process version is a clean port not a redesign.
3. drop-in. the dispatching layer (redis.call(cmd, ...)) is exactly the same surface a real redis client exposes, so the consumer of the engine has one code path instead of two.

the upstream

PrefectHQ/burner-redis is the rust crate this pattern comes from. it backs pydocket + self-hosted prefect. zig port lives at tangled.org/zzstoatzz.io/burner-redis.

architecturally:

Store { data: HashMap<Bytes, ValueEntry>,
        scripts: HashMap<sha1, source>,
        mutex (single, around whole store) }

ValueEntry { data: ValueData, expires_at: ?Instant }
ValueData = String | Hash | Set | ZSet | Stream | List

every command method on Store locks → checks expiration → branches on variant → mutates → unlocks. that's the whole architecture.

EVAL on top of the store

LuaEngine.eval(script, keys, args):

1. spin up a fresh Lua VM (no state leaks between scripts)
2. open base, string, table, math (skip io/os/package for sandboxing)
3. push KEYS and ARGV as 1-indexed Lua arrays
4. push redis.call and redis.pcall as Lua C functions, capturing the Store pointer via lightuserdata upvalue
5. load + run the script under pcall (so Lua errors don't unwind through C)
6. convert the script's return value into a RedisValue tagged union

cost: a fresh Lua VM per call. for the scale of docket (the distributed-task system this engine backs; ~1 EVAL per task per worker tick) it's invisible; for high-throughput EVAL you'd pool the VMs.

persistence

the upstream uses MessagePack for snapshot + restore. crash-safe write (write to tmp + fsync + rename). expired keys excluded from snapshot. this part is optional — for "memory-mode prefect" with idempotent seed-on-start, restarts re-create the scheduled work and you can skip persistence entirely.

dual-backend deployments

the high-leverage pattern: redis://... vs memory://... URLs that resolve to the same backend interface. docket does this — a single Backend union over RedisBackend and MemoryBackend. the same Lua scripts EVAL against both. prefect-server picks per-deploy.

PREFECT_BROKER_BACKEND=memory  → memory://prefect → in-process burner-redis
PREFECT_BROKER_BACKEND=redis   → redis://host:6379/0 → external redis

the consumer doesn't know which one it has. that's the whole point.

what doesn't fit in-process

• pub/sub at scale: you can implement it locally but you lose the natural fanout-across-processes that real redis pub/sub gives you.
• replication: there's no replica to ship to. if you want HA, swap to real redis.
• the actual "wire" RESP server: you skip it entirely — consumers call the engine's methods directly, no socket layer.