Tag

Distributed Systems

Articles and publications tagged Distributed Systems across the Atmosphere.

45articles9publications

Articles

Publications

DevOps - The Web's Largest Collection of DevOps ContentJul 24, 2026

When the Message Broker Becomes the Bottleneck: Scaling Push and SMS With MongoDB Polling

BlogsBusiness of DevOps

The Cloudflare BlogJul 8, 2026

Introducing Meerkat: an experiment in global consensus

ResearchNetwork

Redowan's ReflectionsJun 27, 2026

Request coalescing with Go singleflight

GoConcurrency

Redowan's ReflectionsJun 27, 2026

Request coalescing with Go singleflight

A hot cache key expires and a hundred requests issue the same query at once, saturating the database. Go's singleflight package coalesces those duplicate calls into one. How to wire it up, how to measure whether it's firing, and why per-pod coalescing is usually enough.

GoConcurrency

Redowan's ReflectionsJun 13, 2026

Reading your own writes with WAIT FOR LSN in Postgres 19

DatabaseSQL

Redowan's ReflectionsJun 13, 2026

Reading your own writes with WAIT FOR LSN in Postgres 19

PostgreSQL 19's new WAIT FOR LSN command lets a replica block until it has replayed your write. The read-after-write problem it solves, the workarounds it replaces, and what the timeout, status, and mode options are actually for.

DatabaseSQL

Alessandro BahgatMay 31, 2026

The nearest hospital to every place on Earth, in a single S2 range query

For every locality on Earth, how far is the nearest hospital? Naively, that's 437 billion great-circle distances. S2 cell indexing reduces it to a single integer range-join: build the index once, and the answer for every place on Earth comes back in seconds. The same primitive also resolves which country, region, and locality each hospital belongs to, in a single pass over one index.

algorithmsspatial

waowMay 26, 2026

typeahead? more like typebehind! amirite

typeahead.waow.tech is an independent, free-to-use typeahead service used by several atproto apps today operating it thus far has been not the worst! i seeded the index by listening to select collections from jetstream and following bsky moderation events. i use turso.tech bc it rocks and i synced a read-replica to the backend machine to avoid extra turso reads... same playbook as pub-search.. which was fine until it was not! yes. yes, it was. ty for reporting! (pckt is our second biggest user!)

atprotoindexing

Jeff BaileyMay 25, 2026

What Is the AT Protocol? A Developer's Mental Model

Distributed SystemsProtocols

Sahil Kapoor's PlaybookMay 12, 2026

Replica Set

A replica set is MongoDB's name for a group of database nodes that maintain the same data and provide automatic failover. One node is the primary (accepts writes); the rest are secondaries that asynchronously apply the primary's operation log. If the primary becomes unreachable, the secondaries elect a new primary among themselves. How it works Writes go to the primary, which records them in its operation log (oplog). Secondaries tail the oplog and apply the same operations locally, eventuall

MongoDBReplication

Sahil Kapoor's PlaybookMay 12, 2026

Sharding

Sharding is the practice of horizontally partitioning a dataset across multiple database instances so each shard holds a subset of the data and serves a subset of the load. Sharding is the standard answer when a single database server cannot keep up with storage, read throughput, or write throughput. How it works A shard key (one or more fields) determines which shard each row or document belongs to. A router or proxy uses the shard key to dispatch each query to the right shard, gather partia

MongoDBPostgreSQL

Sahil Kapoor's PlaybookMay 12, 2026

CAP Theorem

The CAP theorem states that a distributed data store cannot simultaneously provide all three of: Consistency (every read returns the latest write), Availability (every request receives a response), and Partition tolerance (the system continues to operate despite network splits). Under a network partition, a system must give up either consistency or availability. What the three properties mean here * Consistency (C). Specifically linearizability: a global, real-time ordering of operations suc

Distributed SystemsConsensus

Sahil Kapoor's PlaybookMay 12, 2026

Eventual Consistency

Eventual consistency is the property that, given enough time without new updates, all replicas of a piece of data will converge to the same value. Reads may see stale or out-of-order data in the short term, but the system guarantees convergence rather than instantaneous global agreement. Why it exists Strong consistency (every read sees the latest write) requires coordination on every write, which costs latency and reduces availability under network partitions. Eventual consistency relaxes th

Distributed SystemsConsensus

Sahil Kapoor's PlaybookMay 12, 2026

Consensus

Consensus is the problem of getting a set of distributed processes to agree on a single value despite failures, message delays, and out-of-order delivery. Consensus protocols are the foundation under any replicated state machine, leader election, distributed lock, or strongly consistent database. The classical guarantees * Agreement. All correct processes decide the same value. * Validity. The decided value was proposed by some process. * Termination. All correct processes eventually decid

Distributed SystemsEventual Consistency

Sahil Kapoor's PlaybookMay 12, 2026

Saga

A Saga is a pattern for executing a business transaction that spans multiple services or databases by chaining local transactions together, with compensating actions to roll back partial progress on failure. Sagas replace distributed transactions (two-phase commit) in microservice systems where global ACID across services is impractical. How it works A saga breaks a business transaction into a sequence of steps. Each step is a local transaction in one service. If any step fails, the saga runs

MicroservicesEvent-Driven Architecture

Sahil Kapoor's PlaybookMay 12, 2026

Event-Driven Architecture

Event-Driven Architecture (EDA) is a style in which services communicate by publishing and subscribing to events on a message bus, rather than calling each other synchronously. Producers emit events when something happens; consumers react to those events independently. The pattern reduces coupling and enables fan-out, replay, and asynchronous processing. How it works A producer publishes an event (typically a JSON or Avro message) to a topic on a broker. The broker stores the event durably an

MicroservicesSaga

Sahil Kapoor's PlaybookMay 12, 2026

Modular Monolith

A modular monolith is a single-deployable application built with strict internal module boundaries: clear ownership per module, explicit interfaces between them, no shared mutable state across boundaries. The result combines monolithic operational simplicity with most of the structural benefits of microservices. How it works Each module owns its own data, its own domain logic, and exposes a typed interface (functions, ports, or in-process events) that other modules call. Module-private code c

MonolithMicroservices

Sahil Kapoor's PlaybookMay 12, 2026

Monolith

A monolith is an application packaged and deployed as a single unit: one codebase, one build, one process, one database. Everything ships together. Most applications start as monoliths and many remain so for their entire useful life; the term has acquired a pejorative tone in recent years that obscures how often it is the right choice. Strengths * Simple operations. One thing to deploy, one set of logs, one stack trace per error. * Easy local development. Run the whole system on a developer

MicroservicesModular Monolith

Sahil Kapoor's PlaybookMay 12, 2026

Cache Stampede

A cache stampede (or thundering herd) is the failure mode where many concurrent requests for the same hot key all miss the cache at the same time, hit the origin simultaneously, and overwhelm it. Stampedes typically happen the instant a popular key expires, when a cache is cold-started, or when a downstream system briefly fails and recovers. How it happens Consider a homepage feed cached with a 60-second TTL. Hundreds of requests per second hit the cache and get a fast cached response. The mo

CachingTTL

Sahil Kapoor's PlaybookMay 12, 2026

Idempotency

Idempotency is the property that repeating an operation produces the same outcome as performing it once. An idempotent operation can be safely retried after a network failure, timeout, or partial failure, without risking duplicate side effects. Idempotency by HTTP method * GET, HEAD, OPTIONS: safe and idempotent by definition * PUT, DELETE: idempotent (replacing or removing the same resource repeatedly has the same final state) * POST: not idempotent by default; APIs that need it expose an

REST APIHTTP

Redowan's ReflectionsApr 11, 2026

Dynamo: Amazon's highly available key-value store

Distributed SystemsDatabases

Redowan's ReflectionsApr 11, 2026

Dynamo: Amazon's highly available key-value store

Key takeaways from Amazon's 2007 Dynamo paper.

Distributed SystemsDatabases

Jeff BaileyApr 8, 2026

Fundamentals of Software Traffic Management

FundamentalsSoftware Engineering

Redowan's ReflectionsApr 7, 2026

Stacked log lines considered harmful

GoDistributed Systems