Skip to content

reconciled configuration

nate
Oct 23, 20262 min read

reconciled configuration

when an automation reconciles a configuration value toward a source of truth (a deploy script, gitops, terraform, a kubernetes operator), the live system stops being a place to change that value. a hand edit there lasts until the next reconcile. the failure is worse than drift because the operator verified the change, at the wrong layer, and the belief "i fixed that" survives until the next alert.

  • change the source the reconciler reads, not the object it writes. if an emergency forces a live edit, change the source in the same session; the live edit is a cache write.
  • find the reconciler before recording a fix as durable. search the repository and CI configuration for the value you changed. a builder timeout raised with fly machine update -e read back correctly and was reverted by the next deploy, whose script wrote and asserted the old constant.
  • an assertion enforces the source, right or wrong. a deploy script that asserts a value stops drift, and stops your fix while the source is wrong. once the source is corrected, the same assertion rejects a deploy that would regress it.

a reconciler is an offer of durability: put the change where it reads and it applies everywhere. the other valid end state is to remove the owner. policy-enforced-by-accident is the opposite case, a rule with no owner.

deploy paths that reuse stored state

the failure flips when the deploy path reads its configuration from the previous deployment instead of the declared source. with helm upgrade --reuse-values, the release record becomes the source of truth and the values file in version control is consulted only when a deploy passes it explicitly. one reconcile from hand-built values silently drops whatever the file declared, and every later --reuse-values deploy carries the loss forward while the file in git still shows the intended config; a dropped memory limit shows up as OOM kills weeks later. every upgrade should render from the versioned source; the release stores output, never input.

sources

  • pub-search docs/builder-offbox-plan.md; commit 71b4e1e (the fix, applied in the reconciler source); commit d3348bf (the reconciler deleted when the builder moved off fly)
  • my-prefect-server deploy/prefect-values.yaml, commit f5e11ef (2026-08-25): env and a memory limit restored after a hand-built helm reconcile dropped them

Did you enjoy this article?

Recommend it — Standard Reader surfaces well-loved writing to more readers across the network.

Across the AtmosphereDiscussions