When Does a Specification Become a Program?

Someone asked me recently where the boundary lies between a specification and an implementation.

The question has been floating around a lot this year. Gabriella Gonzalez put it directly in an essay titled A sufficiently detailed spec is code. Her argument is partly economic: if producing a specification precise enough for an agent requires doing essentially the same intellectual work as programming, we haven't escaped the hard part.

Mario Zechner made a related point in his AI Engineer talk Building Pi in a World of Slop: whatever you leave unspecified gets filled in by the model, and those decisions are no longer yours.

Both objections matter for regenerative software.

Imagine starting with an English description of a system. It isn't precise enough, so we add requirements. Then invariants. Examples. Schemas. Conditions. State machines. At some point pseudocode seems clearer than another paragraph of prose. Eventually we may include executable fragments because they express some requirement more accurately than English can.

Keep going and the specification starts looking suspiciously like a program.

But there are really two questions hiding here.

How much work does it take to specify a system precisely enough that we trust what gets generated?

And once our durable description contains prose, tests, schemas, formal constraints, executable examples, and maybe actual code, what distinguishes that description from the implementation itself?

The first question is about whether specification actually saves intellectual labor. The second is about where the identity of a regenerative system lives.

I'm interested in both, but especially the second.

There will be no perfect specification

One seductive version of AI-native development starts with an oracle.

Describe the desired system perfectly. Give that description to a sufficiently capable machine. Receive a correct implementation. Delete it whenever you like and regenerate another.

The fantasy is hiding inside the word perfectly.

We have never been very good at describing complex systems completely before building and operating them. There is little reason to expect that changing the recipient of the description from a programmer to an LLM suddenly fixes that.

The first version of a specification represents our current understanding of the problem. Reality immediately begins editing it.

A dependency fails in a way nobody anticipated. A customer discovers a workflow we never modeled. Something that is logically correct takes eleven seconds in production and is therefore unusable. A regulatory requirement appears. An implementation passes every acceptance test while a business metric quietly deteriorates.

Those discoveries change what we know the system must do.

I wrote about this in Production Is a Compiler Input. Production gives us evidence that should flow back into the inputs to the next implementation. The durable description of the system has to be able to learn from that evidence.

This is also where Zechner's gap-filling concern becomes important. Every specification is incomplete. The generator will make decisions inside those gaps.

Humans have always filled gaps too, but there is an important difference in scale. A human programmer's decisions were at least made by someone present in the process and could plausibly be reviewed. An agent can fill thousands of unspecified details in seconds, most of which nobody will ever inspect individually.

Some of those decisions are harmless implementation choices. Others will turn out to matter.

The problem is knowing which of those decisions should remain ephemeral and which ones should become part of what the system knows about itself.

That distinction only becomes visible over time.

Specifications are allowed to contain code

Suppose a requirement is most accurately expressed as an executable function.

Maybe there is a financial calculation with enough edge cases that ten lines of code capture its meaning more faithfully than two pages of English. Maybe a property is naturally expressed in temporal logic. Maybe the clearest description of acceptable behavior is a collection of executable examples.

There is no advantage in translating those into prose simply to preserve a neat border between specification language and programming language.

An English sentence can describe an implementation detail. A piece of executable code can define an obligation.

We already accept this in testing. A test is code, but nobody concludes that the test suite and the application under test are therefore the same artifact.

If an executable property says:

decode(encode(x)) == x

then every implementation of the encoder and decoder is constrained by that property.

One implementation might use JSON. Another CBOR. Another a custom binary representation. The executable assertion remains valid across all of them.

I wrote about this in Evaluations Are the Real Codebase. Durable evaluations survive implementation changes because they express obligations at boundaries that outlive the current code.

The useful distinction is not whether something runs. It is what authority that artifact has over future implementations.

A specification is probably not a document

The word specification still carries an unfortunate image with it: a large document.

Requirements Document v7-final-FINAL.docx

I don't think that shape survives. I mean I seriously hope it doesn’t. Sheesh.

A real system accumulates many different kinds of assertions.

A checkout service may have an obvious behavioral requirement: a valid payment should result in an order.

It may also have an operational constraint saying p95 response time must remain below 400ms under a particular traffic shape. There may be an invariant that the same payment authorization cannot create two orders, a jurisdiction-specific rule, examples of known edge cases, and a business constraint saying that abandonment above some threshold is unacceptable even if the software is technically functioning.

Different representations are appropriate for different kinds of knowledge.

Natural language is good at some of them. Schemas are good at others. Property tests, formal constraints, examples, diagrams, measured production evidence, and executable reference functions all have their place.

I suspect the useful abstraction looks closer to a knowledge graph than a requirements document: a collection of structured assertions with relationships, provenance, evidence, and different projections for different consumers.

A human might ask it for a readable design document. An agent might ask it for all constraints governing checkout. An evaluator might ask which obligations can be mechanically checked. A production system might attach new evidence to an existing assertion.

The document becomes one view onto the underlying knowledge.

How do you read a specification like this?

If the specification is a graph of assertions, constraints, evidence, provenance, and executable fragments, an obvious question follows:

How does a human read it?

Probably the same way we read almost no large structured system directly.

You don't read the raw representation from top to bottom. You query it.

Ask for everything related to performance. Everything that constrains checkout. Every requirement introduced after a particular incident. Every business rule that depends on a specific regulation. Every assertion that has weak evidence. Every part of the system whose current implementation depends on knowledge discovered in production.

Or ask an LLM to explain a subsystem to you.

Generate a dependency graph. Produce a requirements document for a particular team. Show the relationship between business objectives and the evaluations that enforce them. Trace a production failure backward through the constraints and decisions that led to the current implementation.

The raw representation is storage, not necessarily an interface.

This is part of what makes the graph model appealing to me. It separates the canonical knowledge from the way any particular person needs to consume it.

There is an old idea hiding in here. More than twenty years ago, XML and XPath promised something similar: keep information in a structured form, then query and transform it into whatever view you need. When I wrote the first edition of The Passionate Programmer, I did it in a proper XML editor with first class support for XML as a data model. It was an amazing glimpse into a future that didn’t quite pan out. The reality of XML was often much uglier than the promise, but the underlying idea was powerful.

LLMs make that idea much more interesting.

You no longer need to know the exact query language or schema before asking a useful question. You can ask, "Why does this service have a 100ms latency requirement?" or "Show me everything we learned from the outage last March," and have the system traverse the underlying knowledge for you.

The specification stops being a thing you read.

It becomes a thing you interrogate.

Detail does not collapse the distinction

Hillel Wayne made a useful distinction in his response to this debate: a specification describes a set of acceptable implementations. Making the specification more precise shrinks that set.

That is a cleaner way to think about detail.

Suppose we completely specify an API's externally visible behavior: schemas, error conditions, authorization rules, latency envelope, idempotency guarantees, ordering constraints, examples, invariants, and compatibility requirements.

There may still be many programs that satisfy it.

One implementation could use PostgreSQL. Another could use FoundationDB. One could be written in Rust, another in Go. One could use actors internally, another conventional request handlers. They might have almost no source code in common.

More precision narrows the implementation space. It does not automatically collapse specification and implementation into the same artifact.

And if some small portion of a system becomes so tightly constrained that only one sensible implementation remains, that does not create a philosophical crisis. That implementation can simply become part of the durable system knowledge.

The more interesting question is what needs to survive if we decide to generate a different implementation later.

What mature implementations know

There is a harder problem.

Existing systems already contain knowledge that their specifications don't.

I wrote about this recently in The Implementation Remembers. Mature codebases accumulate odd retries, strangely specific timeout values, defensive validations, exceptional workflows, and ugly-looking conditionals. Some are junk. Others are compressed history.

That history matters enormously if we want to regenerate software.

Suppose a query has an inexplicable index hint. An agent regenerating the service might reasonably remove it. Then production latency explodes because the hint was added three years ago after discovering pathological planner behavior at a particular data distribution.

The index hint preserves the discovery, but only indirectly. What we learned was something about acceptable behavior under real conditions.

Perhaps the real lasting fact is:

Under the observed production cardinality, this query must remain below 100ms at p95.

Once that fact is represented explicitly, a future implementation is free to solve the problem differently.

The index hint might survive. It might be replaced by a different index, a rewritten query, a cache, or an entirely different storage engine.

The important thing is that the production lesson no longer depends on preserving the particular code that first embodied it.

This is where regeneration becomes interesting to me. Mature software contains a great deal of encoded experience. We need mechanisms that progressively move that experience into artifacts which survive the current implementation.

What is actually permanent?

This is why I would not make specification carry the entire conceptual load.

Earlier in this series, in The Phoenix Primitives, I described four things that need to survive regeneration: specification, evaluation, context boundaries, and provenance.

They answer different questions.

The specification records what we require.

Evaluations provide evidence that an implementation satisfies those requirements.

Context boundaries define the obligations between independently regenerable parts.

Provenance records how the current state arose.

None of these needs to be a particular kind of file or language.

A schema can define a boundary. Python can express an evaluation. English can state a requirement. Production telemetry can supply evidence. A conversation can create provenance.

The architecture comes from the relationships among them.

Where the boundary matters for Phoenix

The set-of-programs definition tells us something useful about what a specification is, but Phoenix has a slightly different architectural problem.

We need to know what survives.

The counterfactual I find most useful is:

If I throw this implementation away and generate another one, does this information have to survive?

If it does, the information belongs somewhere in the durable system.

It may be specification. It may be an evaluation, a boundary definition, provenance, a reference implementation, or production evidence. Phoenix does not require all durable knowledge to fit under the word specification.

That distinction matters because some executable artifacts should survive regeneration and some prose should not.

A 300-line state machine may encode a protocol that every future implementation has to preserve. Meanwhile, a page of English explaining the internal organization of today's cache implementation may safely disappear with the code.

The format tells us very little. The regeneration test tells us much more.

This changes the economics of precision

This is where Gonzalez's objection bites hardest.

If maintaining the durable description requires humans to manually reproduce every implementation decision in another form, we have gained nothing. We’ve created two codebases and called one a specification.

Historically, that is close to what formal specification often meant in practice.

You could write the program, or you could write an elaborate model of the program and then write the program too. Keeping the two synchronized created a permanent tax. Most teams understandably allowed the code to become the most precise description of the system.

Generative systems may change that cost structure.

Agents can extract candidate constraints from implementations. They can compare production behavior with declared expectations. They can generate evaluations from requirements and flag disagreements. They can connect production evidence to assumptions. They can produce multiple human and machine-readable views from the same underlying facts.

Humans still have to decide what matters.

That is the intellectual work we cannot automate away by renaming code a specification.

But we may be able to automate much of the clerical work involved in keeping different representations synchronized.

If that happens, richer external representations of system knowledge become economically plausible in a way they usually were not before.

This does not make Gonzalez's objection disappear. It changes where the cost lies.

The difficult work becomes deciding which facts are authoritative, which observations deserve promotion into durable knowledge, which constraints are accidental, and which freedoms we want future implementations to retain.

That seems like a better problem for humans to be spending their time on.

Regeneration needs a learning loop

There is one more consequence of all this.

If specifications are incomplete, implementations will inevitably contain decisions that do not yet exist in the durable system.

Some of those decisions will later prove important.

A regenerative architecture therefore needs a path in the other direction.

Generation takes durable system knowledge and produces an implementation.

Operation produces evidence.

That evidence changes our understanding of the system.

Some of what we learn gets promoted into the durable layer and constrains the next generation.

This is the loop I care about:

knowledge

    ↓

implementation

    ↓

production

    ↓

evidence

    ↓

revised knowledge

Without that return path, regeneration becomes amnesia.

You can generate fresh software forever while repeatedly forgetting everything previous implementations learned the hard way.

The useful system is the one that gets harder to fool every time it runs.

Has anyone built the complete version?

I haven't seen a substantial production system that can throw away arbitrary pieces of itself and reliably regenerate them from a complete external body of system knowledge without meaningful loss.

The ideas are ahead of the tooling.

But “complete regeneration or nothing” is not a useful adoption threshold.

A system can become progressively more regenerative.

Move one important invariant out of implicit code and into a durable evaluation.

Make one service boundary explicit enough that either side can be replaced.

Recover one production lesson from an implementation and represent the constraint it exposed.

Take one component and reach the point where deleting its implementation is boring.

Each step changes where the system's knowledge lives.

Over time, less of its identity is trapped in the current source tree.

And this gets back to the question in the title. A specification does not become a program merely because it gets detailed, formal, or executable. The boundary that matters appears when we ask what must survive regeneration and what can be allowed to disappear with this particular implementation.

A sufficiently detailed specification may contain prose, schemas, tests, formal models, executable examples, and actual code.

The important question is what remains after the program burns.