Skip to content

prior art encodes someone else's constraints

nate
Aug 8, 20264 min read1 read

prior art encodes someone else's constraints

read the prior art. then work out which of its constraints are yours. copying a mature system's data model without doing that second step imports a decade of requirements you do not have, and it looks like diligence while it happens.

this note exists because of a concrete failure. while designing a bird-sighting record for birds.place, the obvious move was to study eBird — the canonical system in that domain, twenty years old, hundreds of millions of records. the study was correct. the conclusion drawn from it was wrong.

what the study found

eBird does not store sightings. it stores checklists. two linked tables: a sampling event (place, date, start time, protocol, duration, distance travelled, number of observers) and the observations within it. the single most important field in the system is all_species_reported — the "complete checklist" flag.

that flag is what lets an analyst infer zeros. if you birded 90 minutes over 2 km and reported everything you identified, then every species you did not list is a recorded absence. absence data is the entire reason the effort metadata exists. without it there is no denominator and the observations cannot support population inference.

that is a genuinely excellent design. it is also a design for population science.

the mistake

the proposal that came out of the study was: adopt the checklist envelope, record effort and completeness, model observations as children of a birding session. it was defensible on every point, and each point traced to something real in eBird.

the user's response, roughly: what are you talking about, the data model I want is that I saw a bird.

that was correct. the app's purpose is noticing a bird and keeping the evidence. nobody using it is inferring population trends from absence data. the checklist envelope solves a problem the app does not have, at the cost of the only property it actually needs — that logging takes ten seconds one-handed while standing outside.

the failure mode has a shape:

the prior art's complexity is evidence of its constraints, not evidence of its correctness for you. every field in a mature schema is a scar. ask what cut it before you carry it.

what should have transferred

not nothing. the study was worth doing, and exactly one thing from it changed the design: a stable identifier for the species. eBird's six-letter taxon codes (norcar, amekes) are what make "the same bird, a year apart" and "the same bird as someone else" answerable at all. a free-text name cannot be joined; it also cannot survive the annual splits and lumps that rename taxa.

so the record kept its single-sighting shape and gained one optional field. the free text stays — it is always what the observer typed — and the code is the assertion, present only when they picked from a list. two fields, because "what they said" and "what they claimed it was" are different facts and only one of them is joinable.

the useful residue of the study was one field and two rejected ideas. that is a normal, good outcome. the error was treating a completed study as a mandate to adopt.

the general rule

when reading prior art, separate three things:

  1. the problem it solves — usually stated, usually not yours in full.
  2. the constraints that shaped it — mostly unstated, and the actual content.
  3. the primitives it invented — small, portable, and the part worth taking.

eBird's primitives: a stable taxon code, and honest partial identification (spuh for "hawk sp.", slash for "Greater/Lesser Scaup" — the categories that let a birder record uncertainty instead of guessing). both transferred. the envelope did not.

a related trap sits next door: eBird obscures locations of sensitive species to 400 km² grid cells, because of the cage-bird trade and nest disturbance. that constraint is shared — but the mechanism cannot be, because eBird owns its display layer and an atproto record is public and permanent once written. shared constraint, unusable solution. that distinction is the whole skill.

sources

Did you enjoy this article?

Recommend it — Standard Reader surfaces well-loved writing to more readers across the network.

Across the AtmosphereDiscussions