04 · How it was built

The first one never really worked. This is what the rebuild did differently.

87 slices across two products in under four months — 75 in DUDE, 12 in the client portal — each one planned, adversarially reviewed, built, reviewed again, deployed, verified live and signed off before the next one started.

The before

Why we threw the first version away

There was an earlier DUDE. It was rebuilt from scratch specifically because it never really worked: statuses malfunctioned, uploads misfiled, and the work chased features instead of correctness.

Both of the real failures were classes, not bugs. Uploads had to land in the right job folder "directly and reliably" — misfiled uploads were the defining failure of that build. And job folders were being moved between four status-based shared drives by script, so a failed move meant a lost folder and dead links.

That is why the rebuild starts with a line that reads as doctrine rather than a note: this is a fresh build; it inherits no code and no architecture from the first DUDE. And why the governing principles — the ones that override any feature request — are prefaced by an admission: most of them exist because the first build failed on exactly these points.

Less is more. Fewer features, built correctly, beat many features that half-work. The first build chased features while core functions — status tracking, document filing — never worked reliably.
Governing Principle 1, frozen 9 July 2026

Requirements first, with no technology in the room

The requirements document was written and signed off before any stack was discussed — deliberately, so that no technical convenience could shape what the business actually needed. It was then reviewed externally, six architectural directions were argued out and given written statuses, and a read-only proof of concept was run against the live QuickBooks account. Only then was an architecture approved. The reasoning is one sentence: decisions on paper are cheap to change; decisions discovered mid-build are not.

The method

Every slice runs the same eight steps

A "slice" is one capability — small enough to verify completely, large enough to be worth deploying. Only one is ever in flight, and that holds globally — across the main track, the fleet track and the portal, which lives in a separate repository that cannot see this one. Each repo carries a lock line naming what is in cadence; a missing line counts as a stop, because absence is not the same as "nothing". The order is mine to choose and nobody starts one out of turn.

Step 1

Kickoff draft

What we're building, why, and what could go wrong — plus a mandatory pass that tries to refute the draft.

Step 2

Code planning

The implementation plan, file by file, with the tests it will need.

Step 3

Plan review

Fresh adversarial eyes on the plan, before build step zero. Verdict is binary.

Step 4

Build

The code gets written.

Step 5

Code review

A second independent adversarial pass — this time on what was actually built.

Step 6

Fixes

Only findings that carry an instruction, not an opinion.

Step 7

Deploy and live pass

On production, on a real iPad for anything field-facing. Every screen the slice touched gets opened.

Step 7b

Manuals

Written from verified reality, after the live pass, never before.

Step 8

Sign-off commit

The commit is the last act — not the deploy.

Why two separate adversarial reviews

Step 3 reads the plan and cannot see the code. Step 5 reads the code and cannot un-choose the plan. They catch genuinely different things, and the lessons file has been counted: the pre-build passes together are the largest source, with code planning the single biggest step; the post-build review comes next, and deploying and checking it live is a close third. Only two lessons in the whole file came from production itself, which is the entire point. One plan review caught a method call that didn't exist on the object it was called on; shipped, it would have crashed every single promote.

Findings carry a severity that is explicitly "a decision about when, not a feeling about importance", evidence at a specific file and line, and an instruction. The reason for that last requirement is the sharpest process line in the whole project: a finding without an instruction becomes an argument in step 6 instead of an edit.

Step 1's refute pass has two lenses. One re-derives every citation in the draft against the current code — not "does this line still exist" but "does it still say what the draft claims". The other attacks the premise: is this the problem we set out to fix? That second question needed its own owner, because adversarial review is structurally silent on scope. It will happily make a plan more correct without ever asking whether the plan should exist.

Rule 1

The check itself needs checking

A test verified in one direction, a probe reasoned about rather than run, a count quoted without re-reading the output — each is confidence wearing the costume of evidence.

Rule 2

A verification recorded in no file does not exist

If it isn't written down against a checkable value, it will be re-argued in three weeks by someone who wasn't there.

Rule 3

Never re-derive what an earlier step recorded clean

The discipline has to be affordable, or it gets skipped exactly when it matters.

The timeline

From frozen requirements to two live products

2026-07-09Requirements signed off and frozen. Fifteen statuses locked. No technology discussed anywhere in the document.
2026-07-10First commit. Proof of concept specified, run and the architecture approved — all in one day. The probe killed a wrong assumption before it could reach the architecture.
2026-07-13Five slices in one day. Document filing lands. Misfiling becomes structurally impossible.
2026-07-14Six more slices, and a founding decision reversed. Automatic emailing of signed documents was dropped on day four — DUDE would stay email-free.
2026-07-16First full-codebase review. Per-slice history moves out of the boot file, which gets defended as a boot file from then on.
2026-07-21Second review. The cadence is formalised. One high, nine medium, twenty low findings — and the method stops being a habit and becomes a written process.
2026-07-23Third review, plus a hardening programme. Hardening is run as slices, not as cleanup between them.
2026-07-24DUDE 1.0 released. Fourteen days after the first commit. 785 tests at that point.
2026-07-28Two unrelated work stoppages in one day. The worst day in the record — and the source of two permanent rules. Both stories are below.
2026-08-12The slice playbook becomes binding. The lessons file is seeded. The process itself becomes a document with a version and an owner.
2026-08-14DUDE becomes the official company platform. The field-trial rules were made permanent rather than allowed to expire with the trial.
2026-08-18A review that produced 115 candidate findings. 98 verified, 17 refuted. A finding had to survive refutation before it counted as one.
2026-08-19The one permitted identity reset is spent. A permission granted exactly once, consumed on the record, and now closed forever.
2026-08-21Slice 45 signed off. 1,702 tests and 42 migrations. At this point the platform was four weeks old and already running the company.
2026-08-26A second product gets permission to start. The client portal had been in planning for three and a half weeks. The build hold was lifted in writing.
2026-08-30A slice ships with no deploy at all. The first time in the project's history that a sign-off did not move the production revision — because the diff was tests and documents. Story below.
2026-09-01The two products start talking. Three firsts for DUDE in one slice: it serves machine-caller JSON, an outside application authenticates into it, and it stores rows another system originated.
2026-09-18A catch-up plan turns a scattered backlog into one ordered programme. Twenty-one open questions walked through in a single sitting, and the order confirmed rather than proposed.
2026-09-23 → 09-30Seven slices in eight days. Several ran all eight steps inside a single day. The programme's first half closes with a timezone sweep.
2026-09-30A timezone bug is caught on production inside the two-hour window it lives in. After 7pm Central, with the server already on tomorrow's date, a receipt dated tomorrow was refused. Observed, not inferred.
2026-10-07Every slice becomes a branch and a pull request. Merged with a merge commit, never squashed or rebased — because the documentation cites commit hashes and a rebase would rewrite every one of them.
2026-10-08The client portal goes live on its own domain. First real client request on 20 September, on the old address; the domain landed three weeks later.
2026-10-16The fleet problem-report button is switched on for every technician. One database write, given its own sign-off line and its own live pass — including the first two reports, real records against a real van, filed that afternoon to prove it.
2026-10-20The fleet rollout closes, and a planned feature is withdrawn before it starts. Every active vehicle entered and labelled. The withdrawn slice keeps its number forever rather than being reused.
2026-10-21Both halves of request comments ship the same day — and a deep review of both repositories is commissioned that evening. It ran overnight across six phases and returned 194 findings.
2026-10-27All 194 findings placed, exactly once, by script. And the first real cloud bill was read: fifteen weeks of estimates replaced by a measured number, comfortably inside the budget.

The hard parts

Ten things that went wrong

This is the honest half. Every one of these is written up in the build history with what it cost and the rule it produced, because a failure nobody wrote down gets to happen twice.

1. An outage caused by a file permission

What happened: A template file shipped with a restrictive permission mode. The container copies files in and then drops to an unprivileged user — so the app could not read its own template, and every attempt to render that screen failed. Version control tracks only the executable bit, so the change produced a clean status and an empty diff.
Why nothing caught it: 967 tests passed against it. Every test, the dev server and every manual check all run as the file's owner, and the owner could read it fine.
Cost: Three deploys shipped it. Live and silent for about four hours.
Rules: A test suite of 160 cases that asserts file modes as the container user, run before any build. A standing warning: a file's mode is production state that version control does not track. And a rule that a live pass must open every screen the slice touched — because that slice's live pass never opened the one screen the slice was about.

The hotfix exposed a second trap: the deploy script ships the working tree, not the last commit — and the tree at that moment held about 1,700 uncommitted lines. A hotfix deploy is only ever as clean as the working tree.

2. The deadlock that hangs forever instead of failing

What happened: A deliberately mutated test exposed a whole class of bug: if a second database connection writes a row that references a row your open transaction already holds locked, the two lock modes conflict — and because both connections belong to the same client, Postgres sees no cycle. It does not raise a deadlock error. It hangs, indefinitely.
Cost: Nothing. The audit that followed found no live instance of it.
Rule: Multi-statement atomic work uses one connection. That rule has been load-bearing several times since.

3. Techs couldn't clock out, and nothing had regressed

What happened: Shifts created by DUDE carried no accounting class, and QuickBooks Time refuses a clock-out without one. Nothing had broken: DUDE had simply never set that field. It didn't matter until DUDE became the tool of record, at which point every new shift arrived blank.
The uncomfortable part: The evidence was already in the repository and nobody had looked. A proof-of-concept sample from weeks earlier held 233 events — 216 of them carrying the field, and every DUDE-created one showing it empty.
Cost: A work stoppage, on a payroll day.
Rule: Read-only probe, then a bounded write gate, then a live experiment with a negative control. The control is the part that's easy to skip and the part that did the work.

4. A test harness that lied three times before it told the truth

What happened: At clock-out, DUDE would fill a missing accounting class from whichever job held the shift's jobcode at that moment. That is a guessed value written onto a payroll record DUDE cannot un-write — from a code path whose stated rule is never guess.
The three lies: Eleven mutations were run to prove the guard actually caught it. One mutant was a no-op. One referenced a test name that didn't exist, and a collection error reads exactly like a caught mutation. And one was defeated by a stale bytecode cache: the mutation was a line swap, so file size and timestamp never changed, and Python ran mutated bytecode from a correct source file.
Cost: Nothing shipped wrong — every mutation ended red once the harness itself was fixed. The bill was a day spent chasing the harness instead of the code.
Rules: Baseline-pass first, assert the mutant actually differs, clear the bytecode cache. And: verify a sequence reset by reading its value, never by calling the function that consumes one.

5. Three correct reviews on a slice nobody needed

What happened: A slice passed steps 1 through 3, including an adversarial review that found four genuine blockers. Then I asked out loud whether we were fixing a problem that didn't exist. We weren't: the manual repair the slice existed to retire had never once been run, and a note a week earlier had already said so.
Cost: A fully scoped slice half parked. Three files and 89 lines survived.
Rule: Three adversarial passes each made the plan more correct and none of them made it smaller, because none of them was asked to — which is why "is this the problem we set out to fix?" now has a step of its own.

6. A permission granted exactly once, and spent on the record

What happened: Fleet asset labels are derived from database ids and get physically painted on the vans, so reusing an id is not a data problem, it's a paint problem. Four test assets were holding the first four ids. I granted exactly one exception to reset the sequence — before rollout, while nothing was painted — and wrote down that after it, the rule binds absolutely and forever.
Cost: Two cascade findings had to be dealt with first: Drive folders do not follow a database delete, so they would have been orphaned behind reissued ids.
Rule: No further reset on that table, ever. And an applied migration's comment is never revised — the record of what was true when it ran stays as written.

7. "Additive is safe" was true of the database and false of the client

What happened: Migrations run before the new revision takes over, because the old revision can't see a new column. It can't — but the database driver silently starts pre-preparing a query after its fifth execution, and one of ours selects every column of a table. Adding a column changed that query's result shape, so the old revision started failing for the life of its prepared statement.
Cost: A tech's gas-receipt submission failed at 12:54:40 UTC, before any write — nothing recorded, no purchase posted. It was found by the post-deploy error scan of production logs. No test and no review had seen it, and it had probably been latent on every previous additive migration to that table.
Rule: A deploy-order safety claim is about the client's query cache as much as the server's schema. Closed as a class by turning statement preparation off entirely.

8. What 1,702 green tests did not prove

What happened: A slice was spent asking what the suite didn't cover. The answer: of 154 web addresses, 36 were swept by a test. The other 118 were protected by nothing but the author remembering to type one line. Ship an admin screen without its guard and all 1,702 tests stayed green.
The interesting part: None of it was a bug. Every one of the 118 was gated correctly. They were guarantees the code genuinely held, held by discipline, pinned by nothing. Two other promises turned out to be proved at their ends and not their middle — including a nightly endpoint whose entire defence was replaced by a stub in every test that got past it.
Why the obvious fixes were rejected: Asserting "every route refuses a technician" is false — plenty are supposed to admit one. Reading the guard off the running function is circular: it proves the route carrying the guard carries the guard. And a hand-maintained list of expected answers is the same author discipline, one directory over. So the test parses the guard out of the source text, calls the live route, and asserts the two agree.
Cost: Eighteen new tests, no application code, and no deploy — the first sign-off in the project's history that left the production revision where it was. The session was authenticated and able to deploy, and deliberately didn't.
Rules: Available credentials are not a reason to deploy. And the slice wrote down what it does not prove: that the routes are gated as declared, never that the declarations are right — and the hand-check that is the only evidence for that does not renew itself.

9. A reviewer's explanation standing in for a test

What happened: A code review found a comment giving a false reason why a particular screen was safe — refuted with a single crafted request. The fix was written, and a mutation was written to prove the fix worked.
The part worth telling: That mutation survived the entire 2,109-test suite. It only died once the missing test arm was added. For as long as that gap existed, a confident paragraph of prose had been doing the job of a test, and nobody could tell the difference by reading.
Rule: Every mutation is now run against a green control first — the same test, the same harness, alone — because a kill only counts against a control. And the harness fails loudly when it is handed a test name that doesn't exist, having been fooled by that twice.

10. Two tripwires that had already fired, and nothing was listening

What happened: An overnight review of both repositories found one high-severity item: an imaging library, pinned in both products, against which twenty-eight advisories — three of them critical — had been published the night before the review ran. Nothing in the estate had done anything wrong; the advisories post-dated every security check ever taken.
The real finding: Underneath it: two dependencies were each pinned under a written rule saying "this moves only when an advisory is published against it" — and both triggers had already fired, weeks earlier. Four deploys had shipped straight past them. A test environment dated two months earlier had never been rebuilt and was testing a dependency set production did not run.
Cost: An emergency upgrade in both products, each with its own live verification.
Rule: A trigger keyed on an outside event is not a trigger until something reads that event. A rule with no reader is a hope. The remedy is an advisory query wired into the deploy's own preflight, so the check runs whether or not anyone remembers it.

The same review then reviewed its own method and found it wrong: a deduplication step had merged 51 candidate findings on the assumption they were duplicates, and a recovery pass found 28 of them were distinct defects standing on their own. The lesson it wrote for itself: deduplicate by defect, not by the file the finder happened to be looking at.

The paper trail

Why there is so much writing

262 documents, roughly 2.5 million words, against about 65,000 lines of Python — 24,000 of application code and 41,000 of tests. That is around 38 words of record per line of code, and I would not run it any other way — because every session starts without memory of the last one, and the written record is the only thing that carries a hard-won rule forward.

Kickoffs — 79

One per slice and the file of record for it — steps 2 through 8 append to the same document — plus the template, the fix passes and the incident write-ups.

Walkthroughs — 77

A plain-language code tour per slice. Shipping one is part of the definition of done.

Reviews — 45

Whole-codebase code reviews, fleet reviews, documentation passes, and the overnight deep review of both products — plus the ledger that answers "was this dealt with, and where?" for every finding any of them produced.

The build history

Over 6,000 lines, one section per slice, opening with the standing warnings and still-live traps — a preamble that ends: nothing here is theoretical.

The idea backlog

What might ship next, what will never ship, and a numbered decision log now past a hundred entries. Nothing in it is ever deleted — a deleted entry is a decision nobody can audit.

The lessons file

107 cross-slice lessons under an eight-code taxonomy, written down because the project kept rediscovering the same ones. Its first rule is the one that governs the rest: the check itself needs checking — and it deliberately refuses to print its own total, because a count written in prose is a promise to update prose.

One rule about citations, learned the hard way. Documents cite each other by identifier and section, never by line number. The lessons file was born with sixteen line-number citations, every one correct that day — and every one wrong twenty-four hours later, broken by the very commit that created the file.

Previous← The client portal NextUnder the hood →