Ahmed YassinCase study 04 / 04

← All work

Building a team's worth of checks when there was no team

Company
DropX
Role
Co-founder & Product Lead
Dates
Jan 2025 – Present
Domain
Platform delivery & risk

“Nobody was going to review my pull requests, so I built the reviewer.”

release lane · schematic

worktree → pull request → reviewtest environment → acceptancemasterproduction — denied by defaultan override leaves a marker in the transcript

Context

DropX is a live B2B dropshipping marketplace. It takes real orders, moves real goods, and settles real money to merchants who are counting on it. I am a co-founder and its product lead, and for most of its life I have also been the person shipping it — while holding a full-time product role elsewhere.

That is the constraint this study is about. Everything below exists because of one fact: there was no second pair of eyes.

The challenge

The usual answer to “how do you ship safely” is a team — someone reviews your code, someone else owns releases, someone notices when a change touches the payments path. None of that was available. The realistic failure modes were specific and expensive:

  • Double-paying a merchant, or paying against numbers that silently recomputed.
  • Leaking customer PII into a log, an error report or an analytics event — in a market where phone numbers and addresses are the payload of every order.
  • Touching production while intending to touch test. One flag, one wrong project, and the mistake lands on live orders.
  • Quietly editing something load-bearing — a schema, a security rule, a credentials file — as a side effect of doing something else.

Moving slowly and carefully is not a defence against any of these. Care is exactly the thing that runs out at 1am when a merchant is waiting.

What I did

I stopped treating the missing team as a gap to compensate for, and started treating each of its functions as something to specify.

Made review mandatory rather than virtuous

Every code-writing agent in the system is deliberately incapable of committing or pushing. They read, edit and write within a narrow scope, and then hand back. A reviewer runs on the resulting diff before anything lands; a second, architectural reviewer is required when a change spans three or more feature domains or touches the database schema.

The point is not that a review happens. It is that shipping without one is not a path that exists — I cannot skip it by deciding I am confident today.

Encoded the risk model as auditors that fire on the risky paths

The checks that matter are domain-specific, so they trigger on the files that carry the risk rather than running everywhere. Editing the payments or settlements path triggers an audit that asks the questions I would want asked: is every settlement mutation idempotent, and is there a test proving a duplicate call cannot double-pay? Is the insufficient-data case handled explicitly instead of failing open? Does the audit trail record every state change?

Editing observability or analytics triggers a different one, which checks new logs, error reports and analytics payloads against a written list of the fields that count as personal data here — names, phones, addresses, national IDs, card fields, IP addresses — and fails if any of them can escape unscrubbed.

Those checklists are the same ones a good reviewer would carry in their head. Writing them down is what made them survive my own bad days.

The clearest catch was the PII one. Two WhatsApp endpoints that had already merged were writing the raw client IP into structured logs. The audit found it on live code, and the fix replaced the address with a boolean — the on-call engineer still sees that the rate limiter had something to key on, and the IP never reaches the log. Elsewhere the reviewer blocked a release where error responses on an encrypted path bypassed the encrypting sender, and caught a trigger that would have let an admin record an automated action as though the system had done it, which would have made the audit trail lie.

Made production hard to touch and impossible to touch accidentally

Production and test are separate projects, and the rule is that the target is always named explicitly — never inherited from whatever the local environment happens to point at, because that default is exactly how the wrong project gets deployed to.

A guard sits in front of every command and refuses destructive operations against production outright. Overriding it requires an explicit marker on the command, which stays visible in the transcript afterwards as a record that the override was deliberate. Test deployments are pre-approved and need no ceremony. The asymmetry is the design: the safe path is frictionless, and the dangerous one costs a conscious act.

A second guard escalates any edit to a schema, a security rule, an environment file or a credentials file into a decision I have to make by hand, rather than something that happens inside a larger change I am half-reading.

The stated default, written down, is: when uncertain about production — don’t.

Built a release lane with a way back

Work happens in an isolated worktree off the shared test branch, goes to test through a pull request, gets exercised there, and only then is promoted to production — manually, from a specific known-good commit, with a deployment manifest kept so a rollback has a real target instead of a guess.

Feature flags with email allowlists carry the same idea into the product: per-order settlement went live switched off, visible only to an email allowlist, while the model it replaced sat dormant behind the same flag rather than being deleted.

Outcome

One person shipped the whole product surface of a live marketplace — catalogue, orders, fulfilment, courier integrations, settlement, an operations console, an analytics warehouse — from concept to public launch in January 2026, and it has run live since. Behind it sits a written release lifecycle, 43 scoped agents and 6 enforcement hooks.

The part I would put in front of an engineering leader is not the count. It is that the system is structural rather than advisory. Nothing in it depends on me remembering to be careful: the reviewer cannot be skipped, and the payments audit fires whether or not I thought the change was risky.

With one honest asterisk. The review layer has a record — I can point at the diffs. The production guards have none, because they write no log when they fire. I believe they have never had to. I cannot prove it, and a control you cannot audit is a control you are trusting rather than verifying, which is the exact thing this system was built to stop me doing.

What I’d do differently

I built the checks in the order I got scared, not in the order of expected loss. The production guard is dated the day after the incident that prompted it. That is not a system preventing failures; that is a system commemorating them, and I only noticed the pattern when I went looking for evidence that any of it had ever worked.

Worse, the review layer has a blind spot it cannot see. A reviewer correctly flagged that a release would break — and the fix it endorsed took down every admin order list in production, on a query that had passed every gate, because the test suite mocks the database and the failure only exists against the real one. The check was right, the fix was wrong, and nothing in the system was positioned to tell the difference. A green pipeline over a mocked dependency is a confidence generator, not a safety net.

If I were starting again I would keep the same architecture and far fewer pieces: guard the money path, guard personal data, guard production, and make review unskippable. Those four are load-bearing. Everything else was a file that has to stay true as the codebase moves underneath it — and a check that has quietly gone stale is worse than no check, because it still reports PASS.

The honest tension underneath all of it: this system is a substitute for colleagues, and it is a good one for correctness and a poor one for judgement. It catches a duplicate payment. It has never once told me I was building the wrong thing — which, as the settlement rebuild shows, was the more expensive mistake.