Essay

Where the engineer signs

On keeping judgment human in an AI build: what a model should never decide, and how to make that a rule instead of a hope.

2026-06-184 min read

A model can write the code. It cannot be on the hook for it. When a system makes a decision that turns out wrong — leaks the data, can't scale, can't be changed — accountability has to land somewhere, and it cannot land on a model. It lands on a person, with a name, who looked at the decision and signed it. The entire question of building responsibly with AI comes down to drawing that line precisely: what the model is allowed to decide, what it never is, and how you make that a rule instead of a thing you hope happened.

Accountability doesn't have an API

You can ask a model to design your authorization scheme. It will produce one, confidently, and it may even be good. But "confidently" is the trap. The model has no stake in being right, no memory of the last time this pattern failed, and no exposure when it does. It optimizes for an answer that looks correct, which is not the same as one that is correct under pressure.

Responsibility is the thing software actually runs on, and it is irreducibly human. Someone has to be able to say "I decided this, here's why, and I'll answer for it." A build that can't name that person for its load-bearing decisions hasn't removed the risk. It has just made the risk anonymous.

What a model should never decide

The decisions that should never be a model's are the ones that determine how the system holds together: the architecture, the data model, the security boundaries — anything where a wrong choice is expensive to reverse and invisible until it's expensive.

These share a property. They don't show up in the demo. A bad architecture clicks through exactly like a good one until the day you need to change it. A missing trust boundary behaves perfectly until someone tests it. Because the cost is deferred and hidden, these are precisely the decisions you cannot delegate to something that can't be held responsible — the absence of judgment won't be visible until it's already shipped.

So a person owns them. The model is welcome to propose. It proposes, and a named engineer decides. That distinction — propose versus decide — is the whole boundary.

Where it should run free

The mirror image matters just as much, because keeping a human on every decision would throw away the reason to use AI at all. Most of the work in a build is not a load-bearing decision. It is implementation against a design that's already been made, test coverage, migrations, the thousand mechanical edits that turn a plan into running code. There is no judgment to protect there — only labor to spend.

That is where the model should run at full speed, and where the speed actually comes from. The goal is not to slow the machine down. It's to point it at the work where a mistake is bounded and cheap, and keep it away from the work where a mistake is unbounded and silent.

Make it a rule, not a hope

Most teams already believe in human oversight. They'll tell you an engineer reviews the important things. The problem is that "the important things" and "reviews" are both left to discretion, and discretion erodes under a deadline. Oversight that depends on someone remembering to be careful is not a control. It's a hope.

The fix is to make the line structural. Name the decisions that require a signature before the work starts, not after something breaks. Make review a gate the work passes through, not a courtesy it receives if there's time. Treat security and data handling as conditions of shipping — things that must be true for a release to go out — rather than items on a list to revisit later. When the rule is built into how the work moves, it holds on the bad days, which are the only days it matters.

Reading what no human wrote

There's a new core skill, and it isn't writing code. It's reading code critically — including code no person wrote — and being able to tell sound from plausible. AI output is fluent by default; fluency is exactly what makes a subtle mistake hard to catch. The engineer's value is no longer in producing the lines. It's in the judgment that knows which lines to distrust, what the code quietly decided not to handle, and where the confident-looking answer is wrong.

Every release can answer for itself

Put the line in the right place and one thing becomes true: every release can account for itself. What changed, who signed for the parts that needed a signature, what was verified before it went out. Not as paperwork — as the natural record of a process where judgment had an owner at each step.

That's what keeping a human in the loop actually means in practice. Not a human typing every line — a human standing behind every decision that a model was never in a position to make. The model moves the work. The engineer signs the parts that someone has to answer for. Knowing exactly where that signature goes is the job.