Your agent nailed fifty drafts in a row, so you stopped reading them. Somewhere around draft thirty the review became a skim, then a rubber stamp, then a Friday afternoon decision to flip the thing to auto. Nobody wrote that decision down. Nothing can take it back.
That is how almost every unattended agent got its autonomy. A vibe hardened into a setting.
Why does agent autonomy get granted by vibes?
Because nothing in the stack owns the decision. Eval tools score outputs. Dashboards chart the scores. But a score is a measurement, and somewhere a human still has to convert measurements into "this agent may now send email without me." That conversion has no primitive, so it happens informally, once, and sticks forever.
The failure runs in both directions. Flip to auto too early and the agent that aced the demo writes bad CRM rows at 2am, weeks after anyone last looked. Never flip at all and a human reviews forever, which means the agent never actually earned you anything.
There is a second wrong belief stacked on the first. Teams treat a passing LLM-judge score as the permission slip. But the judge reads the content it grades, and adversarial content can grade itself. The research consensus is blunt on this. No current judge reliably resists prompt injection, per Lakera's work and the JudgeDeceiver paper. A permission slip the applicant can forge is decoration.
What does earned autonomy look like?
Treat the agent the way you treat a new hire. Nobody grants a new hire unattended production access on day one. They earn responsibilities one clean stretch at a time, and a bad incident takes responsibilities back.
The mechanic needs three parts.
A floor the judge cannot override. Deterministic rules run first. Unresolved merge tokens, missing opt-out line, dead links, whatever must never ship. One rule violation blocks the action no matter what any score says. Compliance checks live here, never in the rubric, because rules cannot be sweet-talked.
An advisory rubric on top. The LLM judge scores the quality dimensions, personalization, voice, grounding. A fooled score up here costs you polish. It cannot cost you compliance, because the floor already held.
A streak that earns and revokes. Every check writes a row. N consecutive passes under the same config earns clearedForAutonomy. One block resets the streak to zero. Changing the rules resets it too, because yesterday's clearance was earned under yesterday's rules.
I have run client delivery on this split for about two months. Tasks classified safe execute unattended into real deliverables. Everything above the line waits for a human. The line has held, and the interesting part is what stopped being a debate. Nobody argues about whether the agent is trustworthy anymore. They look at the streak.
What mistakes should you skip?
I packaged this mechanic as earn-autonomy, a free CLI. Before it shipped, it had to pass its own kind of gate, a stranger-walk against the real install. The gate blocked it twice. Four real failures, caught at full price, yours free.
A safety guarantee that was false under my own recommended pattern. The docs promised that changing a config's rules resets the streak. Then the walk set a config id, the exact pattern the docs recommended, built a cleared streak, tightened the rules, and stayed cleared. The identity check honored the label and ignored the content. Sixty-two tests were green the whole time, because every test checked the code and none of them checked the claim. Write the test that attacks the guarantee sentence in your README, then make it pass.
A default that waved everything through. The pack flag defaulted to content. Feed it outbound JSON without the flag and the tool scored the blob as prose, ran none of the outbound safety rules, and returned a clean PASS that counted toward autonomy. A default on a safety-relevant flag is a silent failure path. The fix made the flag required, and forgetting it now fails loudly.
Missing --pack. Available: content, outboundA quickstart that dead-ended every documented install. The README's copy-paste command referenced a path that exists under neither install method the README itself teaches. Docs are code your reviewers never execute. Run them, literally, as a stranger, from a clean machine.
A shipped doc that deleted the tool. The example README carried a repo-dev build command. Run it from the one place a non-cloning user can reach the files and the prebuild step removed the built CLI, leaving a dead binary and no warning. The fix rewrote the doc for the installed world and added a guard.
refusing to build inside node_modules (installed copy)The pattern under all four. The failures lived in the gap between the code and a stranger's ignorance, which is exactly the gap an autonomy gate exists to protect. Reviews read code. Tests check code. Somebody has to run the ignorance.
How do the numbers calibrate your decision?
Sparse, and only what helps you decide whether this is worth ten minutes. The install is 13KB and one command. The keyless path runs the deterministic rules only, costs nothing, and needs no API key. With a rubric configured you pay one model call per check on your own key, and the tool ships no spend cap, so set one in your provider console. Three stranger walks and two fix waves ran before anything shipped, and every finding above came out of them.
How do you install it?
npm install -g earn-autonomySave these two files and run the command. No API key needed.
{ "to": "sam@acme.com", "body": "Hi Sam, saw Acme shipped. Reply STOP to unsubscribe.", "fields": {} }{ "rules": [{ "name": "no-merge-token" }, { "name": "opt-out-present" }], "rubric": { "dimensions": [] }, "gateN": 3 }earn-autonomy check message.json --config config.json --pack outbound --jsonYou get a verdict, the streak, and clearedForAutonomy as machine-readable JSON. Wire your agent to it in one branch. Exit code 1 means do not act. Exit 0 while clearedForAutonomy is false means a human reviews, then acts. Exit 0 once it flips true means the agent has earned this action unattended, under these rules, until a block or a rule change takes it back.
Point it at whatever your agent does unattended today. The repo at github.com/derrtaderr/earn-autonomy ships a worked example inside the package, and the README states the one judgment that matters. Put the checks you cannot afford to lose in rules. Give the judge the rest.
FAQ
Does this replace my eval suite? No. Evals measure quality. This converts measurements into a decision about unattended execution. Run both. The gate is where a score stops being a number and starts being permission.
Why can't compliance checks live in the LLM rubric? Because the judge reads the content it grades, and injected content can inflate its own scores. No current judge reliably resists this. Deterministic rules cannot be argued with, so anything you cannot afford to lose goes there.
What actually earns autonomy? N consecutive passing checks under an unchanged config. One block resets to zero. Any change to rules, rubric, or threshold resets too, even under the same config id. The streak lives in a local runlog your output names on every run.
What does it cost to run? Rules-only checks are free and offline. A configured rubric costs one model call per check on your own Anthropic key. Nothing phones home. The only network calls are your rubric call and an optional link checker you enable knowingly.
My agent is Python. Does this work? Yes. The gate is a CLI with a JSON boundary. Anything that can shell out and read an exit code can use it, LangGraph, CrewAI, a cron job, a GitHub Action.
What would make me distrust the streak?
A changed working directory silently starts a fresh runlog, so pin --runlog to an absolute path when an agent or scheduler calls the gate. The README covers this.
Which of your agents runs unattended right now, and what did it do to earn that? Tell me in the comments. If the answer is "it looked good for a week," that is the Friday flip, and it is exactly the thing a streak replaces.