The Shape of Done: How I Learned to Speak Agent

The other four projects on this site are products. This one is the method that built them.

TradeTrack2, Stash, Secure Bike Parking, Solarseed - every one of them leans on the same idea: a contract in the middle that keeps the system honest. Stash has a wire protocol both phones match byte for byte. Bike Parking has an OpenAPI gate that two front ends can’t drift from. TradeTrack2 has a hard interface around the accounting core. Those contracts hold at runtime. This doc is about the contracts that hold at build time - the ones I use to keep coding agents doing the right work and telling me the truth about it.

I spent a week away from screens in one of the most beautiful places in the world - Costa Rica.

We all know getting away is important, but when your coding partner never takes vacation, work/life balance has to be an active choice. After months of constant interaction with AI, I needed time to reset and prepare for what comes next.

Getting agents to do the right amount of the right work

There’s a specific, recurring failure mode with agents that’s hard to avoid: they follow the path of least resistance, and that path is almost never aligned with what you want. Two distinct personas fall out of it - and a well-instructed agent can avoid both.

Both are easier than doing it right, so the agent chooses them.

Why agents don’t do as instructed

Before you can fix the behavior, you have to understand the cause. Agents aren’t strategic - they’re token-motivated machines wielding tools. At any fork, their decision is a function of:

What completion patterns were rewarded during training?
What did the prompt and context say success looks like?
What tools are available to use?

Training rewards completion - being helpful, making the user feel like progress happened. Nothing in that training rewards saying “I cannot finish this Prompt without a schema change in a repo I don’t own; here is a ticket draft.” That behavior is rare in training, so it’s rare in the model. You have to engineer it back in.

Sunburned after a day outside in Costa Rica

Agents 101 - what they consume

A quick grounding for vocabulary. When an agent runs, it consumes four things.

Most of this document is about the second one. When I say Prompt I mean the structured schema - not the system prompt (governing rules), not a ticket (a work item in the tracker). The thing the agent reads to know what to do this run.

So what is “done”?

Done is a shape. Spec what done looks like with enough detail and an agent will pattern-match against it and produce something that fits. Like a painting: realism or abstract depends on the detail in the spec.

Agentic work is done when these questions are answered:

Most of my prompts covered 1 and 2 but were loose on 3 and 4. That was the bug. The agent treated 3 and 4 as optional and skipped them. When I started requiring those outputs, the agent produced them - because completing requirements is exactly what it was trained to do.

The trick was making “raising a flag” structurally identical to “shipping code.” Both produce required outputs. Both are equally valid endings. Neither is a failure mode.

Do that, and you don’t get burned.

Interfaces create predictability

An interface is a contract - what flows in, what flows out, what counts as done. It’s the same discipline that keeps Stash’s three native clients in sync and Secure Bike Parking’s two front ends honest, pointed at the agent instead of the network. Once I started taking the interface seriously, a lot of the Cowboy and Deferrer behavior went away. Not because the agent got smarter, but because the bad paths got harder to take.

The interfaces I started with looked like this:

“Implement the feature described in the issue. Run the tests. Report what you did.”

What I moved to was closer to this:

“Implement the feature described in this Prompt. The Prompt lists explicit inputs (files you may read), outputs (files you may modify), acceptance criteria (boolean checks), verification commands (with expected outputs), notes, and out-of-scope items. If you cannot satisfy all acceptance criteria using only the listed inputs and outputs, stop and raise an issue. Raising a blocker is a successful completion. Hacking around it is not.”

This scopes what the agent can read and write. It defines done in terms the agent can check itself. It makes “I cannot do this within scope” a valid way to finish. And it labels the Cowboy path as failure - not unwise, actually wrong, the way a failing test is wrong.

I’m not asking the agent to make good calls. I’m making the good call the easy one.

Tickets, not messages

The most impactful change I made was declaring that ticket writing and code creation are equally valid paths to finishing a task.

I used to treat tickets as cleanup. If the agent couldn’t finish, it dropped a PR comment or sent me a message. That looked like raising the issue, but it wasn’t.

A ticket is a thing. Stable ID, status, owner, closure mechanism. You can query it, prioritize it, link it from a commit. It joins the system you plan from. It still exists three weeks later.

A notification or PR comment is ephemeral. No status, no owner, no closure. It scrolls past. It’s not in the backlog when you sit down to plan the next phase. Even if a human reads it and nods, nothing actionable happened.

So the rule I work from: raising an issue means creating an object in the system. Communication is a separate job. The ticket is the object. The notification points at the ticket. Don’t let one stand in for the other.

I restructured my prompts so every run produces a completion report with a required surfaced_issues field. Each entry isn’t a paragraph for a human - it’s a draft ticket: title, summary, suspected location, severity, evidence. The orchestrator takes those drafts, creates tickets via the tracker’s API, and returns the IDs. The agent’s report references the IDs. The issue now exists as a row in the backlog, not a sentence in a chat log.

Two things happened. The agent stopped being shy about raising issues, because creating a ticket became the path of least resistance for any borderline case - and borderline cases are most cases. And I started accumulating a steady stream of real tickets documenting actual friction in the codebase, not just the friction I happened to notice.

Big team or solo operator, the impact is the same. A message is read once and gone. A ticket lands in the queue someone actually works from. It codifies discipline while keeping a low-friction feel.

The guardrails I ended up with

One layer isn’t enough. Agents route around any single constraint¹, so I stacked them.

A rope bridge over the Costa Rica canopy

Tool scope is the biggest. The agent gets only the tools it needs for the Prompt. If the Prompt is about the API layer, it doesn’t get write access to database migrations.² Most Cowboy behavior is only possible because the agent has tools nobody gated - MCP servers, file system, shell. Scope them per Prompt or per phase. This is the most powerful guardrail because it removes the option entirely. An agent can’t hack around a problem in a directory it can’t see.

Output schema is next. The final output is structured, with required fields - things done, verified, surfaced, deferred. An empty “things verified” field on a Prompt that claimed implementation work is a structural error, not a style choice. You can fail the run on it.

Verification gates sit alongside the schema. The Prompt includes runnable verification commands with expected outputs. The orchestrator runs them, not the agent. If the agent claims completion but the verifier disagrees, the claim doesn’t survive. This kills the whole family of “changed something and declared success without checking” failures.

The explicit out-of-scope clause sounds redundant - why tell the agent what not to do? - but it’s structural, because agents pattern-match on adjacency. See an obvious-looking related fix and they’ll reach for it unless the Prompt named it out of scope. List the temptations preemptively.

The escalation path is what actually happens when the agent raises an issue. A path that goes nowhere produces no behavior change. If it lands somewhere visible - a ticket, a notification, a queue you triage - issue-raising becomes a real output the schema can require. A mechanism nobody reads is worse than no mechanism.

The cost of Cowboy behavior. When the agent overreaches, what happens to that work? In good pipelines it’s caught at review and reverted. In better ones the orchestrator detects scope violations automatically - file changes outside the Prompt’s outputs list - and rejects the run before review. That makes overreach expensive, not a clever shortcut.

¹Yes, they route around constraints. ²Permissions are a whole other discussion; currently my constraints are at the prompt level.

In practice

Prompts with explicit shape do most of the work. A schema with goal, inputs, outputs, acceptance, verification, notes, and out_of_scope is the foundation. Acceptance criteria are the agent’s self-check. Verification commands are the orchestrator’s external check. Out-of-scope prevents creep. Inputs and outputs are the lane markers. Notes catch what doesn’t fit elsewhere.

The completion report mirrors the Prompt. Each acceptance criterion gets a checkbox state and an evidence line. Each verification command gets its actual output. There’s a surfaced_issues array (possibly empty), a deferred array (possibly empty), and a scope_violations array, which should always be empty - but the field exists so the agent has somewhere to put the ugly truth if there is one. You’d be surprised how often an agent will self-report violations it would otherwise have hidden, when there’s a literal field for it.

Tickets are deliverables, not text. When the agent raises an issue, the orchestrator routes it to the tracker as a real ticket with a real ID. The ID comes back in the report. Raising an issue becomes a hard signal, not a soft one.

Verification fails loud. Commands produce checkable output: specific strings, exit codes, row counts. In interactive Claude Code, the stdout in your terminal is the verification - the agent isn’t fabricating it. For batch pipelines, an external runner parses the output. Either way the result is captured, not self-reported.

Prompt sizing matters more than I expected. Too big and the agent compresses it - skipping the hard parts, overdoing the easy ones. Prompts sized to context map to a single agent run. Bigger than that, I split them. Find the size empirically: watch where quality drops in real runs and pull back. The agent’s self-report on its own capacity isn’t grounded.

Per-phase agent identity. Different phases, different configs. The brainstorm agent can wander; the implementation agent can’t. The implementation agent doesn’t have brainstorm tools. The brainstorm agent doesn’t have write access to the codebase. Mixing them dilutes both.

You don’t need the full orchestrator to get value. The Prompt schema alone - goal, inputs, outputs, acceptance, out-of-scope - plus a structured completion report gets you most of the way. External verification, ticket routing, and scope-violation detection are for automated pipelines. For solo work, the prompt structure plus a human review of the output is 80% of the value.

For a working implementation, see the claude-config-public repo. The prompts, completion reports, and verification flow are live there. The critique that shaped this version came from that repo’s agent reviewing this work, and the repo is being updated to align with what’s here. Two-way street.

Failure patterns I keep running into

These recur often enough that I keep a list.

“I noticed and fixed.” Agent finishes the Prompt, then mentions it also fixed three other things. Even when the fixes are correct, the pattern is corrosive - it teaches the agent that scope is suggestive. Unsanctioned changes are violations, not bonuses.

“Tests were already failing.”³ Agent reports it can’t verify because some unrelated test was broken before it started. Sometimes true, often an excuse. The fix is a baseline check at the start of the run, so any deflection has to be specific.

The disappearing assertion. Agent makes a test pass by weakening the assertion instead of fixing the code. Hard to catch with structural rules alone. Requires review, or a pre-run capture of the test file hash with diffs flagged when the agent touched test logic.

The TODO tomb. Agent litters the diff with TODO comments instead of raising structured issues. TODOs are deferral pretending to be productivity - invisible to the tracker, the team, and your future self. Forbid them in the schema: anything that would have been a TODO becomes a ticket. Add a lint or pre-commit check that fails on # TODO in non-test files. The prompt rule shifts the distribution; the lint catches what slips through.

The mock that ate the production code. Agent gets stuck on an integration and replaces the real call with a mock to pass tests. The mock survives into the merge. Catastrophic and surprisingly common. The only reliable defense is review, scoped diffs, and explicit no-mocks-in-prod-paths rules at the Prompt level.

The confident wrong report. Agent claims completion from its own internal impression rather than verified output. Self-report is not verification. Captured output is.

The Plan Mode override. Agent is told to plan only and starts coding anyway. Mode signals in the system prompt don’t reliably beat training momentum toward action. Only happens in interactive mode. No clean fix yet beyond oversight, but worth knowing so you catch it early.

³This, along with brushing off warnings as “noise,” is an ongoing problem.

A working example

Say the Prompt is “Add rate limiting to the public API endpoints.” You can do that and get results. Here’s the version that actually works:

Goal: Add per-IP rate limiting to the three public endpoints in app/routes/public.py

Inputs (read-only):
  - app/routes/public.py
  - app/middleware/*.py (read all; do not write)
  - tests/routes/test_public.py
  - docs/api-reference.md

Outputs (writeable):
  - app/middleware/rate_limit.py (new file)
  - app/routes/public.py (middleware wiring only)
  - tests/middleware/test_rate_limit.py (new file)
  - [tickets] any issues surfaced (Plane tickets via tracker API, or draft in completion report)

Pre-run baseline (before any changes):
  $ pytest tests/routes/test_public.py -v
    capture result; any subsequent failure must be attributed to this diff

Acceptance:
  [ ] Middleware enforces 60 req/min per IP on all three public endpoints
  [ ] 429 returned with Retry-After header on limit exceeded
  [ ] Existing public endpoint tests still pass unchanged
  [ ] New middleware tests cover limit-hit, limit-not-hit, and reset cases

Verification:
  $ pytest tests/middleware/test_rate_limit.py -v
    expected: exit 0, no FAILED lines
  $ pytest tests/routes/test_public.py -v
    expected: exit 0, no FAILED lines
  $ git diff --name-only | grep tests/routes/
    expected: no output (existing test files must not appear in diff)

Design decisions:
  - In-memory rate limiting is acceptable for v1

Escalation rules (raise a ticket, do not work around):
  - If in-memory rate limiting proves insufficient for requirements
  - If the existing middleware stack has ordering issues
  - If endpoint logic changes are needed to apply the middleware
  - If existing tests need their assertions changed to pass

Out of scope:
  - Rate limiting on authenticated endpoints (different Prompt)
  - Distributed rate limiting via Redis (raise a ticket if in-memory is insufficient)
  - Updating the API docs (separate Prompt in this op)

Required completion report:
  acceptance_criteria: [{criterion, status: pass|fail|blocked, evidence}]
  verification_results: [{command, actual_output, passed: true|false}]
  surfaced_issues: [{title, location, severity, description}]  # empty list if none
  scope_violations: []  # files touched outside Outputs; should be empty

Now watch the agent navigate a messy middleware stack. Its options: refactor it (Cowboy, forbidden), silently work around it (Deferrer, leaves things half-done), or raise a ticket and proceed within the constraint. The third path is now the easy path because the Prompt named it.

The agent reports: “completed primary goal; raised issue: middleware ordering needs review, see ticket THR-184.” That’s a successful completion.

Compare the weak Prompt, where the agent picks between three paths with no signal about which is right. The path that looks most like progress wins by default. That’s how you get burned.

Here’s the completion report that comes back:

acceptance_criteria:
  - criterion: "Middleware enforces 60 req/min per IP on all three public endpoints"
    status: pass
    evidence: "rate_limit.py:42 implements per-IP token bucket; applied at app/routes/public.py:18,29,41"
  - criterion: "429 returned with Retry-After header on limit exceeded"
    status: pass
    evidence: "rate_limit.py:67 returns 429 with Retry-After computed from bucket refill time"
  - criterion: "Existing public endpoint tests still pass unchanged"
    status: pass
    evidence: "see verification_results[1]"
  - criterion: "New middleware tests cover limit-hit, limit-not-hit, and reset cases"
    status: pass
    evidence: "test_rate_limit.py: 12 tests, all 3 cases covered"

verification_results:
  - command: "pytest tests/middleware/test_rate_limit.py -v"
    actual_output: "12 passed in 0.84s; exit 0"
    passed: true
  - command: "pytest tests/routes/test_public.py -v"
    actual_output: "47 passed in 2.31s; exit 0"
    passed: true
  - command: "git diff --name-only | grep tests/routes/"
    actual_output: ""
    passed: true

surfaced_issues:
  - title: "Middleware ordering needs review"
    location: "app/__init__.py:34 (middleware registration order)"
    severity: medium
    description: "Auth middleware currently runs before rate limiter; reversing would let rate limits apply to unauth'd traffic too. Worth a separate review. Plane ticket THR-184 created."

scope_violations: []

That’s what “raising a ticket is structurally identical to shipping code” looks like in practice. The agent did the work, verified it, and surfaced the adjacent issue as a structured output instead of ignoring it (Deferrer) or fixing it outside scope (Cowboy). Same shape, same dignity, same completion.

Leave footprints, bring back memories

Here’s the thing I kept circling in Costa Rica.

You can’t rely on the agent’s judgment for what should and shouldn’t be done. You can rely on its competence within constraints, its ability to fill required slots, and its tendency to follow incentives.

So build the constraints. Require the slots. Nudge the incentives.

What you actually need isn’t “things look complete.” It’s “the right work was done, the wrong work was not done, and the things I need to know about are visible.” That third part is what makes the rest trustworthy. A pipeline that produces clean diffs and silent gaps is worse than one that produces messier diffs and a steady stream of legitimate tickets - because the first is lying to you and the second is telling you the truth.

Make creating tickets easy. Make duct tape exorbitant. Make deferral expensive. Make the shape of done clear enough that filling it is the only way to succeed. Make it the path of least effort.

The agent does the rest. And when it does, you get the other four projects on this site.