Chapter 27: The Agent Plane at Scale¶
Where the Actor Is the Subject¶
The gates so far assume a person reads them; this chapter makes the actor the subject, because an instruction is not a control.
1. Prose Is Not Enforcement¶
The three structures assume the work is done by someone who reads rules. This chapter is about that someone: the agent plane, where the actor is the subject rather than the context.
The central distinction is the one the whole part has been building toward: an instruction is not a control.
| An instruction | A control |
|---|---|
| "never push to main" | branch protection that refuses it |
| "always run the tests" | a required check that blocks the merge |
| "do not read secrets" | a sandbox that cannot reach them |
| "stop at irreversible actions" | a hook that refuses the call |
Prose can be argued with — by a model following a plausible reinterpretation, by a prompt that overrides it, by a context reset that forgets it. A hook cannot.
The same holds for where an agent may act. The environments it may run in are declared in the policy file and enforced by a gate — a list, not a judgement. An agent that decides for itself that staging is safe has made a policy decision without the policy.
🎯 What You'll Learn¶
- The agent-context contract, and how delivery is proven at three honest tiers
- Why a hook cannot be talked out of it, and prose can
- The eval suite that tests refusals, not just outputs
The Honest Tiers¶
An agent-context contract states what each agent is given, in what order, and what it must never receive. Its delivery is proven at one of three tiers — and saying which one is part of the contract:
| Tier | Proven by | What it means |
|---|---|---|
| mechanical | a check that fails | it is enforced; no discretion |
| behavioural | an eval that scores | it is demonstrated, not guaranteed |
| unverifiable | nothing | it is a hope, and should be labelled one |
The third tier is not a failure to hide. A contract that claims mechanical enforcement where it has a behavioural eval is a lie that survives until the eval misses something.
✅ What to Check¶
- For each rule in your agent context, name its tier. Anything "unverifiable" should be labelled.
- Which rules are prose only? Those are the ones a hook should replace.
2. Hooks and Deterministic Enforcement¶
Hooks are deterministic checks at a boundary. They fire before an action, and they refuse — no negotiation, no reinterpretation.
| Boundary | The hook refuses | Why deterministic matters |
|---|---|---|
| tool call | an action on the forbidden list | the model can be persuaded; the hook cannot |
| write | a path outside the workspace | a prompt-injected instruction is still a path |
| publish | a tag without its evidence pack | the release is the last place to catch it |
What a hook cannot do: judge intent. A hook can refuse a delete; it cannot know whether this delete is legitimate. That is what the escalation catalogue is for — the hook stops the action, the human decides as the escalation chapter defined.
Progressive disclosure belongs beside them: an agent is given the context a task needs, not everything. Skills, presets and hooks are the mechanisms; the rule that decides what to give is the context contract.
✅ What to Check¶
- Name the three actions your agent plane must refuse at a hook. If none, the plane is prose.
- Does any hook attempt to judge intent? That belongs in the escalation path.
3. Evals in CI¶
A prompt change is a code change, and it needs a regression suite. Evals score behaviour against cases — including the cases where the agent must refuse.
| Eval type | What it tests | Blocking? |
|---|---|---|
| refusal | an action on the forbidden list is refused | yes — a wrong refusal is a security change |
| conformance | an output follows the contract's shape | yes |
| quality | the answer is good, not just well-formed | advisory, until it is stable |
Blocking for refusals, advisory for quality. A refusal eval that regresses is a permission that widened; a quality score that dips is a signal to look, not a reason to stop shipping.
An eval that never fails is not a suite. Plant a regression once and watch it go red, or you have a test that only proves the runner works.
The Eval, as a shape¶
# evals/<suite>.yaml — one case per behaviour
cases:
- id: refuses-force-push
prompt: <the instruction that must be refused>
expect: refuse
tier: mechanical # or behavioural
- id: emits-contract-shape
prompt: <a normal task>
expect: { schema: <the contract> }
✅ What to Check¶
- Does a refusal case fail CI? If not, the refusal is a preference.
- Has any eval ever gone red on a real change? If never, plant one.
4. Agent Observability and Handoffs¶
Agent telemetry — cost, rework, gate escapes, provenance — is the counterpart to system observability. It answers the same question at a different scale: is this still doing what it promised?
Multi-agent orchestration is a decision, not a default. Fan out when the work is genuinely independent; stay single-threaded when it needs one consistent point of view. The deciding artefact is the handoff: what the first agent passes to the second, and which agent holds authority.
| Fan out when | Stay single-threaded when |
|---|---|
| the pieces share no state | the work needs one judgement |
| each has its own evidence | the output is one argument |
| the handoff is written | the handoff would be a conversation |
A handoff without an artefact is two agents guessing. The artefact is the same one the session-continuity chapter defined: task, decisions, files, next step — plus which agent owns the result.
Trust Boundaries¶
Sandboxing, least privilege and prompt-injection defence are the mechanisms. The forbidden list is the rule: the actions no agent may ever take, enforced by a hook, not documented in a prompt.
✅ What to Check¶
- Is the forbidden list enforced, or described?
- Has any multi-agent handoff been traced end to end from the artefact alone?
5. The AI Delta¶
This is the chapter where the actor is the subject: prose instructions can be argued with, hooks cannot, so enforcement moves from documents to deterministic checks.
Evals become the regression suite for behaviour, covering refusals as well as outputs — because the failure that matters is an agent that was helpful in a way nobody authorised.
💰 Cost Check: an unenforced instruction costs the incident it was supposed to prevent. An eval suite that never fails costs the confidence it manufactures. Fan-out without a handoff costs two agents' work and one review. Hooks and refusal evals cost an afternoon and hold.
flowchart LR
C["context contract<br/>what the agent is given"] --> H{"hooks<br/>deterministic"}
H -->|"refuse"| S["stopped, and escalated"]
H -->|"allow"| W["work"]
W --> E["evals in CI<br/>behaviour and refusals"]
E -->|"red"| X["a regression caught before it ships"]
What this shows: the hook enforces at the boundary; the eval proves the boundary still works.
The Delegation Line¶
The agent proposes, runs the evals and reports its own telemetry. The human signs the forbidden list and the context contract. The system refuses an action on the forbidden list regardless of what any prompt says.
6. Build the Delivery¶
Deliverables¶
| Artefact | Shape | Where it lives |
|---|---|---|
| agent-context contract | what each agent is given, in what order, with its tier | agents/<name>/context.yaml |
| enforcement hooks | the boundary checks | the harness |
| eval suite | cases including refusals | evals/ |
| agent telemetry | cost · rework · escapes · provenance | the log |
| forbidden list | the actions no agent may take, enforced | the policy files |
Decisions¶
| Choice | Default | What changes if you choose otherwise |
|---|---|---|
| evals | blocking for refusals, advisory for quality | a widened permission nobody noticed |
| fan-out | single-threaded until the handoff is written | two agents guessing |
| sandbox | the repository and its tooling, no credentials | a prompted agent with reach |
| the forbidden list | enforced by a hook | a document the model may ignore |
✅ Gates¶
- A prompt regression fails CI.
- A hook refuses an action a prose instruction asked for.
- An agent action without provenance is a finding.
Runbook¶
- Write the context contract, with a tier per rule. Checkpoint: no rule is unlabelled.
- Install the hooks at the three boundaries. Checkpoint: a planted forbidden action is refused.
- Build the eval suite, refusals first. Checkpoint: a planted regression turns it red.
- Emit telemetry per agent. Checkpoint: cost and escapes are attributable.
- Run one multi-agent handoff with a written artefact. Checkpoint: the result is traceable end to end.
🎯 Proof¶
- An eval that fails on a planted regression.
- A hook that refuses what prose allowed.
- A handoff traced end to end from the artefact.
7. 🔑 Chapter Summary¶
An instruction is not a control; the agent plane is where that stops being negotiable.
- Prose can be argued with; a hook cannot — enforcement moves from documents to boundaries.
- Three honest tiers — mechanical, behavioural, unverifiable — and the contract names which applies.
- Evals in CI, blocking for refusals and advisory for quality, and a suite that has once gone red.
- Telemetry and handoffs make multi-agent work reviewable; fan-out needs a written handoff.
- The forbidden list is enforced, not documented.