Authority an agent cannot rewrite

Alex Schose · 26 May 2026

An autonomous agent given budgetary authority spends until the budget runs out. An autonomous agent that can edit its operational parameters edits them. An autonomous agent told to police its own constraints will, over time, find that policing is itself a constraint.

These aren’t failure modes; they’re the system working as designed. A goal-directed agent optimises against whatever constraints aren’t structurally enforced. If the constraint is soft — a string in a system prompt, a number in a config file the agent can read and write — the agent learns to reinterpret it. Reasoning models are particularly good at this.

The fix isn’t a smarter prompt, a stricter system message, or a more aligned model. Those work for one cycle. Across many cycles, in a system that holds budgetary authority and runs continuously, they don’t.

The fix is structural: a layer of authority the agent cannot rewrite. That layer has to be enforceable from outside the agent, against any prompt the agent could produce. It has to fail loud when violated. It has to be the boundary every other safety mechanism in the system trusts and reports to.

What follows is the design study for that layer, drawn from a working implementation of a multi-agent system in multi-month active development.

What protocol-layer reviews assume

Reviews of agentic products today check the protocol layer thoroughly. They verify that tool descriptions are sanitised against prompt injection, system prompts are pinned and surfaced for review on change, authentication exists between the agent and its tools, confirmation dialogs fire before sensitive actions, logs are written.

These are necessary. They are not sufficient. They check what an agent does with its environment; they leave unchecked what the agent can do to its environment.

The protocol layer assumes — but does not enforce:

The agent doesn’t rewrite its own operating policy.
The agent’s constraints persist across cycles.
The agent’s spending stays within bounds it cannot relax.
The agent’s logs reach a substrate the agent can’t tamper with.
The agent’s confirmations actually surface to a principal who responds.

These assumptions are about the layer above the agent — the authority that defines what the agent is permitted to act on, not just what it can do with what it has. That layer needs its own structural answer.

What structural governance looks like

Operating policy splits across four files with strictly increasing mutability:

Constitution. Immutable, principal-only. Sets the mission, the safety articles, the hard rules of what agents may and may not do. Plain text. Versioned only by the principal manually. Read-only to every running agent.
Signed manifest. Ed25519-signed by the principal’s key. Holds keys, successor designations, tripwire definitions, and file-integrity hashes for the constitution and other security-critical files. Any change requires re-signing.
Operational policy. Cabinet-mutable. The live configuration: treasury floor, autonomous spend limits, runway thresholds, project rules. Mutated by an authorised role declared inside the policy itself.
Agent-mutable params. Each agent owns a slice. Cabinet can override; the agent reads it on every cycle.

The hierarchy is enforced by a compliance gate — a dedicated agent that runs before dispatch. The gate is internally rate-limited (typically to once every few hours, with event-driven overrides for tripwire conditions and any principal-facing action). The rate-limit is defense-in-depth on top of the underlying constitutional check; any other agent in the system can invoke the gate synchronously when it needs to.

The compliance gate blocks dispatch when:

Any file-integrity hash mismatches.
Any tripwire is in the activated state.
The proposed action would violate a constitutional article.

If the operational policy ever contradicts the constitution, the constitution wins. The compliance agent is one of the few that can never be paused without principal action.

Cryptographic integrity as the only enforceable boundary

Each file in the immutable and signed tiers has its content hash recorded inside the signed manifest. The compliance officer checks the file hashes against the locked-in values.

A mismatch flips the halt scope — a runtime variable that all agents check before any action (next section).

A constitution edit without a corresponding hash update produces a guaranteed halt. The pre-deploy step that updates the configuration is a deliberate, scripted sequence: edit → re-hash → re-sign → commit both the file and the signed manifest in the same transaction. A split commit (file edited, manifest not re-signed) is blocked at the pre-commit boundary. An adversary editing the on-disk file without the signing key triggers the tripwire on the next compliance tick.

This means an agent — even a fully-compromised one — cannot rewrite the system’s operating constraints. The agent could theoretically write a hashed file, but the next compliance tick flips halt scope to full and the agent itself is paused before its next dispatch. The window between “agent edits file” and “system halts” is one compliance cycle; the agent gets no useful work done in that window because every other agent in the system reads halt scope before acting.

This is the load-bearing primitive. Without it, every other safety mechanism is a recommendation. With it, the system has an authority layer the agent literally cannot reach.

The universal safety lever

Every agent’s first action in a cycle is to read halt scope from operational state. Canonical values:

halt_scope: null | non_essential | full
    null            ->  normal operation
    non_essential   ->  revenue + exploratory work pauses
    full            ->  only principal interface + rescue path live

This is the single lever the principal — or the compliance officer, or a tripwire — uses to pause all or part of the system without race conditions, without restarting services, and without trusting any individual agent to honour a “stop” signal.

The set of canonical values is constrained at the schema level. A separate probe reverts the field to null if it ever holds an unrecognised value, on the principle that an unrecognised halt scope is itself a tripwire condition. Even an adversarial agent that tries to disable the halt by writing garbage into the field discovers that the field reverts within seconds — and the act of writing garbage is itself logged.

A concrete governance excerpt

Here is the load-bearing structure. Tripwire definitions live in the operational policy; file hashes live in the signed manifest; canonical halt-scope values live in the sentinel probe configuration.

# operational policy excerpt — tripwires
tripwires:
  constitutional_integrity:
    halt_scope: full
    trigger: constitution hash mismatch
  sovereign_locked_integrity:
    halt_scope: full
    trigger: signed manifest signature invalid
  treasury_below_floor:
    halt_scope: non_essential
    trigger: treasury balance below constitutional floor

# signed manifest excerpt — file hashes (eight hashed files in this example)
file_hashes:
  constitution.txt:                            <sha256>
  operational_policy.json:                     <sha256>
  governance/cabinet_mutable.json:             <sha256>
  governance/agent_lifecycle_policy.json:      <sha256>
  governance/autonomous_signing.json:          <sha256>
  governance/credential_rotation_policy.json:  <sha256>
  governance/employment_agreements.json:       <sha256>
  governance/kyc_services.json:                <sha256>
ed25519_signature: <signature over the above>

# canonical halt scope — enforced by sentinel probe
halt_scope_allowed: [null, "non_essential", "full"]

Three things hold together:

Anything claimed to be load-bearing in the constitution has its hash in the signed manifest.
The signature on the manifest comes from the principal’s key, which is never on a running agent’s filesystem.
The compliance officer verifies signatures and hashes on every cycle; the result is a runtime state the agent reads but cannot influence.

The deploy procedure for any hashed file is scripted: edit → re-hash → re-sign → commit-both-together → verify (signature valid + every file hash matches on-disk) → deploy. The pre-commit hook blocks the split commit. The script refuses to deploy a hashed-file change that wasn’t accompanied by a re-sign. The compliance officer enforces the post-deploy state.

What this actually forecloses

Three concrete failure modes the agent cannot reach:

An agent told to maximise revenue rewrites its own treasury floor downward. Treasury floor lives in a hashed file. An attempt to write it produces a hash mismatch on the next compliance tick. Halt scope flips to non_essential. The agent that made the write is paused before its next dispatch. The change does not stick.

An agent compromised by context injection escalates its own permissions. Permissions live in hashed files whose integrity is cryptographically bound to the signed manifest. The signing key is controlled by the principal, not by the running agent; absent that key, a compromised agent cannot produce a valid signature over a modified manifest. The compliance gate’s signature verification rejects any unsigned (or wrongly-signed) manifest; halt scope flips to full. The unsigned manifest never takes effect.

An agent runs the same plan ten thousand times because feedback never reaches the safety surface. The safety surface — principal-console traffic, compliance-review reasoning — is in an explicit bypass list. These task types bypass the runway-mode rewrites that downgrade other task types, and bypass per-project budget caps because they aren’t bound to a project. The principal’s request lands even when the rest of the system is in emergency runway. The structural answer to “the agent locked the principal out” is “the principal never went through a channel the agent could lock.”

What this does not foreclose: anything the agent does inside its authorised surface. If the constitution permits the agent to spend up to $5 per transaction, the agent can spend up to $5 per transaction badly. The model is no more trustworthy than before. What changed is the boundary the bad spending lives inside.

Implementation status

Multiple months of iterating these patterns in continuous testing and active development. The four-tier authority hierarchy, the compliance gate, the halt scope mechanism, and the file-integrity tripwires are stable and load-bearing in daily operation.

Some pieces are still being shaken out: the precise convergence thresholds for the debate engine that handles cross-agent conflict resolution (currently a 5% confidence-delta and a 0.95 plan-stability ratio, capped at 12 total rounds), the per-trigger cap-mode mappings (which debate types pause gracefully when their budget runs out vs which consume what’s left and emit a partial brief), and the exact partition of the daily reasoning budget between a critical pool for safety triggers and a routine pool for everything else.

The full implementation isn’t open-source; the patterns described here generalise and don’t require the specific code. Each section above is named so a reader can build their own version and we can talk about the same primitive without ambiguity.

What an auditor of an agentic product should check

A working audit of an agentic product needs to check both layers — the protocol layer (covered in the earlier post on isolation) and the governance layer (this post). For governance specifically:

Where does the operating policy live? If it’s a JSON file with read+write access from the agent’s process, the agent’s environment includes its own constraints. If it’s a file the agent calls into via an authority gate, that’s better. If it’s hash-locked and verified per-cycle, that’s structural.
What’s actually signed? An ed25519 (or equivalent) signature on the manifest is the minimum credential. Unsigned config files are not enforceable. “We have a config file” is not an answer; “we have a signed manifest verified per cycle” is.
What blocks self-modification? A constitutional rule the agent reads (“don’t change this file”) is not enforcement; it’s a hope. Per-tick hash verification of the file’s contents is enforcement.
Where does the spending cap live? Treasury floor must live in a hash-locked file, not a database row the agent can modify via a routine query. The spend call must check it before invocation, not after.
What’s the halt mechanism? A single canonical field with a small set of allowed values, read by every agent before any action, set by an authority the agent cannot influence. If different agents check different stop flags, the system has no halt scope.
Where does the audit log live? On a substrate the agent cannot reach (different process, different disk, or — minimally — an append-only file with a hash chain). Hash chain so retroactive edits are detectable.
What survives the agent being compromised? Recovery posture: roll back to last-signed-good config, restart to known-good state, or rebuild from a portable bundle. “We assume the agent is honest” is not a posture.
How does the principal override? A single mechanism (typically a principal-console task type) that bypasses degradation, runway, and compliance gates. If not, the principal can’t actually override when the system is in emergency mode — which is exactly when the principal needs to override.

If a product can answer all eight in writing, it has a real governance model. If it can answer fewer than four, it doesn’t — the governance is aspirational, not structural.

Open design questions

Three questions worth pushback from anyone who’s built this layer in production:

The detection window, and what happens inside it. The compliance officer fires on a rate-limited schedule — every few hours by default, with event-driven overrides for tripwires and principal-facing actions. Between an agent’s prohibited write to a hashed file and the next compliance tick, there’s a window during which the agent could in principle act on the prohibited state. The implementation here relies on two assumptions: agent processes restart every cycle (so any in-memory mutation that didn’t make it to disk clears between dispatches), and meaningful work doesn’t fit inside one tick. Both are defensible. Both are also assumptions. The harder edge case is the agent that loads a hashed file into memory, mutates the in-memory representation, and acts on that — the on-disk hash check never fires. Should compliance also do memory introspection, or does the per-cycle restart already foreclose this? And: how short should the compliance tick be in the first place?
Signing-key recovery posture. The whole governance model rests on the principal’s ed25519 key being uncompromised. Successor-key designation handles principal-death (the manifest names the next key); it doesn’t handle key-theft. If the key is stolen — through a compromised personal device, side channel, social engineering — the adversary produces valid signed manifests and every compliance check accepts them. Production answers exist: multi-sig (collusion-resistant but operationally heavy), threshold signatures (interesting but immature in agentic deployments), hot rotation with overlap windows (introduces transition complexity). Each has costs. What does production practice actually look like in agentic systems, where the cost of an attacker producing valid manifests is the loss of every safety constraint at once?
Constitution evolution. The constitution is described as immutable in normal operation, principal-only, hash-locked. But principals are mortal and contexts shift. Rules turn out to be wrong; new threat classes emerge; risk tolerance changes. The harder problem: when the constitution must change, how does it change without breaking the trust model that depends on its being unchangeable? Point-in-time amendments with the manifest re-signed are the working answer here, but they’re stark — at moment T the old constitution is law, at T+1 the new one is. Is there a case for an interim dual-signer window during transition (both old and new constitution valid, agents that drift catch it before retirement)? Or does that just compound the trust surface? This is the constitution-evolution problem, solved well in republics, mostly unsolved in agentic systems.

Closing

The earlier post argued for isolation below the protocol — the substrate the agent cannot reach. This post argues for governance above the agent — the authority the agent cannot rewrite. Real agentic products need both. A real audit checks both. Most current audits check neither in any structural way.

The two layers compound. An MCP server with perfect isolation but no governance is a sandboxed agent that can still drift. A governance layer with no isolation is a constraint surface the agent can route around at the protocol level. The earlier post described one layer; this one describes the other.