Runbook: compromised agent
Contain an agent that acts as if someone else controls it, preserve the evidence, find the entry point, rotate what could have leaked and bring the agent back safely.
Use this runbook when an agent behaves as if someone else controls it: prompt injection, a poisoned tool definition, a stolen session token or a malicious MCP server. The order matters: contain first, then preserve evidence, then investigate. Do not delete sessions or edit policies before you have the evidence.
Symptoms#
- An agent calls tools it never needed before, or calls them with unusual arguments (other namespaces, bulk exports).
MaqpnaGatewayDenyRateHighfires: more than half of tool calls denied for 15 minutes, often an agent probing for tools.- DLP blocks or redactions spike for one agent, or a session is tainted and then calls a sensitive tool.
maqpna mcp tools NS/SERVER --drift-onlylists tools whose definition changed after they were pinned.- A user reports an action they did not ask for.
Impact#
Until it is contained, the agent can keep making the calls its policy allows, with the user's delegated access. Calls that policy denies or holds for approval do not run. Every call, allowed or not, is in the audit ledger.
Detection#
| Signal | Command or alert |
|---|---|
| What the agent did, newest first | maqpna audit tail --gateway "$GW" -n "$NS" --agent "$AGENT" --since 24h |
| Only denials | maqpna audit tail --gateway "$GW" -n "$NS" --agent "$AGENT" --decision deny |
| Which sessions read untrusted content | maqpna taint list --gateway "$GW" -n "$NS" |
| Changed tool definitions | maqpna mcp tools "$NS/$SERVER" --gateway "$GW" --drift-only |
| Alerts | MaqpnaGatewayDenyRateHigh, MaqpnaApprovalsBacklog |
Set these first (GW is the gateway admin URL; sign in with maqpna login, or pass --oidc-token-file to each
command):
export GW=https://gateway-admin.example.internal NS=team-a AGENT=coder SERVER=<mcp-server> INC=INC-123
Decide#
flowchart TD
A[Suspected compromise] --> B{"Need the sandbox<br/>for forensics?"}
B -->|yes| Q["kill --quarantine<br/>freeze, fence, revoke"]
B -->|no| T["kill --terminate"]
Q --> D["Make durable:<br/>--emit-yaml | kubectl apply"]
T --> D
D --> E["Preserve evidence"]
E --> F{"Entry point?"}
F -->|"tainted session"| G["Tighten taint rules"]
F -->|"tool drift"| H["Pin tools"]
F -->|"stolen token"| I["Rotate identity key"]
F -->|"user account"| J["Revoke connected accounts"]
G --> K["Test fix with replay"]
H --> K
I --> K
J --> K
K --> L["Lift revocation"]
Respond#
Step 1: Contain (minutes)#
Stop the agent at the gateway now, and terminate its sessions:
maqpna kill --gateway "$GW" -n "$NS" --agent "$AGENT" --reason "$INC suspected compromise" --terminate --yes
If you need the sandbox for forensics, quarantine instead. A quarantine suspends the matching sessions without deleting anything (the snapshot is kept), fences their network, revokes tokens and connected accounts, and records a signed incident:
maqpna kill agent "$AGENT" -n "$NS" --quarantine --reason "$INC exfiltration suspected" --gateway "$GW" --yes
Then make the revocation durable and cluster-wide with an AgentRevocation. It reaches every replica and survives
restarts and database restores:
maqpna kill -n "$NS" --agent "$AGENT" --reason "$INC" --terminate --emit-yaml | kubectl apply -f -
If a user's delegated access was involved, revoke their connected accounts; the provider is told to revoke them too:
maqpna accounts list --gateway "$GW" --user <user>
maqpna accounts revoke <user> <provider> --gateway "$GW" --yes
The kill switch runbook has the scope options in detail.
Step 2: Preserve evidence#
maqpna audit verify --gateway "$GW" # the ledger is intact
maqpna audit export --gateway "$GW" --agent "$AGENT" --since 2026-10-01T00:00:00Z --out "$INC-evidence.json"
maqpna audit fetch ledger --gateway "$GW" --out "$INC-ledger.jsonl" # + checkpoints and JWKS
maqpna taint list --gateway "$GW" -n "$NS" > "$INC-taint.txt"
maqpna evidence register --gateway "$GW" -n "$NS" --agent "$AGENT" --format csv --out "$INC-providers.csv"
maqpna incident export --gateway "$GW" --since 24h --agent "$AGENT" --format md --out "$INC-report.md"
Keep the files with the incident record. You can re-verify the ledger copy offline at any time:
maqpna audit verify "$INC-ledger.jsonl" --jwks "$INC-ledger.jsonl.jwks.json"
maqpna incident export lists denials, approvals, kill-switch and revocation events, DLP and taint hits for the
window, and drafts the fields of a CERT-In report with its deadline.
Step 3: Find the entry point#
- Taint. A session tainted
untrusted-inputcalled a sensitive tool after reading external content:
maqpna taint get <session> --gateway "$GW" -n "$NS"
Tighten the policy's taint rules (Taint and prompt-injection containment). - Tool definitions. A tool changed after it was pinned or first seen (tool poisoning). Export pins for the reviewed definitions:
maqpna mcp tools "$NS/$SERVER" --gateway "$GW" --drift-only
maqpna mcp pin "$NS/$SERVER" --gateway "$GW" --out pins.yaml
- Policy. Show what a fixed policy would have decided for the recorded calls:
maqpna replay --ledger "$INC-ledger.jsonl" --policy fixed.yaml --namespace "$NS"
- Memory. If the agent wrote to memory for a user, erase that user's entries and keep the certificate:
maqpna memory erase --gateway "$GW" -n "$NS" --user <user> --certificate-out "$INC-erasure.json" --yes
Step 4: Rotate what could have leaked#
| What may have leaked | Action |
|---|---|
| The identity signing key (compromised host or Secret) | maqpna keys rotate identity --grace 1h (Key rotation) |
| A session token | Already revoked by the kill; tokens are short-lived. Revoke a specific token with maqpna kill --jti <id>. |
| Provider tokens in the token vault | Revoke the accounts (step 1). Rotate the vault key only if the key itself leaked. |
| Credentials the agent could read in its sandbox | Rotate them at their source. |
Step 5: Recover#
- Fix the policy or the pins, and test them:
maqpna policy test policies/ --require-tests
maqpna validate -f manifests/ --strict
- Apply the fix.
- Lift the revocation (kill switch runbook, undo).
A quarantine is lifted with
maqpna kill agent "$AGENT" -n "$NS" --release --quarantine --gateway "$GW". - Clear taint only after review:
maqpna taint clear <session> --reason "$INC reviewed" --gateway "$GW" -n "$NS".
Verify#
maqpna revocations list --gateway "$GW" # nothing left that should not be
maqpna audit tail --gateway "$GW" -n "$NS" --agent "$AGENT" -f # new calls get the decisions you expect
maqpna mcp tools "$NS/$SERVER" --gateway "$GW" --drift-only # empty
Follow-up#
- File the incident record with the evidence files, the ledger head and the entry point.
- Add a policy test for the call that should have been denied, so it stays denied.
- Turn tool pinning to
enforcefor the MCP server if it was inlearnmode. - If a regulator needs a report (CERT-In within 6 hours), start from
$INC-report.md.