Governed cluster operations (AI SRE)

Give an SRE agent typed, read-only-by-default Kubernetes tools through the gateway's built-in maqpna-k8s server, with writes as GitOps proposals or human-approved changes, and every call audited.

flowchart LR
  A[SRE agent] -->|tools:maqpna-k8s| G[Gateway pipeline<br/>scope, revocation, DLP,<br/>namespace scope, ToolPolicy]
  G -->|read| K[maqpna-k8s<br/>typed tools]
  G -->|write| W{writes.mode}
  W -->|proposal| P[YAML patch<br/>audited, not applied]
  W -->|apply| H[Human approval<br/>always]
  H --> K
  K -->|Impersonate-User:<br/>maqpna-k8s-ops SA| API[Kubernetes API<br/>RBAC decides]
  K --> L[(Audit ledger<br/>resource refs, patch)]

Goal#

Let an AI agent diagnose a cluster, and propose or make small fixes, in a way an operator can trust:

• Read-only by default, enforced by RBAC. The gateway acts as a dedicated ServiceAccount whose ClusterRole only has get and list. A prompt cannot widen that.
• Narrow typed tools. There is no generic kubectl tool and no free-form flag or API path, so an argument can never become a command-line option (the class of CVE-2026-47250 in community Kubernetes MCP servers). Kinds come from a fixed table and names must be DNS names.
• Every write needs a human or goes through GitOps. Writes return the YAML patch for a pull request (proposal, the default), or wait for an approval and then apply (apply).
• Everything is audited with the resource it touched, and container output taints the session.

Prerequisites#

• MAQPNA installed with Helm in a cluster, and the admin role for approvals.
• metrics-server, if you want get_resource_usage.

Tools#

The built-in server is maqpna-k8s at POST /mcp/maqpna-k8s. Agents need the scope tools:maqpna-k8s (listing maqpna-k8s under Agent.spec.tools grants it); ToolPolicy rules use servers: [maqpna-k8s].

Kinds: pod, service, configmap, secret, persistentvolumeclaim, serviceaccount, event, node, namespace, persistentvolume, deployment, statefulset, daemonset, replicaset, job, cronjob, ingress, networkpolicy, endpointslice, horizontalpodautoscaler, poddisruptionbudget.

What the agent never sees:

• Secret values. Secrets are requested as metadata only (PartialObjectMetadata), and any data keys are shown as <redacted>. Reading Secrets at all needs secretsMetadata: true.
• ConfigMap values. Only keys and sizes are returned.
• Container env values. They are replaced by <redacted>; valueFrom references stay.

Steps#

1. Enable the server#

# values.yaml
gateway:
  k8sOps:
    enabled: true
    namespaces: [shop, payments]   # empty = every namespace
    writes:
      mode: proposal               # disabled | proposal | apply

helm upgrade maqpna oci://ghcr.io/maqpna/charts/maqpna -n maqpna-system -f values.yaml

The chart creates:

2. Create the agent and its policy#

kubectl apply -f config/samples/toolpolicy-sre.yaml -f config/samples/agent-sre.yaml

The sample policy allows the read tools (rate-limited), requires approval for every write, and denies writes in kube-* and prod-* namespaces.

3. Diagnose#

maqpna call --server maqpna-k8s --tool explain_failure \
  --arg kind=deployment --arg namespace=shop --arg name=checkout

deployment/shop/checkout: MinimumReplicasUnavailable (Deployment does not have minimum availability.)
- [error] deployment/shop/checkout NotReady: 1 of 3 replicas ready
- [error] pod/shop/checkout-7d9c container app OOMKilled: restarted 4 times; last exit code 137
- [warning] pod/shop/checkout-7d9c BackOff: Back-off restarting failed container

4. Propose a fix#

With writes.mode: proposal, a write returns the patch and changes nothing:

Proposed change (NOT applied; commit it to the GitOps repository that manages deployment/shop/checkout):

# maqpna-k8s scale_workload deployment/shop/checkout
apiVersion: "apps/v1"
kind: "Deployment"
metadata:
  name: "checkout"
  namespace: "shop"
spec:
  replicas: 5

The patch is a strategic-merge patch (kustomize patches:); deletions use $patch: delete. Commit it through your normal review. The ledger has an observe record k8s_write_proposed with the patch and its SHA-256.

5. Or apply with approval#

gateway:
  k8sOps:
    writes:
      mode: apply
      rbac: true          # bind the write ClusterRole
      approval: required  # the default

Every applied write is held as an approval (JSON-RPC -32002), even when a ToolPolicy allows it: approval: required is a floor the policy cannot lower. Approve it in the console or with maqpna approvals approve --gateway URL ID; the agent's retry applies the change. A ToolPolicy deny still wins. approval: policy lets the ToolPolicy decide alone.

Verify#

• maqpna audit tail --agent sre shows every call with ext.k8sResource (for example deployment/shop/checkout) and ext.k8sVerb; writes also carry ext.k8sMode, and an observe record k8s_write_proposed or k8s_write_applied with ext.k8sPatch and ext.k8sPatchSha256.
• kubectl auth can-i --as system:serviceaccount:maqpna-system:maqpna-k8s-ops patch deployments -n shop answers no until writes.rbac is set.
• After get_pod_logs or explain_failure, maqpna taint list shows the session tainted untrusted-input, so a policy can gate exfiltration-capable tools.

Troubleshooting#

Next steps#

• Stop a misbehaving SRE agent at once: Kill switch and revocations.
• Approval routing and quorum: Human approvals.
• Contain what logs can inject: Taint and prompt-injection containment.