Run the stack on Kubernetes

Install MAQPNA with Helm, run llamay as its in-cluster inference backend, point agents at it with provider llamay, and lock it down with network policy, secrets and sizing, with notes on what kosa8 does and does not do in a cluster.

This page takes the stack from your laptop to a Kubernetes cluster. The MAQPNA Helm chart can run llamay for you as the gateway's private inference backend. Agents select it with model.provider: llamay, and each session gets its own llamay inference context, as on the laptop.

kosa8 is different: it is a laptop and edge sandbox, not a Kubernetes runtime. In a cluster, agents run in MAQPNA's trust tiers instead. The table below says exactly what moves to the cluster.

flowchart LR
    subgraph AG["Agent namespace (team-a)"]
        S["Session sandbox<br/>tier-0 / tier-1 / tier-2"]
    end
    subgraph SYS["maqpna-system"]
        GW["maqpna-gateway"]
        IB["maqpna-identity"]
        OP["maqpna-operator"]
        LL["maqpna-llamay<br/>:8081"]
        K[("Secret<br/>maqpna-llamay-key")]
        A[("PVC maqpna-llamay-audit")]
    end
    S -->|"/llm/team-a.coder<br/>session token"| GW
    GW -->|"Bearer key, per-session context"| LL
    K -.->|"mounted read-only"| GW
    K -.->|"LLAMAY_API_KEY"| LL
    LL -->|"-audit -audit-strict"| A
    A -.->|"read-only, verified"| GW
    S --x|"blocked by NetworkPolicy"| LL

Before you begin#

• A Kubernetes cluster that meets MAQPNA's requirements, with a CNI that enforces NetworkPolicy (Calico, Cilium and others). See Install on Kubernetes with Helm and run maqpna preflight first.
• The maqpna CLI, Helm 3 and kubectl, with access to the cluster.
• The MAQPNA chart and images. Set REGISTRY to the release registry listed on maqpna.com/download.
• A llamay image your nodes can pull. The chart's default is llamay's release image at tag 0.4.5 (<tag>-cuda for GPU nodes). If your nodes cannot pull it, mirror it and set llamay.image.registry and llamay.image.repository. See llamay.com/download.
• A model: a llamay store name or a GGUF file. Either the pod downloads it at start (llamay.models.pull), or you ship it in an image (llamay.models.bundle.image, for air-gapped clusters).
• For audit anchoring: a StorageClass that supports ReadWriteMany. See the platform tabs in Step 1.
• For GPU inference: GPU nodes with the NVIDIA device plugin, so pods can request nvidia.com/gpu.

Step 1: Write the values#

Save this as values-stack.yaml. It turns on llamay, pulls a model at start, anchors llamay's audit and lets agents run in team-a.

llamay:
  enabled: true
  profile: cpu                     # cuda for GPU nodes
  model: /models/qwen2.5-coder-7b-instruct-q4_k_m.gguf
  models:
    pull: hf:Qwen/Qwen2.5-Coder-7B-Instruct-GGUF/qwen2.5-coder-7b-instruct-q4_k_m.gguf
    persistence:
      size: 20Gi
  concurrency: 2                   # generations at once (llamay -concurrency)
  resources:
    requests: {cpu: "4", memory: 8Gi}
    limits: {memory: 12Gi}
  audit:
    persistence:
      accessModes: [ReadWriteMany] # required for anchor.enabled
      storageClass: ""             # an RWX class, see the tabs below
    anchor:
      enabled: true
      intervalSeconds: 60
      failClosed: true             # take llamay out of routing while its chain does not verify

networkPolicy:
  agentNamespaces:
    names: [maqpna-agents, team-a]

The values you are most likely to change:

All values are in the Helm values reference. The values that differ by platform are the RWX storage class and the GPU nodes:

llamay:
  audit:
    persistence:
      accessModes: [ReadWriteMany]
      storageClass: efs-sc            # a StorageClass you created for the Amazon EFS CSI driver
  # GPU: a node group with the EKS-optimised accelerated AMI and the NVIDIA device plugin
  # nodeSelector: {node.kubernetes.io/instance-type: g5.xlarge}

llamay:
  audit:
    persistence:
      accessModes: [ReadWriteMany]
      storageClass: azurefile-csi     # built-in Azure Files class
  # GPU: a GPU node pool with the NVIDIA device plugin
  # tolerations: [{key: sku, operator: Equal, value: gpu, effect: NoSchedule}]

llamay:
  audit:
    persistence:
      accessModes: [ReadWriteMany]
      storageClass: standard-rwx      # Filestore CSI driver
  # GPU: a GPU node pool; GKE installs the driver and taints the nodes nvidia.com/gpu

If you cannot get ReadWriteMany storage, set llamay.audit.anchor.enabled: false and keep accessModes: [ReadWriteOnce]. llamay still writes its audit and stops serving if a record cannot be written. You then verify the chain yourself with maqpna llamay verify-audit, and the ledger has no llamay/audit-anchor records.

Step 2: Validate the values#

maqpna values validate --profile production -f values-stack.yaml

The command checks the values offline against the chart's schema and production rules, and exits with code 3 when something is wrong. The chart itself refuses these combinations when it renders:

Step 3: Install or upgrade#

maqpna preflight --profile prod -f values-stack.yaml
maqpna install --profile prod -f values-stack.yaml \
  --chart "oci://$REGISTRY/charts/maqpna" --version 2.1.0 \
  --set image.registry="$REGISTRY" --wait --timeout 15m

helm pull "oci://$REGISTRY/charts/maqpna" --version 2.1.0 --untar
helm upgrade --install maqpna ./maqpna \
  --namespace maqpna-system --create-namespace \
  --set image.registry="$REGISTRY" \
  -f values-stack.yaml --wait --timeout 15m

With release name maqpna in maqpna-system, the chart creates:

Check it worked.

kubectl -n maqpna-system rollout status deploy/maqpna-llamay --timeout 15m
kubectl -n maqpna-system logs deploy/maqpna-llamay -c pull-model     # only with models.pull
maqpna status -n maqpna-system
maqpna doctor -n maqpna-system

The first start downloads the model, so it can take several minutes. The startup probe allows llamay up to 10 minutes to load a model before Kubernetes restarts it.

Step 4: Point an agent at llamay#

Create the agent and its policy in team-a:

# agent.yaml
apiVersion: maqpna.com/v1alpha1
kind: Agent
metadata:
  name: coder
  namespace: team-a
spec:
  image: registry.example.eu/agents/coder@sha256:<digest>
  tier: tier-1                       # Kata + Firecracker microVM; tier-0 is gVisor
  model:
    provider: llamay
    endpoint: http://maqpna-llamay.maqpna-system.svc:8081   # "/v1" is appended
    name: qwen2.5-coder-7b-instruct-q4_k_m
    jurisdiction: EU-DE
    tokensPerMinute: 60000
    maxTokensPerRequest: 4096
  tools:
  - name: echo                       # the chart's example MCP server
  maxSessionTTL: 4h
---
apiVersion: maqpna.com/v1alpha1
kind: ToolPolicy
metadata:
  name: coder-policy
  namespace: team-a
spec:
  agents: ["coder"]
  defaultAction: deny
  rules:
  - name: echo-allowed
    servers: [echo]
    tools: ["echo"]
    action: allow

maqpna validate -f agent.yaml --strict
maqpna apply -f agent.yaml -n team-a
maqpna rollout status ag/coder -n team-a

What provider: llamay does:

• The operator writes the route team-a.coder with provider: llamay. The sandbox gets MAQPNA_MODEL_ENDPOINT={gateway}/llm/team-a.coder/v1. It never learns llamay's address.
• The endpoint is on the origin of the chart's llamay, so the gateway presents the chart's key. The agent needs no apiKeySecretRef.
• The gateway gives each session its own llamay context and forks it when the session forks (maqpna session fork). It strips any llamay_session the agent sends.

The trust tier must exist and its runtime must be installed on the nodes: maqpna tier list. The chart's sample tiers are tier-0 (gVisor), tier-1 (Kata with Firecracker) and tier-2 (confidential VM).

A llamay outside the chart#

To use a llamay you run yourself, for example on a GPU server outside the cluster, leave llamay.enabled off and give the agent the key:

kubectl -n team-a create secret generic llamay-gpu --from-literal=api-key='<your-llamay-api-key>'

spec:
  model:
    provider: llamay
    endpoint: https://llamay-gpu.internal.example.eu:11435
    name: qwen2.5-coder-32b-instruct-q4_k_m
    jurisdiction: EU-DE
    apiKeySecretRef: {name: llamay-gpu, key: api-key}

The operator copies the key into a gateway-only Secret. It never reaches the sandbox. Per-session contexts and context fork work the same. Audit anchoring does not: the gateway can anchor only an audit file it can read, and the chart mounts only its own llamay's audit volume. If you restrict the gateway's egress (networkPolicy.fqdnEgress, or the sovereignty policy's allowed hosts), add this host there.

Step 5: Verify#

1. Admission. Create a session and wait for it to run:

   maqpna smoke --sandbox team-a/coder
   maqpna session start --agent coder -n team-a --task "Say hello" --wait
   

2. The model call is governed. Follow the ledger while the agent works. You need an admin or auditor login (maqpna login):

   kubectl -n maqpna-system port-forward svc/maqpna-gateway 8080:8080 &
   GW=http://127.0.0.1:8080
   maqpna audit tail --gateway "$GW" -n team-a --agent coder -f
   

   Each model call is a record on route team-a.coder. Export one session and look for the llamay context:

   maqpna audit export --gateway "$GW" --session <session-name> --namespace team-a --out session.jsonl
   grep -c llamayContext session.jsonl
   

3. The audit chain is anchored. With anchoring on, the gateway exports these metrics:

   curl -s "$GW/metrics" | grep -E 'maqpna_gateway_llamay_(audit_verified|audit_head_seq|contexts)'
   

   maqpna_gateway_llamay_audit_verified{anchor="in-cluster"} is 1 when the chain verifies. If you set gateway.config.adminListen, scrape /metrics on the admin port instead.

4. Only the gateway can reach llamay. A pod that is not the gateway times out:

   kubectl -n maqpna-system run np-check --rm -it --restart=Never --image=curlimages/curl -- \
     curl -sS -m 5 http://maqpna-llamay:8081/healthz
   

   The expected result is a timeout. A JSON answer means your CNI does not enforce NetworkPolicy.

Network policy#

The chart renders three layers. Together they keep llamay private to the gateway:

Keep the gateway's label app.kubernetes.io/name: maqpna-gateway (gateway.name). Both the llamay policy and the operator's per-session policies select it.

Secrets#

Sandboxes never get a llamay key. To bring your own key:

kubectl -n maqpna-system create secret generic llamay-api-key --from-literal=api-key="$(openssl rand -hex 32)"

llamay:
  apiKey:
    existingSecret: llamay-api-key
    key: api-key

To rotate the key, update the Secret, then restart llamay so it reads the new value:

kubectl -n maqpna-system rollout restart deploy/maqpna-llamay

The gateway picks up the new key from the mounted file once the kubelet refreshes it, which can take a minute. Model calls in that window answer 401 from llamay and fail over, if the route has fallbacks. The restart also drops all llamay contexts: the next call of each session costs one prefill, nothing more.

Scaling and sizing#

• llamay runs one replica. Contexts live in the llamay process, and the audit chain has one writer. Scale up, not out: more CPU or a GPU (profile: cuda), concurrency for parallel generations, ctxMemBudget to spill idle contexts to disk.
• Size memory for the model plus contexts. The model is mapped into memory, and each active context holds KV state. Start with a memory limit of about twice the GGUF file size and watch llamay's own /metrics (llamay_kv_bytes, llamay_sessions).
• Gateway replicas. More than one gateway replica needs state.backend: postgres. Context bindings live in each gateway process, so a session that lands on another replica gets a fresh context (correct, but one prefill slower), and fork lineage is per replica.
• Idle contexts. The gateway drops a session's context after contextIdleSeconds (default 1800) and keeps at most maxContexts (default 4096). Tune them in gateway.config.llamay; keys you set there take precedence over the chart's.
• Restarts. The Recreate strategy means a short gap in service during upgrades. With fallbacks in Agent.spec.model.fallbacks, calls fail over to another endpoint meanwhile.

Air-gapped clusters#

• Mirror the llamay image into your registry with the MAQPNA images. With airgap.imageRegistry set, the chart pulls llamay from that registry, under the same repository path.
• Ship the model as an image and set llamay.models.bundle.image (and bundle.path, default /bundle). The image needs /bin/sh and cp.
• airgap.enabled refuses models.pull. In a government deployment, use a llamay sovereign build image, which also disables network model sources.

See Air-gapped install.

What can go wrong#

Next steps#

• Day-to-day development with the stack
• llamay reference: every gateway, Helm and Agent setting
• Security model of the stack
• Scaling and HA
• Production checklist