Build a cost-controlled LLM app

Build an app that calls a model through the MAQPNA gateway, price its tokens, cap spend per session and per day, get a webhook alert near the limit, and see what the app receives at the cap.

In this tutorial you build a small app that asks a model to summarise support tickets, and you stop it from spending more than you allow. Every model call goes through the MAQPNA gateway, which meters the tokens, prices them, checks the budget before each call and sends an alert to a webhook as spend approaches the limit. You run everything on your laptop against a scripted model, so no API key is needed and the numbers are the same on every run.

What you'll build#

flowchart LR
    subgraph laptop["Your laptop"]
        A["app.py<br/>summarizer"] -->|"POST /llm/stub/v1<br/>chat/completions"| GW["MAQPNA gateway<br/>:8080"]
        GW -->|"under budget"| M["stub-llm<br/>scripted model :8000"]
        GW -.->|"over budget<br/>HTTP 402"| A
        GW -->|"budget.threshold<br/>budget.exceeded"| W["alerts.py<br/>webhook :9200"]
        GW --> L[("audit ledger<br/>and meter")]
        P["pricing.json<br/>budgets.json"] -.-> GW
    end

Prerequisites#

• The maqpna CLI (see Install).
• Python 3.10 or later. The tutorial uses only the standard library.
• Free ports 8080 (gateway), 8081 (identity broker), 8000 (stub model) and 9200 (alert receiver).
• Two terminals open in the same new, empty folder (mkdir cost-app && cd cost-app).

The concepts behind this page are in Budgets. The full reference for every budget key is Budgets and cost limits.

How budgets decide#

Read this once; every result below follows from it.

• The gateway checks budgets before each model call and tool call, against the spend metered so far in the window.
• A call that starts under the limit runs to the end and is metered in full. So the last allowed call can take spend a little over the limit, and the next call is refused.
• Windows are a session, a calendar day and a calendar month, in the gateway's budgetTimezone (default UTC).
• At the limit, onExceed decides: deny (the default), require_approval (tool calls are held for a person; model calls are denied) or alert (logged and counted, the call runs).

Step 1: Script the model and price it#

maqpna dev up --stub-llm starts stub-llm, an OpenAI-compatible server that answers from a script, as model route stub. Give it a one-step script so every call returns the same summary. Create script.yaml:

model: stub
steps:
  - content: "Summary: the customer reports a broken item and asks for a refund. Suggested reply: apologise, confirm the order number, offer a replacement or a refund."

Without a pricing table every call costs $0.0000. Create pricing.json with a deliberately high price, so a few calls reach the limit:

{
  "currency": "USD",
  "defaultToolCallUSD": 0,
  "models": {
    "stub": {"inputPer1kUSD": 0.5, "outputPer1kUSD": 1.5}
  }
}

Model keys are the model IDs the route serves. Prices are per 1,000 input and output tokens, in currency. A real table, with tool-call prices and a defaultModel, is in examples/pricing.json and described in Budgets and cost limits.

Step 2: Set the budgets#

Budgets are keyed by scope. Agent keys are <namespace>/<agent>. Create budgets.json:

{
  "agents": {
    "support/summarizer": {
      "perSessionUSD": 0.3,
      "perDayUSD": 0.6,
      "alertAtPercent": [50, 80]
    }
  }
}

The same document also takes namespaces and users maps (with "*" as the default entry), perMonthUSD, and the model limits tokensPerMinute and maxTokensPerRequest. The gateway rejects unknown keys.

Step 3: Start the alert receiver#

The gateway sends notifier events to webhooks. Create alerts.py, which prints each event it receives:

"""alerts: prints every MAQPNA notifier event it receives (port 9200)."""
import json
from http.server import BaseHTTPRequestHandler, HTTPServer


class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        event = json.loads(self.rfile.read(int(self.headers.get("Content-Length", 0))))
        print(self.headers.get("X-Maqpna-Event"), json.dumps(event.get("data")), flush=True)
        self.send_response(204)
        self.end_headers()

    def log_message(self, fmt, *args):
        pass


HTTPServer(("127.0.0.1", 9200), Handler).serve_forever()

Start it in the first terminal with python3 alerts.py and leave it running.

Each delivery is a JSON POST with the event type in the X-Maqpna-Event header and a delivery ID in X-Maqpna-Delivery. With an hmacSecretFile, the gateway also signs the body in X-Maqpna-Signature. Slack, Microsoft Teams and Matrix are the other notifier types; see Notifications.

Step 4: Start a local MAQPNA with pricing, budgets and the webhook#

maqpna dev up --gateway-config FILE merges a JSON object over the generated gateway configuration. Write one that names the two files (absolute paths) and the notifier:

cat > gateway-extra.json <<EOF
{
  "pricingFile": "$PWD/pricing.json",
  "budgetsFile": "$PWD/budgets.json",
  "notifiers": [
    {"name": "local-alerts", "type": "webhook", "url": "http://127.0.0.1:9200/maqpna",
     "events": ["budget.threshold", "budget.exceeded"]}
  ]
}
EOF

In the second terminal, start the stack:

maqpna dev up --no-echo --stub-llm --stub-llm-script script.yaml --gateway-config gateway-extra.json

local MAQPNA is up (~/cost-app/.maqpna)
  gateway   http://127.0.0.1:8080
  broker    http://127.0.0.1:8081
  model     stub -> http://127.0.0.1:8080/llm/stub/v1 (OpenAI base URL)
next: maqpna dev run -- <your agent command>

Check it works#

The gateway logs each budgets load:

grep '"budgets loaded"' .maqpna/logs/maqpna-gateway.log

The line names your file and "agents":1. If the file is invalid, the log says why and no budgets apply until it loads.

Step 5: Write the app#

The app sends the same ticket to the model route until it is told to stop. It uses the OpenAI chat-completions format at the base URL the gateway serves, {gateway}/llm/stub/v1, with the session token as the bearer token. Create app.py:

"""summarizer: asks the governed model route to summarise support tickets."""
import json
import os
import sys
import urllib.error
import urllib.request

BASE = os.environ["MAQPNA_GATEWAY_URL"] + "/llm/stub/v1"
TOKEN = os.environ["MAQPNA_TOKEN"]
TICKET = "My headphones arrived broken after two days. Order A-5521. I want a refund or a new pair. " * 4


def summarise(n):
    body = json.dumps({"model": "stub", "messages": [
        {"role": "system", "content": "Summarise the ticket and suggest a reply."},
        {"role": "user", "content": TICKET}]}).encode()
    req = urllib.request.Request(BASE + "/chat/completions", data=body, headers={
        "Authorization": "Bearer " + TOKEN, "Content-Type": "application/json"})
    try:
        with urllib.request.urlopen(req, timeout=60) as resp:
            out = json.loads(resp.read())
    except urllib.error.HTTPError as e:
        err = json.loads(e.read() or b"{}").get("error", {})
        print(f"call {n}: HTTP {e.code} {err.get('type')}: {err.get('message')}")
        return False
    u = out.get("usage", {})
    print(f"call {n}: {u.get('prompt_tokens')} in, {u.get('completion_tokens')} out")
    return True


for n in range(1, int(sys.argv[1] if len(sys.argv) > 1 else 10) + 1):
    if not summarise(n):
        break

Any OpenAI-compatible client works the same way: set its base URL to MAQPNA_GATEWAY_URL + "/llm/stub/v1" and its API key to the session token. In a cluster the operator sets MAQPNA_MODEL_ENDPOINT to that base URL; see Model calls.

Step 6: Hit the per-session cap#

Run the app as agent summarizer in namespace support:

maqpna dev run --agent summarizer --namespace support -- python3 -u app.py

maqpna dev: session dev-488a0877 (agent summarizer, namespace support, on behalf of you@localhost)
call 1: 115 in, 46 out
call 2: 115 in, 46 out
call 3: 115 in, 46 out
call 4: HTTP 402 maqpna_budget_exceeded: agent budget (session) exhausted for support/summarizer: spent 0.379500 of 0.300000 USD
maqpna dev: dev-488a0877: 3 allow, 1 deny, cost $0.3795 (maqpna dev timeline --last)

Each call costs 115 × 0.5 / 1000 + 46 × 1.5 / 1000 = $0.1265. The third call started at $0.253, under the $0.30 session limit, and took the session to $0.3795. The fourth was refused before it reached the model.

What the app receives at the cap:

Your app should treat 402 as "stop for now": do not retry, report the result, and tell the user. Retrying gets the same answer until the window ends or someone raises the limit.

Check it works#

The first terminal shows the alerts the gateway sent:

budget.threshold {"agent": "summarizer", "key": "support/summarizer", "limitUsd": "0.3000", "namespace": "support", "onExceed": "deny", "percent": "80", "period": "support/dev-488a0877", "scope": "agent", "server": "llm/stub", "spentUsd": "0.2530", "window": "session"}
budget.threshold {"agent": "summarizer", "key": "support/summarizer", "limitUsd": "0.6000", "namespace": "support", "onExceed": "deny", "percent": "50", "period": "2026-10-18", "scope": "agent", "server": "llm/stub", "spentUsd": "0.3795", "window": "day"}
budget.exceeded {"agent": "summarizer", "key": "support/summarizer", "limitUsd": "0.3000", "namespace": "support", "onExceed": "deny", "percent": "100", "period": "support/dev-488a0877", "scope": "agent", "server": "llm/stub", "spentUsd": "0.3795", "window": "session"}

There is no 50 % session alert: the gateway checks before each call, and spend went from 42 % straight to 84 %. Each threshold is sent once per limit and period. The gateway also writes each threshold to .maqpna/logs/maqpna-gateway.log as a budget.threshold warning with the same fields.

Step 7: Hit the daily cap#

A new session gets a new session budget, but the day budget carries over. Run the app again:

maqpna dev run --agent summarizer --namespace support -- python3 -u app.py

maqpna dev: session dev-045a1123 (agent summarizer, namespace support, on behalf of you@localhost)
call 1: 115 in, 46 out
call 2: 115 in, 46 out
call 3: HTTP 402 maqpna_budget_exceeded: agent budget (day) exhausted for support/summarizer: spent 0.632500 of 0.600000 USD
maqpna dev: dev-045a1123: 2 allow, 1 deny, cost $0.2530 (maqpna dev timeline --last)

The alert receiver prints the 80 % day threshold and then budget.exceeded for window day. Every session of this agent is now refused until the next calendar day in UTC.

Check it works#

Load the admin token and read the budgets and spend:

eval "$(maqpna dev env)"
maqpna budgets list

SCOPE  KEY                 WINDOW  PERIOD      SPENT USD           LIMIT USD  EXCEEDED
agent  support/summarizer  day     2026-10-18  0.6325000000000001  0.6        true

maqpna costs summary --scope agent --window day

day window 2026-10-18 (2026-10-18T00:00:00Z to 2026-10-19T00:00:00Z, UTC)
SCOPE  KEY                 COST USD            TOOL CALLS  MODEL CALLS  IN TOKENS  OUT TOKENS
agent  support/summarizer  0.6325000000000001  0           5            575        230

The refused call is in the session's timeline and in the audit ledger:

maqpna dev timeline --last
tail -1 .maqpna/audit.jsonl | jq '{decision, server, tool, rule, reason, costUsd}'

{
  "decision": "deny",
  "server": "llm/stub",
  "tool": "chat.completions",
  "rule": "budget/agent-day",
  "reason": "budget_exceeded:agent/day",
  "costUsd": 0
}

Step 8: Raise the limit while the app runs#

The gateway re-reads budgetsFile when it changes, without a restart. Raise the daily limit:

sed -i.bak 's/"perDayUSD": 0.6/"perDayUSD": 1.0/' budgets.json
sleep 3
maqpna dev run --agent summarizer --namespace support -- python3 -u app.py 1

The single call runs again. The pricing file, unlike the budgets file, is read only when the gateway starts.

To warn without blocking, set "onExceed": "alert" on an entry: calls over the limit run, and the gateway logs and sends the alert. "require_approval" holds tool calls over budget for a person; model calls over budget are still denied.

Step 9: Limit tokens per minute#

A runaway loop can burn a day's budget in seconds. tokensPerMinute slows it down per session and model route. Add a second agent to budgets.json:

{
  "agents": {
    "support/summarizer": {"perSessionUSD": 0.3, "perDayUSD": 1.0, "alertAtPercent": [50, 80]},
    "support/limited": {"tokensPerMinute": 400, "perDayUSD": 5}
  }
}

sleep 3
maqpna dev run --agent limited --namespace support -- python3 -u app.py 5

maqpna dev: session dev-0e959daa (agent limited, namespace support, on behalf of you@localhost)
call 1: 115 in, 46 out
call 2: 115 in, 46 out
call 3: HTTP 429 maqpna_rate_limited: token rate limit of 400 tokens/minute reached for this session; retry after 8s
maqpna dev: dev-0e959daa: 2 allow, 1 deny, cost $0.2530 (maqpna dev timeline --last)

A 429 carries a Retry-After header: wait and retry. A 402 does not: stop. maxTokensPerRequest caps each request's max_tokens (or max_completion_tokens, max_output_tokens) instead of refusing it.

Step 10: Read usage for chargeback#

The meter keeps totals per agent, namespace, user and session:

maqpna session cost dev-488a0877

session dev-488a0877
  costUsd: 0.3795
  inputTokens: 345
  modelCalls: 3
  outputTokens: 138
  ...

Export the same charges as FOCUS 1.2, the FinOps cost-export format, from the gateway or offline from the ledger:

maqpna costs focus --ledger .maqpna/audit.jsonl --format csv --out costs.csv

Each row has BilledCost, ChargeDescription (Model tokens via route stub (stub)), ConsumedQuantity in tokens and ResourceId (ns/support/agent/summarizer/session/dev-488a0877).

The gateway also exports Prometheus counters at /metrics:

curl -s http://127.0.0.1:8080/metrics | grep maqpna_gateway_budget

maqpna_gateway_budget_denied_total{scope="agent/day"} 1
maqpna_gateway_budget_denied_total{scope="agent/session"} 1
maqpna_gateway_budget_alerts_total{scope="agent/day",percent="50"} 1
maqpna_gateway_budget_alerts_total{scope="agent/day",percent="80"} 1
maqpna_gateway_budget_alerts_total{scope="agent/session",percent="80"} 2

Alert on maqpna_gateway_budget_denied_total in your monitoring as well as on the webhook. For signed usage reports to bill tenants, see Usage metering and billing.

Take it to a cluster#

In a cluster you do not write budgets.json. The operator renders it (ConfigMap maqpna-budgets) from:

• Agent.spec.budget (perSessionUSD, perDayUSD, perMonthUSD, currency) and Agent.spec.model (tokensPerMinute, maxTokensPerRequest).
• A cluster-wide BudgetPolicy for namespaces and users, with onExceed and alertAtPercent:

apiVersion: maqpna.com/v1alpha1
kind: BudgetPolicy
metadata:
  name: default
spec:
  namespaces:
    support: {perDayUSD: "50", perMonthUSD: "1000"}
    "*": {perDayUSD: "10"}
  users:
    "*": {perDayUSD: "5"}
  onExceed: deny
  alertAtPercent: [80, 100]

Notifiers are Helm values under approvals.notifiers, with secrets mounted from approvals.notifierSecret. The Helm chart has no value that mounts a pricing file today: set gateway.config.pricingFile and mount the file into the gateway pod yourself. Field details: BudgetPolicy and Helm values.

What you built#

• An LLM app whose every model call is metered and priced by the MAQPNA gateway.
• A per-session cap and a per-day cap on one agent, enforced before each call with HTTP 402 maqpna_budget_exceeded.
• Webhook alerts at 50 % and 80 % of each limit and at the cap, plus the same events in the gateway log and Prometheus counters.
• A token-rate limit that answers HTTP 429 with Retry-After.
• Spend you can read per session, per agent and per day, and export as FOCUS.

Clean up#

maqpna dev down

Stop alerts.py with Ctrl+C in the first terminal. Metered spend is saved next to the ledger (.maqpna/audit.jsonl.finops.json) and restored on the next maqpna dev up in this folder. Delete the .maqpna folder to start from zero.

Troubleshooting#

Next steps#

• Govern the tools your app calls, too: Build a support agent with DLP and approvals.
• Use a real provider instead of the stub: Model providers.
• Send alerts to Slack, Teams or Matrix: Notifications.
• Bill tenants from signed reports: Usage metering and billing.
• Command reference: maqpna budgets list, maqpna costs summary, maqpna costs focus.