Enforce LLM API cost budgets and spend limits

An offline-first budget guard for operators running several LLM agents. It turns provider usage exports into deterministic daily, monthly, per-project, and per-request decisions: continue, review_usage, or block_new_llm_calls.

It does not intercept provider traffic or guarantee that a remote API stops billing. It produces a machine-readable gate that your scheduler, proxy, or CI job can enforce. The core requires no network and no LLM. An optional local Claude, Kimi, or Codex CLI can explain incidents after the hard rules run.

Who it is for

• operators running autonomous jobs with shared provider accounts;
• teams needing project-level limits before receiving a monthly invoice;
• self-hosters who want auditable policy decisions without a billing SaaS.

Quick start

cp policy.example.json policy.json
cp sample_usage.jsonl usage.jsonl
python3 agent.py --policy policy.json --ledger usage.jsonl check
python3 test_agent.py

Ingest an export without duplicating known request IDs:

python3 agent.py --ledger usage.jsonl ingest provider-export.jsonl

Use the exit status as a gate: 0 means ok or warn; 2 means a hard stop.

python3 agent.py --policy policy.json check || systemctl stop my-llm-worker

Architecture

provider exports / proxy logs
             |
             v
     validate + deduplicate
             |
             v
       append-only JSONL
             |
             v
 OBSERVE -> ORIENT -> DECIDE -> ACT
    |          |         |        |
 totals     policy     severity   JSON gate
             |
             +---- optional BYO-LLM incident explanation

Event contract

Each JSONL row must contain timestamp, project, provider, model, and cost_usd. Token counts and request_id are optional. Timezones are mandatory. Rows with duplicate non-empty request IDs are ignored during ingestion.

Included operational paths

• check: evaluate once and write latest-report.json;
• watch: reevaluate continuously for systemd;
• report --llm: add a non-authoritative incident explanation;
• backend: show whether an optional supported CLI is available.

See CUSTOMIZE.md for adapters, SPEC.md for exact semantics, and CHECKLIST.md for deployment.

Use a local model (Ollama)

The optional LLM step can also run on your own machine through Ollama, so no text leaves your server. No account and no API key.

ollama pull <model>               # any chat model you like
export PACK_LLM=ollama            # prefer Ollama even if a coding-agent CLI is installed
export PACK_OLLAMA_MODEL=<model>  # optional: default is the first chat model the server lists
python3 -c "import llm_adapter; print(llm_adapter.detect())"   # prints: ollama

• PACK_OLLAMA_HOST points to another server (default http://127.0.0.1:11434); OLLAMA_HOST is also read when PACK_LLM=ollama.
• PACK_OLLAMA_NUM_CTX sets the context size if your reports or prompts are long.
• Without PACK_LLM, a local Ollama server is used only when no Claude Code, Kimi Code or Codex CLI is found.
• If the server is down or the model is missing, the pack keeps working in its deterministic mode.

Learning memory (local and private)

Every LLM call this pack makes is journaled in ./.pack_memory/ next to the code, so the pack can improve for you. Nothing is ever sent anywhere.

• What is kept: the task name, the masked input, the masked answer, status and latency. Keys, tokens, e-mails, phone numbers, IP addresses, card and IBAN numbers and personal paths are masked before writing; the system prompt is never stored.
• Teach it: python3 pack_memory.py feedback last ok confirms a result, python3 pack_memory.py feedback last bad --fix "the right answer" corrects it. Confirmed and corrected runs are reused as validated examples when a similar input comes in. Unrated runs are never injected into a prompt.
• Find similar cases: python3 pack_memory.py similar "some text" and python3 pack_memory.py clusters (groups of near-identical failures) work in plain Python, no model needed. Optional meaning-based matching uses an embedding model you already run locally: python3 pack_memory.py enable-semantic (Ollama on this machine only).
• Share, only if you want: python3 pack_memory.py export writes an anonymised summary (counts, statuses, size buckets, no text) for you to review.
• Control: PACK_MEMORY=0 turns it off, PACK_MEMORY_STORE_TEXT=0 keeps fingerprints only, python3 pack_memory.py purge --yes wipes it.