Prove API SLA breaches and claim service credits

Never grant a service credit — or lose a dispute — on a number you cannot prove. This pack turns raw reliability telemetry into a defensible SLA evidence bundle. It reads your latency and error samples, your incident notes, and the contract thresholds, and emits, per contract:

• a pass / fail per objective (uptime, latency p95, error rate),
• the credit due on a proven breach, computed from your credit schedule,
• the evidence that is still missing to make a call.

It is built for the agent economy. When an autonomous client disputes your API's quality, you answer with a bundle, not an argument.

Why this exists

Providers sell reliability guarantees. When something goes wrong the fight is never about goodwill — it is about evidence. Two failure modes cost you money:

1. Paying credits you did not owe, because nobody checked the raw data.
2. Losing a dispute you would have won, because the proof was never assembled.

This agent is deliberately conservative: it never claims a breach it cannot prove from the attached telemetry, and it never credits a customer without the evidence to back it. A contract with no data is insufficient_evidence, not a silent pass.

What it does (deterministic core)

For each contract it matches samples and incidents by contract_id only, then evaluates each objective present in the contract:

• Downtime is summed only from incidents whose counts_against_sla is not false, clipped to the SLA period.
• An objective with no evidence (no samples, no latency values, or no measurable period) becomes missing_evidence — never a pass.
• A proven breach earns the highest matching tier from the contract's credit_schedule; an unproven one earns nothing.

Per-contract status is met, breached, or insufficient_evidence. escalation_required is true for everything except met.

A sample or incident referencing a contract that is not in the batch goes to unmatched_evidence and never invents a contract.

The core is stdlib-only and needs no API key.

Bring your own LLM (optional)

If a Claude Code, Kimi Code, or Codex CLI is logged in on the machine, the agent can attach a reviewer note and draft a customer-facing credit note through PROMPTS.md — using your existing subscription, no API key. If no CLI is present, the agent runs in deterministic mode and stays fully useful. See llm_adapter.py and set PACK_LLM=claude|kimi|codex to pin a backend.

Quick start

sudo bash install.sh                 # deps + venv + systemd service
# or, venv only:
bash install.sh --no-service

# one deterministic run against the bundled example data:
venv/bin/python agent.py audit --deterministic

# inspect one contract's decision:
venv/bin/python agent.py explain --contract-id sla-uptime-api --report out/sla_report.json

# health at a glance:
venv/bin/python agent.py status --report out/sla_report.json

Generate fresh sample inputs any time:

venv/bin/python agent.py init-sample --dir examples

Inputs

Three JSON files (list, or object keyed by contracts / metrics / incidents):

contracts.json

{
  "contract_id": "sla-search-api",
  "customer": "agent://search-bot",
  "service": "search-api",
  "period_start": "2026-07-01T00:00:00Z",
  "period_end": "2026-07-08T00:00:00Z",
  "asset": "USDC",
  "monthly_fee": 200.0,
  "objectives": {"uptime_pct": 99.9, "latency_p95_ms": 300, "error_rate_pct": 1.0},
  "credit_schedule": [
    {"objective": "uptime_pct", "below": 99.9, "credit_pct": 10},
    {"objective": "uptime_pct", "below": 99.0, "credit_pct": 25}
  ]
}

metrics.json — one row per probe/log line:

{"contract_id": "sla-search-api", "latency_ms": 180, "ok": true, "ts": "2026-07-03T10:00:00Z"}

A row is an error when ok is false, error is true, status_code >= 500, or status is one of error/failed/timeout/....

incidents.json

{"contract_id": "sla-search-api", "start": "...", "end": "...", "counts_against_sla": true, "note": "..."}

Output

A single JSON report with the roots mode, summary, entries (one per contract), unmatched_evidence and llm_review. Each entry carries the objectives with threshold vs measured, breached_objectives, missing_evidence, credit_pct, credit_amount, an evidence block (sample_count, error_count, latency_sample_count, incident_count, counted_incident_count, downtime_seconds, period_seconds), and escalation_required. summary includes credits_due_by_asset — credits are summed per asset (USDC and ETH never add up into one number).

llm_review is always present: {"available": false, "backend": "..."} in deterministic mode, or the reviewer's JSON (reviewer_status, notes, credits_to_confirm, evidence_to_collect) when a CLI backend is connected.

Example (deterministic, bundled data):

{
  "mode": "deterministic",
  "summary": {
    "contracts_total": 6,
    "met": 2,
    "breached": 3,
    "insufficient_evidence": 1,
    "unmatched_evidence": 1,
    "credits_due_by_asset": {"USDC": 105.0}
  },
  "entries": [
    {
      "contract_id": "sla-uptime-api",
      "status": "breached",
      "credit_pct": 25,
      "credit_amount": 250.0,
      "evidence": {
        "sample_count": 3,
        "incident_count": 1,
        "counted_incident_count": 1,
        "downtime_seconds": 12096.0
      }
    }
  ],
  "unmatched_evidence": [
    {"kind": "sample", "sample_id": "or-01", "contract_ref": "sla-not-in-batch"}
  ],
  "llm_review": {"available": false, "backend": "deterministic (forced)"}
}

Tests

bash smoke_test.sh        # offline, deterministic, no network, no keys

Use a local model (Ollama)

The optional LLM step can also run on your own machine through Ollama, so no text leaves your server. No account and no API key.

ollama pull <model>               # any chat model you like
export PACK_LLM=ollama            # prefer Ollama even if a coding-agent CLI is installed
export PACK_OLLAMA_MODEL=<model>  # optional: default is the first chat model the server lists
python3 -c "import llm_adapter; print(llm_adapter.detect())"   # prints: ollama

• PACK_OLLAMA_HOST points to another server (default http://127.0.0.1:11434); OLLAMA_HOST is also read when PACK_LLM=ollama.
• PACK_OLLAMA_NUM_CTX sets the context size if your reports or prompts are long.
• Without PACK_LLM, a local Ollama server is used only when no Claude Code, Kimi Code or Codex CLI is found.
• If the server is down or the model is missing, the pack keeps working in its deterministic mode.

Learning memory (local and private)

Every LLM call this pack makes is journaled in ./.pack_memory/ next to the code, so the pack can improve for you. Nothing is ever sent anywhere.

• What is kept: the task name, the masked input, the masked answer, status and latency. Keys, tokens, e-mails, phone numbers, IP addresses, card and IBAN numbers and personal paths are masked before writing; the system prompt is never stored.
• Teach it: python3 pack_memory.py feedback last ok confirms a result, python3 pack_memory.py feedback last bad --fix "the right answer" corrects it. Confirmed and corrected runs are reused as validated examples when a similar input comes in. Unrated runs are never injected into a prompt.
• Find similar cases: python3 pack_memory.py similar "some text" and python3 pack_memory.py clusters (groups of near-identical failures) work in plain Python, no model needed. Optional meaning-based matching uses an embedding model you already run locally: python3 pack_memory.py enable-semantic (Ollama on this machine only).
• Share, only if you want: python3 pack_memory.py export writes an anonymised summary (counts, statuses, size buckets, no text) for you to review.
• Control: PACK_MEMORY=0 turns it off, PACK_MEMORY_STORE_TEXT=0 keeps fingerprints only, python3 pack_memory.py purge --yes wipes it.

License

Personal use per purchase. Resale prohibited.