Agent Provenance Pack v1.0

This pack is for operators who publish LLM-generated reports, support summaries, release notes, sales briefs, research notes, or executive decisions and need a repeatable proof trail. It checks each claim, confirms the referenced source is present and current, assigns a confidence score, and writes a JSON ledger that a human reviewer or downstream workflow can inspect.

It is not a legal certification tool and it does not prove that a source is true. It proves that a claim is connected to the sources you provided and flags missing, stale, weak, or unsupported evidence.

Quick Start

cd agent-provenance-v1
python3 agent.py --help
python3 agent.py init-sample --dir examples
python3 agent.py verify \
  --deterministic \
  --claims examples/claims.json \
  --sources examples/sources.json \
  --out out/ledger.json
python3 agent.py status --ledger out/ledger.json

One-shot Debian install:

sudo bash install.sh

Manual no-service install:

bash install.sh --no-service
venv/bin/python agent.py run --once --deterministic

What It Does

• Reads a claims file: claim id, text, risk, owner, artifact, and source ids.
• Reads a sources file: source id, URL, owner, verification timestamp, summary, and quotes.
• Scores each claim using deterministic token overlap, quote match, source age, and missing-source checks.
• Writes a ledger with statuses: verified, needs_review, unsupported, missing_source, stale, and invalid.
• Optionally asks your own local LLM CLI for a reviewer note. No API key is required by this pack.

Architecture

          claims.json                      sources.json
              |                                |
              v                                v
        +-------------+                +----------------+
        | Claim input |                | Source ledger  |
        +------+------+                +--------+-------+
               \                                /
                \                              /
                 v                            v
              +------------------------------------+
              | agent.py deterministic verifier    |
              | - source id resolution             |
              | - quote and token evidence score   |
              | - stale-source guard               |
              | - risk escalation flag             |
              +----------------+-------------------+
                               |
                               v
                       out/ledger.json
                               |
                               v
                 optional BYO-LLM reviewer note

Deterministic vs LLM

The LLM path uses llm_adapter.py to call a local CLI you already installed and logged in to: Claude Code, Kimi Code, or Codex. If none is present, the pack continues in deterministic mode.

Input Contract

claims.json can be a list or an object with a claims list:

{
  "claims": [
    {
      "claim_id": "c-refunds",
      "artifact": "weekly-report.md",
      "owner": "support-ops",
      "risk": "high",
      "text": "The refund queue has 18 open cases and needs triage today.",
      "source_ids": ["support-dashboard-2026-07-01"]
    }
  ]
}

sources.json can be a list or an object with a sources list:

{
  "sources": [
    {
      "source_id": "support-dashboard-2026-07-01",
      "title": "Support dashboard export",
      "url": "https://example.invalid/support/dashboard",
      "owner": "support-ops",
      "last_verified_at": "2026-07-01T09:00:00Z",
      "summary": "Daily support metrics export for refunds and triage.",
      "quotes": [
        {"text": "The refund queue has 18 open cases and needs triage today.", "locator": "row 14"}
      ]
    }
  ]
}

Useful Commands

# Generate sample input files
python3 agent.py init-sample --dir examples

# Run one deterministic audit
python3 agent.py verify --deterministic --claims examples/claims.json --sources examples/sources.json

# Inspect health
python3 agent.py status --ledger out/ledger.json

# Inspect one claim
python3 agent.py explain --claim-id c-refunds --ledger out/ledger.json

# Run the service loop once through environment defaults
PROVENANCE_CLAIMS_PATH=examples/claims.json \
PROVENANCE_SOURCES_PATH=examples/sources.json \
PROVENANCE_OUTPUT_PATH=out/ledger.json \
python3 agent.py run --once --deterministic

Expected Output

The ledger contains a summary and one entry per claim:

{
  "summary": {
    "claims_total": 2,
    "verified": 1,
    "unsupported": 1,
    "escalations": 0
  },
  "entries": [
    {
      "claim_id": "c-refunds",
      "status": "verified",
      "confidence": 1.0,
      "evidence": [{"source_id": "support-dashboard-2026-07-01"}]
    }
  ]
}

Operational Fit

Use this pack when the repeated pain is "the LLM produced a polished answer, but I cannot tell which facts are backed by which sources." It fits daily or hourly pipelines where claims are already extracted by another tool, or where a human exports claims to JSON before publication.

Do not use it as the only approval gate for regulated decisions. Treat it as a provenance and triage layer before expert review.