Agent ToolSpec Pack v1.0

Contract guard for the tool/function schemas your LLM agent exposes to a model.

You rename a parameter, tighten a type, or make an optional field required. Your tests pass — the code is internally consistent. But the model was prompted with the old schema, so it keeps emitting the old argument shape, and the tool call fails at runtime. Often silently. Often only in production, on the calls you never wrote a test for.

Agent ToolSpec diffs the schemas you are about to ship against the baseline the model already knows, classifies every change as breaking or compatible, and cross-checks that a real handler exists for every tool you declare. It exits non-zero on breaking drift, so you can wire it into CI as a gate.

The promise (and its limits)

It does: deterministic, explainable schema diffing — parameter removals, type changes, new required arguments, narrowed enums, missing handlers, and handlers the model can never reach. Every verdict is derived only from the files you attach.

It does not: read your mind about semantics. If you keep a parameter's name and type but silently change what it means, that is a semantic break no schema diff can see. It also does not call your live model or replay traffic.

Who this is for

• Teams shipping an LLM agent whose tool definitions change between releases.
• Anyone running function calling / tool use where a bad arg fails quietly.
• Operators who want a cheap, offline pre-flight gate before a deploy.
• Builders who need a small, editable starting point for schema governance.

Quick start

bash install.sh                       # one-shot Debian install
python agent.py init-sample --dir examples
python agent.py audit \
    --baseline examples/baseline_tools.json \
    --current  examples/current_tools.json \
    --handlers examples/handlers.json \
    --deterministic
echo $?                               # 2 => breaking change found

The sample ships a deliberate break: search_docs.top_k flips from an optional integer to a required string. The agent flags it and blocks.

How it reads a change

   baseline_tools.json          current_tools.json         handlers.json
   (what the model knows)       (what you ship)            (what code accepts)
            |                          |                         |
            +------------+-------------+------------+------------+
                         |                          |
                    diff_parameters()         handler cross-check
                         |                          |
                         +-------------+------------+
                                       |
                                  classify()
                                       |
        +-------------+----------------+----------------+-------------+
        |             |                |                |             |
     stable    compatible_change  breaking_change  missing_handler  removed_tool
        |             |                |                |             |
        +------ ALLOW -+                +--------- BLOCK (exit 2) ----+

What counts as breaking

Set TOOLSPEC_STRICT_NEW_TOOLS=1 to also flag a brand-new tool that ships with a required argument.

Bring your own LLM (optional)

With claude, kimi-code, or codex on your PATH, the agent adds a written migration review through the Reviewer prompt in PROMPTS.md. No API key — your existing CLI subscription is the billing relationship. The LLM only comments; it never overrides a deterministic verdict.

python agent.py status                # which backend is connected, if any
python agent.py audit --baseline b.json --current c.json    # LLM review attached

Without a CLI, the deterministic core runs alone and the report says so.

CI gate

python agent.py audit --baseline baseline_tools.json --current current_tools.json \
    --handlers handlers.json --deterministic || exit 1

Commit baseline_tools.json as the schema your deployed prompt was built against. Refresh it only when you re-prompt or re-deploy the model.

Files

License

Commercial pack. White-label friendly — rename, re-theme, and ship it.