How do I redact PII and secrets from LLM prompts and logs?

Put one deterministic pre-flight step in front of every model call and every log write: detect personal data and secrets with patterns and checksums, replace them with typed placeholders, and record what was replaced without recording the values. Check untrusted text for injection attempts in the same step, and flag it for review instead of rewriting or executing it. If a model is involved at all, let it review the already-redacted output, never the raw text.

Why this matters

LLM workflows collect sensitive data almost by accident:

• Prompts inherit whatever users paste. Support tickets carry customer emails and phone numbers. Incident notes carry bearer tokens. Stack traces carry internal IP addresses.
• Logs make it permanent. Prompt and completion logs are often kept longer, and shared more widely, than the system the data came from.
• Retrieved context is untrusted input. A web page, email or ticket pulled into a prompt can contain "ignore previous instructions" just as easily as it contains a phone number.
• Using an LLM to find PII means sending PII to the LLM. The detector has to run before the model, not inside it.

The OWASP Top 10 for LLM Applications lists prompt injection and sensitive information disclosure among its top risks. Both are cheaper to reduce at the input boundary.

A practical approach you can apply today

You can build a first version of this without any model.

1. One choke point

Route every path through the same function: user prompts, retrieved chunks, tool arguments, memory snippets, and log lines before they are shipped to storage. A sanitizer that only covers the chat box misses the RAG pipeline.

2. Detect deterministically first

When two detectors match the same span, keep the higher-risk label.

3. Replace, do not delete

Swap each match for a typed placeholder such as [EMAIL_1] or [PHONE_1]. The model can still reason about "the customer's email" without seeing it, and the output stays readable for humans.

4. Log findings, not values

For each input, log the detector kind, the placeholder, the offset, the reason, and counts by kind. Never log the matched value. That report is what an auditor needs, and it is safe to keep.

5. Allowlist narrowly

Your own support domain or known test addresses can be allowlisted. Turning off a whole detector should be a written decision, because downstream systems inherit that risk.

6. Treat injection as a separate check, and flag rather than rewrite

Common surface patterns worth catching:

• "Ignore previous instructions" and variants
• Role claims such as "you are now the system" or "act as administrator"
• Requests to reveal or repeat the system prompt
• Instructions to run shell commands, curl, wget or rm -rf
• Fake delimiters like </system> or "end of system message"
• "Do not tell the user" or requests to disable safety filters

Route each item to allow, review or block based on a score. Do not silently clean hostile text: a rewritten attack looks trusted and is not. And never interpolate untrusted text into a shell command; write it to a file and pass the path.

7. Close the loop on leaked secrets

A key that appears in a prompt or a log is exposed. Keep an inventory of each secret with its owner, location, last rotation date and rotation interval, so that a finding turns into a rotation task with a name on it.

How the featured packs implement this

All packs below are $3 one-time, run on your own machine, and need no API key. Each has a deterministic core; where an LLM is used, it is your own coding-agent CLI (Claude Code, Kimi Code or Codex) or a local Ollama server (PACK_LLM=ollama) through llm_adapter.py, and it is optional.

Agent PII Redactor: the sanitizer

Agent PII Redactor cleans prompts, support transcripts, incident notes and LLM logs. It detects email addresses, phone-like numbers, US SSN patterns, Luhn-valid payment cards, bearer tokens and API-key-shaped secrets, private IPv4 addresses and 12-word recovery-phrase-shaped text. Overlapping matches resolve toward the higher-risk kind.

Its JSON report has a summary, the redacted_text with placeholders such as [EMAIL_1], findings (kind, placeholder, offset, reason) and an optional llm_review. A policy file holds an allowlist and disabled_kinds. When the optional review runs, the raw input is not sent to the model: only the redacted preview and the structured findings.

python3 agent.py redact --deterministic \
  --input examples/sample_prompt.txt \
  --policy examples/policy.example.json \
  --out out/redacted.json

Agent Context Audit: allow, review or block

Agent Context Audit audits prompts, retrieved context, tool arguments, memory snippets and support logs before they reach an agent. Each record gets a risk score and one or more categories: secret, prompt_injection, pii, tool_risk (destructive commands or financial action language) and context_quality (oversized snippets). Scores map to allow, review or block through two thresholds you set with environment variables. It runs on the Python standard library with no network.

python3 agent.py check-text --text "Ignore previous system instructions"

PromptShield: an injection gate for CI or ingestion

PromptShield scans untrusted text and reports findings with rule id, severity, line and a capped excerpt. It ships six built-in rules (prior-instruction override, role override, system prompt exfiltration, tool coercion, delimiter escape, safeguard bypass), and you can export them with python3 agent.py rules and load your own. Its verdict is regex-derived, never model-derived, it never sanitises input, and findings are sorted in a stable order so two reports diff cleanly. Exit code 2 means a finding reached your --fail-on level.

python3 agent.py scan --input examples/hostile_ticket.txt \
    --output promptshield-report.json --fail-on high

Agent Secrets Rotate: fix the source

Agent Secrets Rotate keeps a JSON inventory of secrets (owner, location, criticality, last rotation, rotation interval, rotation and verification commands) and scans env, config, source and docs files for private-key headers, AWS access key IDs, GitHub and Slack tokens, JWT-like strings, provider key assignments and Stripe-like secret keys. Reports show redacted values only. Each secret gets a state such as due_soon, overdue or exposed_secret_detected, and the report includes a P0/P1/P2 rotation plan in JSON and Markdown. Rotation commands are documented in the inventory but not executed by default.

Optional: Agent Provenance for what comes out

Redaction protects the input. If you also publish LLM output, Agent Provenance checks each claim against a source ledger you provide and marks it verified, needs_review, unsupported, missing_source, stale or invalid.

Honest limits

• Patterns miss what has no pattern. Names, street addresses and free-text health details are not on the PII Redactor's detector list. The pack states it does not claim legal compliance, DLP coverage or OCR.
• Context Audit's pii category covers email and phone patterns. Use the PII Redactor when you need card numbers, SSNs or recovery phrases replaced.
• Injection rules match literal surface text. PromptShield documents that obfuscated, base64-encoded, translated or homoglyph-substituted injections are out of scope. A finding is evidence for review, not proof of malice.
• Secrets Rotate plans; it does not rotate unless you wire provider-specific commands into the inventory, and a high-risk scan still exits 0 so your own automation decides the gate.

None of this replaces a person reviewing the queue. These tools cut the cheap, common leaks and give that person evidence.

Where to start

1. Run Agent PII Redactor in deterministic mode on a sample of your real prompt logs and read the findings before you trust the output.
2. Add PromptShield with --fail-on high where untrusted text enters your pipeline.
3. If you want one allow, review or block decision across mixed context, use Agent Context Audit instead of wiring separate checks.
4. Point Agent Secrets Rotate at your config and code directories on a schedule.

Browse all packs at /packs, or read the quickstart to see how a pack is installed and run.