Build a technical knowledge base from your own docs

Self-hosted agent pack for building and maintaining an autonomous technical knowledge base. Version 1.0.0 — actionable kit with scripts, templates, named prompts and a checklist.

Overview

This pack provides an operational autonomous agent for the knowledge-base domain. It includes a dedicated OODA loop, versioned prompts, configuration templates and a set of scripts for installation, testing and execution.

The core of the product is a SQLite database of knowledge articles. Each article carries a unique identifier, a topic, content, tags, a source, a content signature (hash) and a status. The agent collects new sources, groups them by topic, removes duplicates, updates the database, then rebuilds a searchable index. It does not replace a large-scale vector search engine and does not execute any arbitrary command supplied in the sources.

The pack targets teams that accumulate technical documentation, runbooks, tickets, conversations and snippets, and that need an up-to-date operational "second brain" without manual effort.

The pack is designed to be self-hosted (BYO-LLM): no API key is included. The LLM adapter detects a local or CLI provider (ollama, claude-code, kimi, codex).

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                         External sources                             │
│   Markdown files     /  tickets  /  conversations  /  runbooks       │
└──────────────────────────────────┬──────────────────────────────────┘
                                   ▼
┌─────────────────────────────────────────────────────────────────────┐
│  IDLE → COLLECTING → CLUSTERING → DEDUPLICATING → UPDATING →         │
│  INDEXING → REPORTING                                                │
│                           OODA State Machine                         │
└──────────────────────────────────┬──────────────────────────────────┘
                                   ▼
┌─────────────────────────────────────────────────────────────────────┐
│  Named prompts → Deterministic handler  →  BYO-LLM Adapter          │
│  scripts/test_prompt.py    scripts/run_agent.py                     │
└──────────────────────────────────┬──────────────────────────────────┘
                                   ▼
┌─────────────────────────────────────────────────────────────────────┐
│  SQLite KB  →  templates/*.yaml  →  Articles / Index / Reports       │
└─────────────────────────────────────────────────────────────────────┘

Pack contents

Quick installation

cd /opt/autonomous-stack/agent-knowledge-base-v1
bash scripts/install.sh

The installer checks for Python 3.10+, creates a venv, installs the dependencies, detects a local LLM provider and configures a systemd unit.

Immediate test

# Test the main prompt
python3 scripts/test_prompt.py --prompt collector --input templates/source_schema.yaml

# Run the agent in dry-run mode
python3 scripts/run_agent.py --config scripts/config.yaml --dry-run

# Run the offline tests
bash smoke_test.sh

Execution contract

• pending: source collected and awaiting processing.
• clustered: source assigned to a topic.
• deduplicated: duplicate detected and merged.
• updated: article inserted or updated in the database.
• indexed: article referenced in the searchable index.
• archived: obsolete article, kept for history.

The unique identifier of an article (entry_id) combines the topic and a hash of the content. The content signature (content_hash) prevents unnecessary updates. Unknown or malformed sources are rejected with an explicit status.

Ingestion is strictly file-based: the agent only reads files that actually exist under source_dir (via _read_sources). It never invents files from operator_note or claimed_sources — such fields, when present in the input, are ignored for ingestion and only used as context. An empty or missing source directory is a valid deterministic outcome (ok=true, collected=0, mode="deterministic"), never a failure.

Common operations

python3 agent.py ingest --source data/runbooks --type markdown
python3 agent.py run --once --deterministic
python3 agent.py status
python3 agent.py query --topic deployment
python3 agent.py report --format json

The deterministic mode covers the included handlers without an LLM. The BYO-LLM mode is reserved for generating summaries and tag suggestions; its output must remain a JSON object and any error follows the same retry policy.

Customization

1. Copy scripts/config.yaml to config.knowledge-base.yaml.
2. Adapt the templates in templates/.
3. Plug in your own sources in agent.py or via the ingest_files handler.

Costs and limits

This pack favors deterministic logic for critical decisions. The LLM is used for summarization, semantic clustering and tag suggestion. Refer to SPEC.md for the operating cost estimate.

Use a local model (Ollama)

The optional LLM step can also run on your own machine through Ollama, so no text leaves your server. No account and no API key.

ollama pull <model>               # any chat model you like
export PACK_LLM=ollama            # prefer Ollama even if a coding-agent CLI is installed
export PACK_OLLAMA_MODEL=<model>  # optional: default is the first chat model the server lists
python3 -c "import llm_adapter; print(llm_adapter.detect())"   # prints: ollama

• PACK_OLLAMA_HOST points to another server (default http://127.0.0.1:11434); OLLAMA_HOST is also read when PACK_LLM=ollama.
• PACK_OLLAMA_NUM_CTX sets the context size if your reports or prompts are long.
• Without PACK_LLM, a local Ollama server is used only when no Claude Code, Kimi Code or Codex CLI is found.
• If the server is down or the model is missing, the pack keeps working in its deterministic mode.

Learning memory (local and private)

Every LLM call this pack makes is journaled in ./.pack_memory/ next to the code, so the pack can improve for you. Nothing is ever sent anywhere.

• What is kept: the task name, the masked input, the masked answer, status and latency. Keys, tokens, e-mails, phone numbers, IP addresses, card and IBAN numbers and personal paths are masked before writing; the system prompt is never stored.
• Teach it: python3 pack_memory.py feedback last ok confirms a result, python3 pack_memory.py feedback last bad --fix "the right answer" corrects it. Confirmed and corrected runs are reused as validated examples when a similar input comes in. Unrated runs are never injected into a prompt.
• Find similar cases: python3 pack_memory.py similar "some text" and python3 pack_memory.py clusters (groups of near-identical failures) work in plain Python, no model needed. Optional meaning-based matching uses an embedding model you already run locally: python3 pack_memory.py enable-semantic (Ollama on this machine only).
• Share, only if you want: python3 pack_memory.py export writes an anonymised summary (counts, statuses, size buckets, no text) for you to review.
• Control: PACK_MEMORY=0 turns it off, PACK_MEMORY_STORE_TEXT=0 keeps fingerprints only, python3 pack_memory.py purge --yes wipes it.

License

Personal use per purchase. Resale prohibited.