Execution order
Start with the corpus, not the chat box.
The Python pipeline cleans source documents into a processed cache, then applies paragraph, section, fixed, or parent-child chunking. Nomic embeddings run locally through Ollama and are stored in a FAISS index. Chunk metadata keeps source, type, URL, section, ordinal, and dataset attached to the text, so attribution survives retrieval.
Dense similarity is only one vote
At query time, file-backed aliases and extracted entities produce lower-weight expansion terms. Dense FAISS candidates and lexical BM25 candidates are gathered independently, then combined before an optional cross-encoder rerank. Reranking is machine-aware: it is enabled only within documented CPU, RAM, and index-size gates unless explicitly forced.
Parent-child mode embeds small child chunks of roughly 200 tokens for precise matching but gives the model their larger, roughly 800-token parents. That separates the unit best suited to search from the unit best suited to reading. It avoids making every chunk mediocre at both jobs.
Context and grounding are separate checks
The final candidates are packed into the prompt with source metadata and passed to a local Ollama model. Answer grounding is then checked against the retrieved context. The default verifier uses token overlap with a paraphrase-tolerant threshold; semantic similarity is available as a separate mode. Neither is proof of truth. They are observable rejection signals for answers that drift away from supplied evidence.
Evaluation is part of the pipeline
The gold set contains more than eighty questions across easy, medium, hard, mismatch, negative, and fluency cases, with expect_all and expect_none assertions. Fast, standard, and deep profiles let development checks and expensive runs share the same evaluator. Balanced sampling prevents a quick smoke run from accidentally selecting only easy cases.
| Tool | Question it answers |
|---|---|
| Quality evaluator | Did retrieval and the answer satisfy the gold expectations? |
| Latency benchmark | Where did cold and warm time go: embedding, retrieval, rerank, or generation? |
| Index health | Do vectors, metadata, and source state still agree? |
| JSONL pipeline log | Which rewrite, chunks, scores, context size, answer, and timings produced this result? |
What remains experimental
The working corpus is deliberately narrow Skyrim material. Broader Australian datasets are planned, not implemented. Cross-encoder quality depends on an optional downloaded model, and grounding heuristics remain heuristics. The useful result is not “RAG solved”; it is a pipeline where changing chunk size, fusion, expansion, or reranking produces evidence instead of vibes.