K81R

K81R was a 2023 personal project for searching and questioning the notes, decision records, release manifests, case-study drafts, and technical references accumulated across the earlier projects. P6X4 had consolidated those materials into a coherent document model. The arrival of widely accessible generative AI made a conversational interface feel suddenly plausible.

The first answer sounded better than the evidence

The first prototype embedded document chunks, retrieved the nearest few, and asked a language model to answer. Its demonstration was impressive until I asked which retry policy Z29C used for an adapter with no idempotency and no status lookup.

The answer said Z29C attempted three retries with exponential backoff. It was fluent, specific, conventional, and wrong. The source said the opposite: a possibly accepted request with no query path entered outcome-unknown and did not retry automatically.

The model had turned a common pattern into a false claim about my own system. Adding a citation did not solve it; the cited paragraph did not support the sentence.

K81R changed direction. It would not be a chat box over embeddings. It would be a search and evidence system that could synthesize only when the collection supported a bounded answer, show how every claim connected to a source revision, and decline when retrieval was insufficient.

The project used a private personal corpus and synthetic evaluation questions. It did not serve customers or claim production use inside an organization.

Search remained a first-class product

I kept ordinary search beside generated answers. A person could search exact terms, filter by project, year, document type, and status, and inspect matching passages without asking the model to mediate them.

The lexical index handled identifiers, error codes, project names, exact phrases, and rare technical vocabulary well. An embedding index helped when the question and source used different language. Neither was universally better.

The result page separated three actions:

• Find documents and passages.
• Assemble an evidence set for a question.
• Generate a bounded synthesis from that set.

This prevented every query from paying model latency and prevented a failed synthesis from hiding useful retrieval.

Search results showed title, document type, project, effective date, revision, matched passage, and why the item matched. Superseded revisions were excluded by default and available through a history filter.

K81R treated a good list of sources as a successful answer when synthesis was unnecessary or unsafe.

A document was more than text

The ingestion pipeline parsed Markdown, release records, decision documents, incident notes, and structured project metadata into one document envelope:

documentId
revisionId
kind
title
project
effectiveAt
status
sourcePath
contentDigest
accessScope
sections

Stable document identity survived revisions. Revision identity named the exact content used by an index or answer. Content digests made duplicate and stale processing visible.

The parser preserved heading hierarchy, list boundaries, code blocks, table labels, and links. Flattening everything into plain text had made retrieved passages lose their context. A line such as “do not retry” means more when the chunk retains the adapter capability table and the heading “Unknown outcome.”

Ingestion generated lineage records from source file through parsed section, chunk, embedding model version, index version, and answer citation. Rebuilding an index did not overwrite old lineage needed to explain earlier answers.

Documents carried access scope before entering indexes. Retrieval applied scope server-side. A broad candidate set was never sent to the browser for filtering.

The first chunker split text by token count with overlap. It cut lists from their headings, separated claims from caveats, and duplicated partial sentences across citations.

K81R chunked around document structure. A section remained intact when reasonably small. Longer sections split at paragraph or list-item boundaries and carried their ancestor headings. Code blocks stayed whole within a size limit.

Each chunk stored a compact context prefix used for embedding: document title, project, heading path, and content type. The displayed passage remained the original authored text, not the prefixed embedding string.

Overlap was selective. A short definition could travel into an adjacent chunk; a large repeated paragraph could not. Retrieval needed enough context for interpretation without flooding the index with near duplicates.

Tables received a textual representation that repeated relevant row and column labels. A bare cell value was not a defensible citation.

Chunk boundaries were versioned. Changing the algorithm created a new index build and re-ran the evaluation set. I did not assume a new embedding model or chunk size was automatically better.

Hybrid retrieval

K81R combined lexical BM25-style scoring with embedding similarity. Each produced a ranked list. A reciprocal-rank fusion step combined them without pretending their raw scores shared one scale.

Metadata filters applied early. A query about Z29C could favor that project without preventing cross-project results when the question asked about continuity. Recency was a weak feature, not a universal boost; an older decision record could remain authoritative.

A lightweight reranker considered the full question, passage, heading path, document type, and revision status. It operated on a bounded candidate set.

The interface exposed retrieval modes in the diagnostic view, not the ordinary answer. For evaluation, I could see whether an exact identifier came from lexical search, whether semantic retrieval found a differently worded section, and where reranking displaced a relevant passage.

The first hybrid version simply averaged normalized scores. Outlier normalization made result order unstable as the candidate set changed. Rank fusion produced more predictable behavior and made each retriever independently testable.

Evidence sets before answers

For a question, K81R first assembled an evidence set. Passages were grouped by document and heading, near duplicates removed, and contradictory or superseded sources labeled.

The evidence set had a token budget, but diversity mattered more than filling it. Ten adjacent chunks from one document could crowd out a separate decision that changed the conclusion.

An evidence item carried:

• Exact revision and source location.
• Passage text and heading path.
• Retrieval and reranking reasons.
• Effective and superseded status.
• Access scope already applied.
• Relationships to other selected passages.

The user could inspect the evidence set without generating prose. Removing one source and regenerating created a new answer revision linked to the modified set.

This stage made retrieval a reviewable artifact. The model did not receive an invisible context window assembled by application code and then disappear behind its response.

Answerability was separate from fluency

The first wrong answer occurred because the system asked the model to answer every query. K81R added an answerability decision before synthesis.

Answerability considered:

• Whether retrieval contained the entities and relationship asked about.
• Whether top evidence directly addressed the requested claim.
• Whether important sources contradicted one another.
• Whether the question required information outside the collection.
• Whether the evidence was current for a time-sensitive question.
• Whether the answer would require combining passages through an unsupported assumption.

The implementation combined deterministic checks, retrieval features, and a small structured model judgment. The model returned reasons and cited the passages used for that judgment. A threshold alone did not become truth; it was calibrated on the evaluation set and reviewed by failure category.

When answerability was low, K81R returned the useful evidence it had, stated the missing relationship, and suggested a narrower query. It did not preface a guess with “based on the available information.”

This refusal path was a product result, not an exception. Its design received the same attention as the generated answer.

Claim-level citations

The synthesis prompt required a structured result: answer sentences or clauses, each linked to supporting evidence IDs. The server rejected unknown IDs and statements with no citation where a factual claim required one.

Citation existence did not prove support. A verification pass compared each claim with its cited passages and classified supported, partially supported, contradicted, or not established.

Failed claims were removed, narrowed, or sent to review. The system could return several supported facts and explicitly leave one part unanswered.

The interface placed citations immediately after the supported claim. Opening one showed the exact passage in context, document title, heading, and revision. If the source later changed, the answer still opened the revision it had used and offered the current version separately.

I avoided one citation at the end of a long paragraph because it was impossible to tell which sentence the source supported. Dense citation was visually quieter when each marker had a clear job.

The original fluent-but-wrong retry answer became a permanent evaluation case. The correct response explained that the policy depended on adapter capability and cited the unknown-outcome section.

Prompt instructions were not a security boundary

Personal documents could contain text that looked like instructions. K81R treated retrieved content as quoted evidence, not as trusted control text.

The model input placed system rules and structured evidence in separate fields. Each evidence item was escaped and labeled with identity. The application did not allow a document to introduce tools or change the output schema.

The answer path had no mutation tools. It could retrieve within the authorized corpus and produce a proposal. It could not send messages, edit source files, or trigger Z29C workflows.

Prompt-injection fixtures appeared in the evaluation set: documents telling the model to ignore the question, reveal other sources, or cite a fabricated ID. Passing meant the output stayed within the allowed evidence schema and access scope.

Security checks occurred outside the model. Retrieval filtered by authorization. Citation IDs were validated by the server. Rendered Markdown passed through a strict policy. Logs excluded private passage contents by default.

The model could still follow malicious text in ways that degraded the answer. The architecture limited consequence and made the failure detectable rather than trusting one prompt to solve adversarial input.

I assembled questions from the personal corpus and labeled relevant documents, required claims, acceptable abstention, and common traps.

The set included:

• Exact lookup: a stable project fact or identifier.
• Paraphrase: a concept described with different language.
• Multi-source synthesis: a lesson carried across projects.
• Temporal question: what was true at a specific revision.
• Contradiction: an early design later rejected.
• Unanswerable question: plausible but absent information.
• Adversarial source: instruction-like text in a document.
• Access boundary: relevant material outside the test identity’s scope.

Evaluation separated retrieval from generation. Recall at a bounded candidate count asked whether relevant evidence was found. Evidence selection measured whether the final set retained it. Claim support measured the generated result. Abstention precision and recall measured refusal behavior.

One aggregate answer score would have hidden whether a failure came from ingestion, retrieval, selection, answerability, generation, or verification.

Every observed defect became a minimized case. The evaluation set therefore grew from product failures rather than a generic benchmark alone.

K81R recorded the complete configuration of an evaluation run: corpus revision, parser and chunker versions, embedding model, lexical index, fusion settings, reranker, prompt, generation model, and verification logic.

A candidate change ran against the fixed set and produced a per-case diff. Improvements in one category could not hide regressions in refusal or citations.

I reviewed examples where metrics moved. A higher retrieval score caused by returning many redundant chunks could make answer context worse. A stricter answerability threshold could improve factuality while refusing simple supported questions.

Latency and cost were tracked by stage. Ordinary lexical search needed no model call. Evidence-only results stopped before generation. Cached embeddings were keyed by content digest and model version.

The project did not publish invented state-of-the-art numbers. Evaluation was a local engineering instrument for deciding whether K81R behaved better on its defined corpus.

The answer record

Every generated answer became an immutable record containing question, normalized filters, evidence-set revision, prompt version, model identifier, structured claims, verification result, latency, and creation time.

Regeneration created a new revision. The interface could compare changed claims and evidence. A current answer was not silently replaced because a model or index changed.

Feedback attached to a specific answer revision and category: wrong source, unsupported claim, missing evidence, unclear refusal, or presentation issue. A thumbs-up count would have been less actionable.

The record made generated text product state without treating it as authoritative document content. An answer could be bookmarked and reviewed. It could not modify the source corpus.

This distinction prepared V0M3. K81R generated ephemeral-but-recorded synthesis from evidence. V0M3 would let model output enter a longer authored document workflow, which required explicit proposals, approvals, tool permissions, and revision ownership.

Index freshness and removal

A source revision did not become searchable until parsing, chunking, lexical indexing, and embedding completed. The first interface updated the document page immediately and left the search index silently behind. A question could therefore miss a correction visible elsewhere in K81R.

I added per-document index status and a corpus generation. Search results declared the generation they used. An answer record named it permanently. The document page showed when its current revision was still processing and linked the latest indexed revision.

Index builds wrote to a new generation and activated it only after completeness checks. Incremental updates used content digests to reuse unchanged chunks, while a reconciliation job compared source revisions with both indexes and lineage records.

Deletion followed the same explicit path. Removing a document created tombstones for lexical and vector entries, invalidated caches, and retained only the minimal lineage required by already saved answer records according to the personal archive policy. An answer whose source was no longer available displayed that fact without substituting a different revision.

Embedding providers received only the chunk text required for the operation and no broader metadata than necessary. The project supported a local embedding path for sensitive subsets. Provider and model choice were part of the index configuration, not an invisible environment default.

If one index failed during an update, the active generation remained unchanged. Search could also enter a declared lexical-only degraded mode rather than combining current lexical results with stale vectors and presenting them as one coherent generation.

This lifecycle work made retrieval infrastructure part of the evidence contract. A citation to revision 12 means little if the system cannot state whether revision 12 was actually present in the index used to select it.

Designing refusal as a useful page

The earliest refusal was one sentence: “I don’t have enough information.” It was safe and unhelpful.

The final refusal view named the part of the question the collection could support, the relationship or fact still missing, the strongest available passages, and possible narrower searches. It did not instruct the user to trust a vague confidence score.

For contradictory sources, the page showed both revisions and their effective dates. For a temporal ambiguity, it asked which period mattered. For an out-of-scope question, it explained the corpus boundary without revealing inaccessible document titles.

The view allowed a person to save the evidence set and add a private note about what source would resolve the gap. If that source entered the corpus later, the saved question could be reevaluated as a new answer revision rather than silently updating the old one.

Refusal evaluation included usefulness as well as frequency. A refusal that avoided error but concealed the relevant evidence failed the product goal. K81R should make uncertainty navigable, not merely cautious.

Next.js and the server boundary

By late 2023, the Next.js App Router had reached production stability. K81R used server-rendered routes and Server Components for document and answer views, with client components only for local query interaction and evidence selection.

This moved data access close to the server without pretending frontend responsibility had disappeared. The page still needed a coherent loading hierarchy, accessible citation interactions, and stable URLs.

Search filters and answer IDs lived in shareable routes. The initial result rendered on the server. Expanding a citation used a small interactive component and never required shipping corpus access credentials or full documents to the browser.

Streaming applied at user-meaningful boundaries. Search results arrived as a stable unit. A generated synthesis could stream after the evidence set was ready, but claim citations appeared only when their structured association had been validated.

The first implementation streamed raw model tokens directly into the article region. Citations arrived late and text reflowed unpredictably. Buffering by structured claim produced slightly less spectacle and a more reviewable result.

The architecture demonstrated a lesson from P6X4: “frontend” names a responsibility, not a mandatory execution location. Server and client boundaries followed data authority and interaction needs.

The answers I should have refused earlier

The embeddings-only prototype failed exact identifiers and overvalued semantically similar passages. Hybrid retrieval corrected the imbalance.

Fixed-size chunking separated claims from context. Structure-aware chunks and lineage made citations meaningful.

The first prompt required an answer every time. Answerability and a designed refusal path stopped the system from turning missing evidence into confident prose.

Citations were initially attached per paragraph and often did not support every sentence. Claim-level association and verification narrowed the output.

Raw token streaming made the answer feel alive and citations feel appended. Structured claim streaming made evidence part of the unit displayed.

I also compared changes through one aggregate score. Stage-specific metrics and per-case review exposed tradeoffs hidden by the average.

Search became an evidence product

K81R was the first project in the journey where probabilistic generation became a product component. The surrounding architecture did most of the trust work: versioned ingestion, hybrid retrieval, evidence sets, answerability, claim verification, immutable answer records, and bounded authority.

Its strongest result was separating the ability to produce fluent text from the right to make a claim. Retrieval and citation were necessary but insufficient. The system needed a first-class decision to abstain and a way to show exactly which revision supported each sentence.

P6X4’s coherent document model made lineage possible. Q2F8’s catalogue discipline shaped versioned schemas and non-collection. W93H’s hypothesis model shaped the distinction between evidence and conclusion. Z29C’s receipts shaped immutable answer records.

V0M3 would build on all of this and take on a harder problem: model output that participates in a document’s actual lifecycle. The generated text could no longer remain a disposable answer. It would need status, provenance, ownership, human revision, tool policy, and a path to publication.

What retrieval could not establish

K81R used a bounded personal corpus and local evaluation set. It could not establish general factual accuracy or defend every adversarial prompt. Relevant evidence could be missing, retrieval could fail, and verification could make mistakes.

The refusal threshold encoded a product tradeoff and required continued review. A citation proved lineage to a passage, not the ultimate truth of that passage. K81R made those limitations visible and constrained what an answer could change.

Technology: structured Markdown ingestion, revision and lineage records, lexical BM25-style search, vector embeddings, reciprocal-rank fusion, bounded reranking, answerability evaluation, structured claim citations, Next.js App Router, React Server Components, and a versioned local evaluation harness.