Glossary — luciddocs

Agent retrieval

Instead of a single fixed search pass, the AI receives a search tool it can call multiple times per question. It issues a query, reads the results, and can try different phrasings or follow-up searches before answering. In these experiments, the agent could call search() up to four times. This is different from standard RAG — in standard RAG the retrieval step is fixed and the AI has no control over it. Agent retrieval is the most effective approach tested, but it trades latency and cost for accuracy: undefined jargon required 3–4 search calls per question.

Coreference

When a pronoun or demonstrative ("it", "this", "they") refers back to a noun mentioned earlier in the text. Coreference is a problem for RAG because the AI may retrieve a chunk that contains "it must be included in the header" without the surrounding context that establishes what "it" refers to. In these experiments, coreference degradation replaced specific nouns ("the API key", "the webhook secret") with ambiguous pronouns across a document with two distinct credentials — so "it" could mean either one.

Canary

A unique phrase embedded in one specific spot in the document — like "key cycling." If the AI's answer includes the canary phrase, we know it actually retrieved and read the right section, rather than hallucinating an answer from its training data.

Chunk

A piece of a document — typically a paragraph or heading section — that gets embedded and stored separately. RAG retrieves chunks, not whole documents. This means a fact buried in the middle of a long page might never reach the AI if its chunk scores poorly against the query.

Context mode

In these experiments: the entire document is placed directly into the AI's prompt. No search, no chunking. The AI has everything and just has to understand it. This is an upper-bound test — if the AI fails in context mode, it's a writing problem, not a retrieval problem.

Contextual retrieval

Before embedding each chunk, ask an AI to write one sentence describing how that chunk fits into the broader document, then prepend that sentence to the chunk. This gives the embedding model document-level context it would otherwise lack.

Control

The clean, unmodified version of a document used as a baseline. Every degraded version is compared against the control. If the control scores 97% and a degraded version scores 53%, the writing problem cost 44 percentage points.

Cosine similarity

The most common way to compare two embeddings. It measures the angle between two vectors — a score of 1.0 means identical meaning, 0 means unrelated. RAG retrieval picks the chunks with the highest cosine similarity to the query.

Degradation type

A specific writing problem introduced into the document in isolation. Each degraded version changes exactly one thing from the control — so any accuracy drop can be attributed to that specific problem, not a combination.

Embedding

A way of turning text into a list of numbers that captures its meaning. Texts with similar meanings get similar numbers, so a search system can find relevant chunks by comparing numbers rather than matching exact words. This is why a question about "rotating credentials" can find a chunk that talks about "cycling your API key" — the numbers are close even though the words differ.

HyDE Hypothetical Document Embeddings

A retrieval trick: instead of embedding the user's question, first ask the AI to write a short hypothetical answer, then embed that. The idea is that an answer-shaped piece of text finds answer-shaped chunks better than a question-shaped one does.

Parent-document retrieval

A two-level chunking strategy: use small paragraph chunks for search (more precise matching), but when a chunk is retrieved, pass the full parent section to the AI (more context). Small chunks for finding; large chunks for reading.

Percentage points (pp)

The raw difference between two percentages. A drop from 97% to 53% is a 44pp drop, not a 44% drop. (A 44% drop from 97% would be 54%. They're different things.)

RAG Retrieval-Augmented Generation

A way of giving an AI access to a document without stuffing the whole thing into the prompt. The document is split into chunks, stored with embeddings, and when a user asks a question, the system finds the most relevant chunks and hands only those to the AI. Most AI-powered documentation search tools (like Inkeep, Kapa, or custom chatbots) work this way.

RAG mode

In these experiments: the document is chunked, embedded, and retrieved. The AI only sees the top-3 most relevant chunks, not the full document. This is closer to how real AI doc tools work in production.

Rubric score

An AI judge (claude-haiku) rates each answer on four dimensions: factual accuracy, specificity, completeness, and hallucination — 0–2 each, 8 total. This catches failures that fact-matching misses, like an answer that echoes vague source language and sounds reasonable but contains no useful information.