Context engineering basics

What is parametric versus non-parametric knowledge?

Parametric knowledge is stored in the weights. Non-parametric knowledge sits in an external store that is retrieved at inference time.

Does inserting a passage into the prompt change the saved weights?

No. It only changes the activations for that call.

What is a knowledge conflict?

A retrieved passage says something different from what the model learned in training. Studies measure which source the output follows.

What is sparse retrieval?

Matching on exact terms with high-dimensional mostly-zero vectors, such as TF-IDF or BM25, served from an inverted index.

What does BM25 score?

Term overlap between query and document, weighted by how rare each term is, with term frequency saturating and a length normalization.

What is an inverted index?

A map from each term to the list of documents that contain it.

What is dense retrieval?

Encoding queries and documents as low-dimensional vectors and ranking by vector similarity.

At query time, what comparison does a bi-encoder make?

It encodes the query once and compares that vector with document vectors computed in advance, usually by dot product or cosine.

What is a cross-encoder?

A model that reads query and document together and outputs one relevance score. It is more accurate but too slow to score a whole corpus.

What is late interaction, as in ColBERT?

Keep one vector per token, and score by summing each query token's best match among the document's token vectors.

What are CLS pooling and mean pooling?

Two ways to turn token states into one vector: take one designated token's state, or average all token states.

When do cosine similarity and dot product give the same ranking?

When all vectors are normalized to unit length.

What is the InfoNCE contrastive loss?

A softmax loss that pushes a query's score for its positive document above its scores for a set of negatives.

What are in-batch negatives?

Using the other examples' positives in the same batch as negatives, which is cheap and makes larger batches help.

What are hard negatives?

Non-relevant documents that look relevant, often mined with BM25 or an earlier model. They make contrastive training sharper.

What is the false-negative problem in mined negatives?

Some mined negatives are actually relevant but unlabeled, so training pushes the model away from correct answers.

What is approximate nearest-neighbor search?

Finding vectors close to the query without comparing against every vector, trading a little recall for large speedups.

What is HNSW?

A layered graph index for nearest-neighbor search where queries walk from coarse upper layers to dense lower ones.

What is product quantization?

Compressing vectors by splitting them into sub-vectors and storing each as a codebook index, which saves memory but adds error.

What are Matryoshka embeddings?

Embeddings trained so a prefix of the vector, such as the first 256 of 1024 dimensions, is still a usable embedding.

What is recall@k?

The fraction of relevant documents that appear in the top k results.

What is precision@k?

The fraction of the top k results that are relevant.

What is mean reciprocal rank?

The average over queries of 1 divided by the rank of the first relevant result.

What does nDCG measure?

Ranking quality with graded relevance, discounting gains lower in the list and normalizing by the ideal ordering.

What is zero-shot retrieval evaluation, as in BEIR?

Testing a retriever on datasets and domains it was not trained on, to measure out-of-domain transfer.

What is a two-stage retrieve-then-rerank pipeline?

A fast retriever picks candidates, then a slower, more accurate model reorders them.

What is reciprocal rank fusion?

Combining several rankings by summing 1/(k + rank) for each document, which needs no score calibration.

Why combine sparse and dense retrieval?

Sparse handles exact names and rare terms well, and dense handles paraphrase. Fusing them usually beats either alone.

Where can metadata filters act in a retriever?

Before the search, restricting candidates, or after it, removing results. Post-filtering can leave fewer than k results.

What is chunking?

Splitting documents into passages that are embedded and retrieved one at a time.

Why use overlapping chunks?

So a fact that sits across a boundary still appears whole in at least one chunk.

What is the trade-off in chunk size?

Small chunks are precise but lose context. Large chunks keep context but dilute the embedding and use more of the window.

What is query expansion?

Adding related terms or rewrites to the query to raise recall.

What is HyDE?

Have a model write a hypothetical answer and retrieve with its embedding instead of the raw question's.

What is asymmetric retrieval?

Short queries matched against long documents. Some embedding models use different prefixes or encoders for each side.

What is faithfulness in RAG evaluation?

Whether each claim in the output is supported by the retrieved text, separate from whether it is true.

The gold passage is in the top k but the answer states an unsupported detail. Which stage failed?

Generation. Retrieval succeeded, and the model added something the passage does not support.

What are citation precision and citation recall?

Precision: cited passages actually support the claim. Recall: every claim that needs support has a citation.

What is a prompt template?

Fixed text with slots, where the question, passages, and examples are filled in for each input.

What is few-shot prompting?

Putting a handful of solved examples in the prompt before the new input.

Why report results over several example orders?

Few-shot accuracy can change a lot when the same examples are reordered, so one order may be lucky.

What is the lost-in-the-middle effect?

Models often use information at the start or end of a long context better than information in the middle.

What does a needle-in-a-haystack test measure?

Whether a model can find one planted fact at different depths of a long filler context. Passing it does not show multi-step reasoning over long text.

What is effective context length?

The length at which a model still uses information reliably, which can be well short of the maximum it accepts.

When do a retrieved passage's keys and values exist in the KV cache?

Only after the passage is placed in the prompt and the model runs prefill over it.

What is prefill?

The forward pass over all prompt tokens at once, which fills the KV cache before generation starts.

What is a distractor passage?

Retrieved text that looks relevant but does not answer the question. Studies add them to test robustness.

What is an oracle retrieval baseline?

Giving the model the gold passage directly, which sets an upper bound for what better retrieval could gain.

What is a closed-book baseline?

Running the same model with no retrieved text, to show how much retrieval actually adds.