Context engineering basics

Question 1

What is the context window on one model call?

Accepted Answer

The tokens this call can attend to. The length is finite for the model serving the call.

Question 2

A call includes a system message, a user message, a tool result, and a retrieved passage. How do those four relate to the window?

Accepted Answer

They are all tokens in the same window. A role label marks a span. It does not assign that span its own model.

Question 3

What do few-shot examples change before the model answers?

Accepted Answer

They add tokens to the prompt. The weights stay as they were.

Question 4

Later text fills the window, and the serving policy drops the opening instructions. What can this call attend to from those instructions?

Accepted Answer

Nothing from the dropped span. Tokens removed before the forward pass are absent.

Question 5

Which pieces does a token budget for one call have to add up?

Accepted Answer

System text, history, retrieved text, and the room left for the answer.

Question 6

A retrieved passage is placed in the prompt and the model reads it. What persistent memory does that create?

Accepted Answer

None by itself. The passage is tokens in this window, and a later call does not gain a store that can reopen it.

Question 7

An embedding model maps a passage to a vector, and a vector index returns the nearest neighbors. What does a high similarity establish?

Accepted Answer

That the vectors were close. It does not establish that the passage answers the question.

Question 8

A sentence says the cap is 30 days and the fee is $12. The chunk cut falls between those clauses, and the returned neighbor contains only the fee clause. What did this retrieval put in reach of the model?

Accepted Answer

The fee clause inside the returned chunk. The 30-day clause stays out unless its chunk is inserted too.

Question 9

Why do neighboring chunks share an overlapping span?

Accepted Answer

So a sentence on the cut is kept whole in a chunk. That shared span is stored more than once, so its tokens repeat.

Question 10

In a usual retrieval stack, what do a bi-encoder and a cross-encoder each do?

Accepted Answer

The bi-encoder embeds the query and the passage separately for the first-stage index. The cross-encoder scores a query-passage pair together and is the usual reranker.

Question 11

When are a dot product and cosine similarity the same score for two vectors?

Accepted Answer

When both vectors have length 1. Cosine divides the dot product by the product of the lengths, which is 1 for those vectors.

Question 12

You insert a long run of retrieved passages ahead of a short question, and the window is full. What can get pushed out?

Accepted Answer

The question, along with anything else that no longer fits. Inserted passages spend the same kind of tokens the question spends.

Question 13

A relevant sentence sits at the start of a long prompt, at the end, or deep in the middle. What is a fair expectation?

Accepted Answer

Models often use the start and the end more reliably than the middle. That is a known tendency, not a fixed rate for every model.

Question 14

The answer cites paragraph 4 for a price. The prompt contained only paragraph 2. How good is that citation?

Accepted Answer

Only as good as the passage that was inserted. Paragraph 4 was not in the window, so the citation does not show it was read.

Question 15

Older turns are replaced by a summary before the next call. A spending cap appeared only in those turns, and the summary never states it. What happened to the cap?

Accepted Answer

It is absent from this window. Compaction can drop a constraint that lived only in the removed turns.

Question 16

A prefix was cached, and then one token in that prefix changes. What happens to reuse of the following tokens?

Accepted Answer

The cached states from the edited token onward no longer match the new sequence. A stable prefix is what can be reused.

Question 17

What does the KV cache hold during generation?

Accepted Answer

Keys and values for tokens already in the window, so later steps do not recompute them. It is not a vector database and does not fetch new documents.

Question 18

You lengthen the prompt, so more positions are processed. What does that do to the KV cache and to the model's trained context length?

Accepted Answer

The cache grows with the positions already processed. That growth does not, by itself, raise the context length the model was trained for.

Question 19

Two users share a long instruction block and then send different requests. Which part can prompt caching reuse?

Accepted Answer

The shared prefix. A suffix that differs for each user is not the reused part.

Question 20

Which count should decide whether a prompt fits the budget?

Accepted Answer

The token count from the tokenizer of the model that will run. A word count is a different unit.

Question 21

How does putting every chunk of a corpus into the prompt differ from inserting a few neighbors?

Accepted Answer

Stuffing inserts every chunk of the corpus. Selecting neighbors inserts a handful of hits. The prompt then holds different text.

Question 22

A vector search applies a metadata filter on collection and year. What does the filter change?

Accepted Answer

Which stored vectors are eligible to be neighbors. It leaves the embeddings themselves as they were.

Question 23

You rebuild queries with a new embedding model and leave the index on vectors from the old one. What is wrong with the neighbors?

Accepted Answer

The new queries sit in a different vector space. An index built with the old model does not match them.

Question 24

A prompt includes the identifier get-user-id several times. How should that string be charged against the budget?

Accepted Answer

By the tokens this model's tokenizer emits for it. A hyphenated identifier can split into several tokens.

Question 25

Call 1 contained a paragraph the user cares about. Call 2 sends only a new question. What of that paragraph is in call 2's window?

Accepted Answer

Only what call 2 actually includes. The model does not keep call 1's tokens on its own.

Question 26

During decoding, the model has already produced 200 answer tokens. Where do those tokens sit relative to the context limit?

Accepted Answer

Inside the same sequence as the prompt. They occupy positions in the window.

Question 27

The limit covers prompt plus completion, and the prompt already uses every position. How much of an answer can this call generate?

Accepted Answer

None, until some prompt tokens are removed. There is no spare position left for a new token.

Question 28

A chat template wraps each message with role markers before the model runs. Do those marker tokens belong in the budget?

Accepted Answer

Yes. Tokens the template adds are part of the sequence the model receives.

Question 29

Two tokenizers encode one paragraph. Why might the budgets disagree?

Accepted Answer

Each tokenizer can split the same string differently, so the token counts can differ. Use the tokenizer for the model that will run.

Question 30

Someone sizes a prompt as four characters per token. What is that figure?

Accepted Answer

A rough estimate. The budget is the count returned by the model's tokenizer.

Question 31

A rare product name is absent from the tokenizer vocabulary. How can that affect the token count?

Accepted Answer

The name can be split into several subword tokens. One word is then more than one token.

Question 32

The prompt contains a long integer. What is a sound way to budget it?

Accepted Answer

Run it through the model's tokenizer. A number can be several tokens.

Question 33

You change spaces, punctuation, or capitalization in a sentence and leave the words' order alone. What can happen to the token count?

Accepted Answer

It can change. Those characters can alter how the tokenizer splits the string.

Question 34

These figures are already counts from the model's tokenizer. The shared limit is 8000 tokens. System text is 500, history is 2000, and 1000 are reserved for the answer. How many tokens can retrieved text still use?

Accepted Answer

4500. That is the limit minus the system text, the history, and the reserved answer.

Question 35

A file path and a long URL appear once each, with no spaces. What is a realistic token cost?

Accepted Answer

Many tokens. A path or URL can split even when a person reads it as one string.

Question 36

A manual sits in a database. This call's prompt never includes it. Can attention on this call read that manual?

Accepted Answer

No. Attention covers tokens present in the window. A document left outside the prompt is outside this call.

Question 37

One fact was learned in training. Another exists only in a private note that was not inserted. Which fact can this call use from those sources?

Accepted Answer

The fact in the weights can be used without retrieval. The private note cannot, until its text is placed in the window.

Question 38

A passage in the prompt contradicts a fact the weights learned. What remains available while the model generates?

Accepted Answer

Both influences. The passage does not erase parametric memory, and the weights do not delete the passage.

Question 39

The prompt says to answer from the inserted passage when it conflicts with prior knowledge. What does that sentence do to the weights?

Accepted Answer

Nothing. It is another instruction in the window. It does not switch parametric memory off by editing weights.

Question 40

Retrieval runs and inserts no passage. What source text from the corpus is in this window?

Accepted Answer

None from that retrieval. The call has the question and the rest of the prompt, and it lacks the passage that was not inserted.

Question 41

You raise the neighbor count and insert every returned chunk. What changes in the window?

Accepted Answer

The extra chunks add their tokens, including chunks that may be irrelevant. The window does not grow to make them free.

Question 42

You raise the minimum similarity until the search returns no chunks, even though a useful chunk exists below the new line. What did the cutoff do?

Accepted Answer

It dropped every neighbor that missed the line, including that useful chunk. A high cutoff can return nothing.

Question 43

Some embedding models expect one short prefix on questions and a different prefix on passages. You embed a question with the passage prefix. What can change?

Accepted Answer

The query vector. On those models the prefix is part of the embedded string, so the neighbors can change.

Question 44

Passages were lowercased and stripped of boilerplate before they were embedded. The query is embedded raw. Why can the neighbors be off?

Accepted Answer

The query vector was built from different text processing than the passages. The same cleanup should be applied before the query is embedded.

Question 45

The index stores 768-dimensional vectors. The query vector has 1024 dimensions. What can the index do with it?

Accepted Answer

It cannot score that vector against this index. The widths differ, so they are not comparable coordinates.

Question 46

You want an old index to serve a new embedding model. What is a valid way to line up the spaces?

Accepted Answer

Re-embed the documents with the new model and query with that same model. There is no standard conversion between two models' vectors.

Question 47

An approximate nearest-neighbor index returns five passages. What can be true of the exact nearest vector?

Accepted Answer

It can be missing from those five. Approximate search is allowed to skip vectors that an exact scan would score.

Question 48

A two-stage stack reranks the first-stage shortlist with a cross-encoder. A relevant passage never entered that shortlist. What can this rerank do with it?

Accepted Answer

Nothing on this search. The reranker scores pairs from the shortlist, and this passage is not among them.

Question 49

A reranker gives a passage a very high score for this query. What does that score mean?

Accepted Answer

The scorer rated that query-passage pair highly. It is not a proof that the passage is true.

Question 50

When can bi-encoder passage vectors be computed?

Accepted Answer

Before any question arrives. The passage encoder does not need the query, so the vectors can be stored ahead of time.

Question 51

Why is a standard cross-encoder a poor index over a large corpus?

Accepted Answer

Its score depends on the query and the passage together, so it does not yield a passage-only vector you can store once and search for every later query.

Question 52

Two chunks overlap, and both rank in the neighbor list. What can inserting both of them do to the budget?

Accepted Answer

It can spend tokens twice on the shared span and take two slots that might have held another document.

Question 53

You can index a short span around one fact or a long section that includes it. What is the tradeoff when the chosen chunk is inserted whole?

Accepted Answer

The short span can match a narrow fact and omit nearby sentences. The long section carries more surrounding text and spends more of the window.

Question 54

A splitter cuts at a fixed token count and lands in the middle of a sentence. What do the two chunks contain?

Accepted Answer

Two pieces of that sentence. Neither piece holds the whole sentence just because the cut was on a token boundary.

Question 55

A splitter tries paragraph breaks, then line breaks, then spaces. What is that order for?

Accepted Answer

To cut on a coarse boundary when one exists. A piece that is still longer than the chunk size can still be cut inside a sentence.

Question 56

A table is chunked so that a data row is inserted without the header row. What can the model fail to see?

Accepted Answer

What each cell means. The header is in the window only if that header text is part of the inserted chunk.

Question 57

One chunk says "this limit is 30" and does not name the thing. The noun is in the previous chunk, which was not retrieved. What did the model receive?

Accepted Answer

The pronoun sentence in the retrieved chunk. The naming sentence is absent unless its chunk is inserted too.

Question 58

The indexed string for a chunk is the section title plus the body, and that same string is what you insert. What does the title do?

Accepted Answer

It changes the text that was embedded, and it is present in the window because you inserted that string.

Question 59

You embed document titles and store only those titles. A search returns a title. What text was the neighbor actually about?

Accepted Answer

The title string you embedded. A body you never indexed is not what this search compared.

Question 60

The budget can hold only part of a long file. You fill it from byte zero. How does that differ from inserting the top-ranked chunks until the budget is full?

Accepted Answer

The front of the file can be irrelevant while a later relevant section is left out. Ranked chunks are the neighbors you chose, trimmed to the budget.

Question 61

You embed three wordings of one question, merge the neighbor lists, and insert the passages you keep. What enlarges?

Accepted Answer

The candidate set you may choose from. The window stays the same size, and every passage you insert still spends tokens.

Question 62

After retrieval scores a passage, you place that passage at the end of the prompt instead of the middle. What happens to its similarity score?

Accepted Answer

Nothing. The score was computed before placement. The slot you choose spends tokens there and leaves the score as it was.

Question 63

A whole report fits in the window and is pasted in. What does fitting guarantee about how the report is used?

Accepted Answer

That its tokens are present for attention. It does not guarantee that every sentence is used equally.

Question 64

A probe hides one fact at a single position in a long prompt, and the model answers it. What does that one success show?

Accepted Answer

That the model used the fact at that position on that probe. It does not show that every position works for every fact.