Prompt engineering

What should a task prompt specify?

The task, relevant inputs, constraints, and expected output. For a support classifier, define allowed labels and how to handle messages that fit none of them.

How do you turn “write a good summary” into a testable instruction?

Specify the audience, length, required facts, and output format. For example: write three bullets for an operations lead, covering the outage cause, customer impact, and next action.

Why separate application instructions from user input?

Application instructions define stable behavior; user input supplies the current request or data. Use the API’s supported message roles and precedence rules rather than concatenating all text into one undifferentiated string.

What do delimiters contribute to a prompt?

Headings, tags, or fences mark boundaries between instructions, examples, and source material. They improve readability but do not make untrusted content safe or enforce access controls.

What is zero-shot prompting?

Requesting a task without including worked examples. A ticket-routing prompt that defines labels and asks for one label is zero-shot if it includes no sample tickets with answers.

What is few-shot prompting?

Including a small set of input-output demonstrations in the context. The model can infer the intended pattern from them without a training update.

Which examples belong in a few-shot classifier prompt?

Use representative inputs, confusing class boundaries, and the exact desired output format. Include relevant edge cases rather than many nearly identical easy examples.

A prompt says “return one label,” but its examples include explanations. What should you fix?

Make the examples obey the requested output contract. Conflicting demonstrations leave the expected behavior ambiguous; retest after aligning both instructions and examples.

Why can the order and balance of few-shot examples matter?

The model may favor patterns or labels that dominate the context or appear near the query. Test alternative orders and class balances on held-out cases before selecting a prompt.

Does asking a model to act as an expert establish factual accuracy?

No. A role can guide tone, scope, and vocabulary, but it is not evidence of expertise. Verify factual claims against reliable sources or task-specific checks.

How can a negative instruction be made more actionable?

State the desired alternative. Instead of only saying “do not guess missing dates,” specify “return null for any date absent from the source.”

When should a prompt ask the model to clarify a request?

When missing information materially changes the correct result. Define which fields are required and when an explicit assumption or an unknown value is acceptable.

When is splitting a task into stages useful?

When intermediate outputs can be checked independently. Extract invoice fields, validate totals with code, then draft a summary. Each stage has a narrower contract, though extra calls add latency and failure points.

Should every prompt demand a long step-by-step explanation?

No. The value depends on the task and model. Ask for the result and any useful checks or concise justification; a long explanation can add cost and can still rationalize an incorrect answer.

Why is asking a model to check its own answer insufficient validation?

The model may repeat the same mistaken assumptions. Use independent evidence, executable tests, or human review where appropriate instead of treating self-approval as proof.

How does a request for JSON differ from schema-constrained generation?

A text instruction asks the model to emit JSON. A supported constrained-output feature restricts generation to an allowed structure. Neither alone proves that the field values are factually correct.

A model returns valid JSON with an impossible delivery date. What failed?

Semantic validation. Parsing and schema checks confirm structure and types; application checks must verify dates, ranges, cross-field consistency, and any business rules.

How should an extraction prompt represent absent information?

Define a consistent missing-value convention, such as null, and distinguish it from an empty string or a confirmed negative. Do not force the model to invent a value to satisfy a required field.

What does grounding an answer in supplied documents mean?

Basing its factual claims on the provided evidence. Specify the allowed sources and what to do when evidence is missing or contradictory, then check whether the answer actually follows them.

Does a citation guarantee that an answer is supported?

No. A source can exist without supporting the claim. Check that each cited passage entails the associated statement and that the model has not invented a source identifier.

A document assistant misses a fact absent from its retrieved passages. Is rewriting the answer prompt enough?

Usually not. Inspect retrieval, filtering, chunking, and context assembly first. The answer prompt cannot reliably recover evidence that never reached the model.

Why avoid filling the context window with every available document?

Irrelevant or repeated material consumes tokens and can obscure useful evidence. Select task-relevant passages, preserve their provenance, and reserve room for the response.

What should you inspect when a long conversation stops following earlier requirements?

Inspect the actual assembled request for truncation or lossy summaries. Preserve essential instructions and unresolved decisions, and test behavior near the application’s context limit.

What is indirect prompt injection?

An attacker places instructions in material the model reads, such as a retrieved page or tool response, hoping the model will treat that data as authority and depart from its intended task.

Why is “ignore malicious instructions” not a complete prompt-injection defense?

A model can still confuse instructions with untrusted data. Enforce tool permissions, data access, and consequential-action checks in application code, and test adversarial inputs alongside prompt defenses.

What belongs in a tool description used by a model?

The tool’s purpose, when to use it, argument meanings, constraints, and important side effects. Clear boundaries help the model choose correctly; the application still validates arguments and authorization.

Does setting temperature to zero guarantee identical, correct answers?

No. It reduces sampling randomness where supported, but serving behavior can still vary, and deterministic output can still be wrong. Measure reliability with repeated evaluations.

What belongs in a prompt evaluation set?

Representative normal requests, boundary cases, missing inputs, and known failures, with explicit scoring criteria. Keep a held-out set separate from examples used to tune the prompt.

How should you evaluate a ticket-routing prompt?

Measure per-class precision and recall, invalid-label rate, and performance on ambiguous tickets. Include latency and cost; overall accuracy alone may hide failures on a rare but important class.

How can you tell whether adding examples actually improves a prompt?

Compare prompts with and without the examples on the same held-out cases while holding the model and decoding settings fixed. Repeat runs when variability matters and inspect which errors changed.

What should be versioned with a production prompt?

The template, examples, model identifier, generation settings, tool schemas, and context-assembly logic. Save evaluation results so a regression can be traced to a concrete change and a prior version restored.

Why use a prompt template with explicit variables?

It separates reusable task instructions from request-specific values. Validate required variables before rendering so a missing customer message or document does not silently become an incomplete request.

How does context engineering extend prompt engineering?

It considers the full information supplied to a model: instructions, retrieved passages, conversation history, tool results, and state. The challenge includes selecting and updating that information across turns.

Two application instructions require incompatible output formats. How should you resolve this?

Remove the conflict and define one output contract or an explicit condition for each format. Adding more emphasis to both instructions does not make them compatible.

How should a document-answering prompt handle conflicting sources?

Define a source precedence rule when the application has one, such as using the latest approved revision. Otherwise, report the disagreement with source references instead of silently choosing a convenient claim.

What is an abstention policy in a prompt?

A rule for declining to provide an unsupported answer, such as returning “insufficient evidence” when no supplied passage answers the question. Evaluate both unsupported answers and unnecessary abstentions.

Why should a model’s stated confidence not be treated as a calibrated probability?

A generated confidence score is another model output and may not match observed correctness. Check calibration against labeled examples before using such scores for routing or acceptance decisions.

A generated JSON object ends halfway through a string. What should you inspect?

Check the response termination reason and output-token limit. Allow enough room for the expected object, reduce the requested payload if necessary, and treat truncated output as incomplete rather than trying to accept it as valid data.

What should a bounded output-repair loop do?

Validate the response, send specific validation errors back for correction, and stop after a fixed retry budget. Preserve the original evidence and return a controlled failure if the result still violates the contract.

Why rerun prompt evaluations when changing models?

Models can interpret the same wording and examples differently. Compare task quality, format compliance, tool selection, latency, and cost before promoting the new model with the existing prompt.

What does overfitting a prompt to an evaluation set look like?

Repeated edits improve the familiar cases while unseen requests still fail. Keep a separate final test set and add diverse examples from real failure patterns without leaking test answers into the prompt.

What makes a model-based grading rubric useful?

Concrete criteria, clear score boundaries, and examples of acceptable and unacceptable outputs. Compare the grader with human judgments and deterministic checks before relying on its scores.

How can answer order bias a model-based comparison?

A judge may favor the first or second response regardless of quality. Swap the order, hide model identities, and inspect disagreements to distinguish position preference from a consistent quality difference.

How should you evaluate a prompt used in several languages?

Test representative inputs in each language, including local date formats, mixed-language text, and domain terms. An English evaluation alone does not establish reliable behavior for other languages.

An extraction task sees “03/04/2026.” What must the prompt clarify?

The expected locale or date convention and the required output format. If the source leaves the date ambiguous, preserve that uncertainty instead of silently assuming month-first or day-first notation.

When should a prompt route arithmetic to an executable tool?

When exact computation matters and an appropriate tool is available. Have the model identify the inputs and operation, validate those inputs, and use the computed result rather than relying only on generated arithmetic.

How should a tool-using prompt handle a failed lookup?

Distinguish a tool error from a successful search with no results. Specify permitted retries or fallback sources, and report unavailable evidence rather than presenting the failed lookup as confirmation that nothing exists.

What should a handoff summary preserve in a multi-step workflow?

The objective, constraints, verified findings, source references, completed actions, and unresolved work. Label assumptions explicitly so the next step does not treat a tentative inference as an established fact.

How should you investigate a prompt that works in a playground but fails in production?

Compare the actual rendered messages, model settings, tool schemas, retrieved context, and response limits. Look for missing variables or changed preprocessing before rewriting instructions that may not be reaching the model.