Harness engineering 2026

How can the same model perform differently in two harnesses?

The harness determines what evidence the model sees, which actions it can take, and how failures are handled. Different tool interfaces, context policies, or stopping rules can change task success without changing model weights.

How does harness engineering differ from prompt engineering?

Prompt engineering changes instructions and examples. Harness engineering also changes executable behavior, such as argument validation, tool dispatch, checkpoint storage, and timeouts. A prompt cannot reliably enforce an operating-system permission.

How does context engineering fit within a harness?

It decides what information reaches each model call. The harness implements that policy through retrieval, history selection, tool-result processing, and compaction, while also managing execution and persistent state.

What happens in a basic agent execution loop?

Build the input, call the model, inspect its response, validate and execute requested tools, then return tool results to the model. Stop on completion, cancellation, a resource limit, or a condition requiring human input.

Why represent an agent run with explicit states?

States such as running, waiting for a tool, awaiting approval, failed, and completed make transitions testable. They prevent a timeout or interrupted stream from being mistaken for successful completion.

What should define task completion in a harness?

Observable acceptance criteria, such as a saved artifact, passing checks, or confirmed external state. A confident final message is evidence of what the model claimed, not proof that the requested action succeeded.

Why distinguish model completion from a budget stop?

They mean different things to the user and recovery logic. A task cut off by a token or time limit needs an incomplete status and saved progress, not an unqualified success response.

What should a harness do with an agent's plan?

Store actionable steps and update their status from actual execution. Allow revision when evidence changes. A plan helps coordinate work, but marking every step complete cannot replace checking the final result.

How can a harness detect an unproductive loop?

Track repeated tool calls, identical errors, and unchanged artifacts across steps. After a bounded number of failed attempts, require a different approach or surface the blocker instead of spending the remaining budget on repetition.

What belongs in a tool contract?

A precise purpose, argument schema, output shape, side effects, and expected error conditions. The agent should be able to distinguish a search that found nothing from a search that failed.

How should you choose the granularity of agent tools?

Expose operations that match useful task steps. Very small tools create unnecessary calls; overly broad tools hide decisions and failures. Compare real task traces to find where a clearer operation reduces mistakes.

Where should tool arguments be validated?

In application code before execution. Check types, required fields, bounds, and resource identifiers. Model-generated arguments are proposals; even schema-conforming arguments may refer to an unauthorized or nonexistent resource.

Why return structured tool results?

Named fields for status, records, identifiers, and errors let the harness process results consistently. Include enough human-readable detail for the model to reason about the outcome without guessing what an unlabeled string means.

How should a harness handle a tool result too large for context?

Store the complete output as an artifact and return a concise summary with a reference and retrieval options. Preserve error details and relevant excerpts so truncation does not hide the reason an operation failed.

When is dynamic tool discovery useful?

When many integrations exist but only a few matter to a task. Load relevant tool descriptions on demand, then validate their availability. Discovery should not grant permissions merely because a tool was found.

Why distinguish validation, transient, and permanent tool errors?

Each needs a different response. Invalid arguments need correction, a temporary outage may justify retrying, and denied access needs another authorized route or user action. Treating every error as retryable wastes work.

What makes a retry policy safe and useful?

Bound attempts and elapsed time, apply backoff for transient failures, and confirm whether the operation can be repeated. Retrying a read is different from retrying a charge whose result was lost.

How does an idempotency key protect a retried action?

The receiving service associates repeated requests with one logical operation and reuses its recorded result. The key must survive retries and restarts. Adding a new key on each attempt defeats deduplication.

A write times out after submission. What should the harness do?

Query the operation or destination state before resubmitting. The action may have succeeded even though its response was lost. If its outcome cannot be established, preserve that uncertainty rather than claiming failure or success.

When can a harness execute tool calls concurrently?

When they do not depend on each other's results or conflict over shared state. Independent reads are good candidates. A write followed by verification must remain ordered so verification observes the intended change.

What should an agent checkpoint preserve?

The run state, completed steps, pending operations, relevant messages, and references to artifacts. Record enough information to resume coherently; model conversation history alone may omit an in-flight external action.

Why can replaying a checkpoint duplicate work?

Restoring saved state may re-execute steps whose external effects already occurred. Use recorded operation results and idempotent actions, and reconcile uncertain outcomes before replaying a mutation.

How does an event log differ from a checkpoint?

An event log records the sequence of actions and outcomes. A checkpoint summarizes state at a point in that sequence. Together they support diagnosis and recovery, provided event ordering and persistence are reliable.

What can break when a saved run resumes after a deployment?

Tool schemas, workflow states, prompts, and artifact formats may have changed. Version the run configuration and define migration or restart rules rather than assuming old state matches new code.

What must survive conversation compaction?

The objective, constraints, decisions, completed work, unresolved failures, and artifact references. Keep verifiable facts distinct from proposed next steps so a resumed agent does not mistake a plan for an executed action.

How do working memory and persistent memory differ?

Working memory supports the current task, such as intermediate findings and pending steps. Persistent memory is retained for future runs. Promote information deliberately, with provenance and scope, rather than saving every model-generated claim.

Why store large intermediate artifacts outside the conversation?

Files or object storage preserve complete results across context limits and restarts. The agent can retrieve the needed portion later. Store stable references and track versions so later steps use the correct artifact.

What is just-in-time context loading?

Retrieve detailed information when the current step needs it instead of injecting everything at startup. It saves context space, but the harness must provide reliable search or lookup tools so important evidence remains discoverable.

How can context assembly affect prompt-cache reuse?

Changing an early portion of a prompt can invalidate reuse of the later prefix. Keep genuinely stable instructions consistent, while still supplying current task state. Cache savings must not come from withholding necessary updates.

What does an execution sandbox isolate?

The environment in which agent-generated code or commands run, including accessible files, processes, and network resources. Its effectiveness depends on enforced boundaries and configuration, not on telling the model to behave carefully.

Why give concurrent coding agents separate workspaces?

It prevents one agent's edits, dependencies, or cleanup from corrupting another's work. Integration still needs conflict resolution and validation; separate directories do not automatically make overlapping changes compatible.

Why enforce tool permissions outside the model?

The model can misinterpret instructions or untrusted content. Application-side checks determine whether the authenticated user and current run may access a resource or perform an operation, regardless of the generated explanation.

What should a human approval authorize?

A concrete action with specific arguments and a known scope. If the target, payload, or consequences change, reassess authorization. Approval of one proposed write should not silently authorize unrelated later writes.

How should a harness treat instructions found inside tool results?

As untrusted source content, not as authority to change the task or grant access. Keep provenance visible and enforce permissions in code. Delimiters help interpretation but do not form a security boundary.

How should a harness provide credentials to tools?

Through scoped runtime credentials or a secret store, rather than inserting secrets into model context. Redact sensitive values from logs and outputs, and limit each tool's access to what its operation requires.

What should a delegated subtask specify?

The objective, available inputs, allowed actions, output format, and completion criteria. A focused contract makes the result usable by the parent without requiring it to reconstruct the subagent's entire conversation.

Why give a subagent a focused context?

It reduces irrelevant history and separates an independent investigation from the main conversation. Include the constraints and evidence it actually needs; missing task context can make a fast subagent confidently solve the wrong problem.

What should the parent check before using a subagent's result?

Whether it answered the assigned question, cited accessible evidence, and produced the claimed artifact or change. Reconcile conflicting findings and inspect important outputs before treating a summary as established fact.

Which budgets should an agent harness track?

Wall-clock time, model tokens or cost, tool calls, and concurrent work. Apply limits across child tasks as well as the parent so delegation does not bypass the intended budget.

What should happen when a user cancels a run?

Stop scheduling new work, cancel active operations where supported, and record which actions completed or remain uncertain. Save useful progress and report outstanding external effects rather than implying cancellation reversed them.

What belongs in an agent execution trace?

Model and harness versions, step identifiers, tool arguments and results, timings, usage, and state transitions. Protect sensitive data while retaining enough detail to explain where the run diverged from the intended task.

Why evaluate the final environment as well as the transcript?

A transcript can claim that a file was saved or a record updated when the environment disagrees. Inspect the resulting artifact or state to measure task completion, then use the trace to explain failures.

When should an evaluation inspect the path an agent took?

When procedure matters, such as respecting permissions, avoiding duplicate writes, or using required evidence. Allow valid alternative paths; matching one reference tool sequence can penalize a correct solution.

Why run an agent evaluation task more than once?

Sampling and environmental variation can change the outcome. Repeated trials reveal reliability that a single successful run hides. Report the trial count and keep execution conditions comparable.

Which failures should a harness evaluation simulate?

Tool timeouts, malformed responses, permission denials, interrupted runs, and lost responses after writes. Check recovery and final state, not merely whether an exception was caught.

How do you test whether a harness feature helps?

Compare runs with and without the feature on the same held-out tasks, model, and resource budget. Measure success, cost, and latency. Extra planning or delegation can add overhead without improving completion.

How do LangGraph and Deep Agents relate to harness construction?

LangGraph supplies workflow execution and persistence primitives. Deep Agents packages capabilities such as planning, filesystem tools, context management, and subagents. The application still chooses tools, permissions, evaluation criteria, and deployment boundaries.

Where do SGLang and vLLM sit relative to an agent harness?

They generate model responses and manage inference resources. The harness manages the task loop and tool execution above that endpoint. Improving token throughput alone does not fix incorrect tool calls or lost workflow state.

How can harness traces support LLM fine-tuning?

Filter successful, authorized trajectories into demonstrations or derive validated reward signals. Preserve tool context and remove secrets. Hold evaluation tasks out of training so apparent improvement does not come from memorizing the benchmark.