AI agents basics

What happens in one turn of the agent loop?

The harness sends the context to the model, the model replies with text or tool calls, the harness runs the tools, and the results are added to the context.

What does the harness own that the model does not?

Running tools, holding credentials, enforcing permissions and limits, managing the context, and recording what happened.

A tool call has been proposed. Who executes the tool?

The harness. The model only produces a structured request naming the tool and its arguments.

Can the model's weights send a message by themselves, without the harness running a tool?

No. The model only outputs tokens. Any effect in the world happens because the harness acted on them.

The model says a write is authorized. What actually decides authorization?

The harness and the systems behind it, based on the user's real permissions. The model's claim counts for nothing.

Why not accept a final answer while one of the turn's tool calls is still pending?

The answer was written without that result, so it may be missing facts or contradict them.

What does a tool definition give the model?

A name, a description of when to use it, and a schema for its arguments. That text is all the model knows about the tool.

Why do tool names and descriptions matter so much?

The model picks tools by reading them. Vague or overlapping descriptions lead to wrong or missed calls.

What keeps two procedures from collapsing into one tool?

Distinct names, distinct descriptions of when to use each, and argument schemas that reflect their different inputs.

What is the trade-off in tool granularity?

Many small tools give flexibility but more steps and more chances to err. A few task-level tools are easier to use but less flexible.

What is a skill in an agent harness?

A packaged set of instructions, and sometimes scripts or files, that the harness loads into context when a task needs it.

Two runs use the same weights but different skill text. Why can their actions differ?

The skill text is part of the context, and the model's output depends on its context as well as its weights.

How does a skill differ from a tool?

A tool is an action the harness runs. A skill is guidance about how to do a kind of task, which may tell the model which tools to use.

What role does the system prompt play in an agent?

It sets standing instructions, such as the agent's job, rules, and output format, that apply on every turn.

The harness trims a paragraph from the window, and a copy is kept elsewhere. What does the next call see?

Only what is in the window. The stored copy matters only if it is put back in.

Why does an agent's context keep growing?

Every turn appends the model's output and the tool results, so long runs fill the window.

What is context compaction?

Replacing older turns with a shorter summary so the run can continue within the window. Details not in the summary are lost to the model.

Why ask an agent for structured output?

Code downstream can parse and check it reliably, instead of guessing at free text.

Why validate tool arguments in the harness?

The model can produce malformed or out-of-range values. Checking before running prevents bad actions and gives a clear error back.

What should the model see when a tool fails?

A clear error message as the observation, so it can retry, change approach, or report the failure.

Why do agent error rates compound?

If each step succeeds 95 percent of the time, ten dependent steps succeed only about 60 percent of the time.

When is an agent worth it over a fixed workflow?

When the steps cannot be known in advance and depend on what earlier steps find. Otherwise a fixed workflow is cheaper and more predictable.

What is a fixed workflow?

A sequence of model calls and tools whose order is set by code, not chosen by the model.

What baseline should an agent be compared against?

A single well-prompted call or a fixed workflow on the same tasks. The agent should beat it by enough to justify its cost.

How is an agent evaluated offline?

On a fixed set of realistic tasks with known checks, scoring success, cost, latency, and errors.

Why must each eval task have a clear success check?

Without one, scores depend on reviewer judgment and cannot be compared across versions.

What is trajectory review?

Reading the full sequence of steps in a run, not just the final answer, to find wrong turns, wasted steps, and lucky successes.

What is a lucky success?

A run that reached the right answer through faulty steps. It counts as success on outcome metrics but signals a hidden weakness.

Why run each eval task several times?

Sampling makes runs vary. Several runs show how reliable the agent is, not just whether it can succeed once.

What does pass^k measure?

The chance that all k runs of a task succeed. It rewards consistency, unlike pass@k, which rewards one success.

What drives an agent's latency?

The number of model calls, the tokens per call, and the time each tool takes, mostly in sequence.

What drives an agent's cost?

Mostly input tokens, which grow each turn as the context is resent, plus output tokens and any paid tool calls.

Why give an agent a step or token budget?

So a stuck or looping run stops instead of running up cost and time without limit.

What is a looping agent?

One that repeats the same call or plan without progress, often because the observation does not tell it what went wrong.

When should a human approve an agent's action?

When the action is hard to undo, costly, or affects other people, such as sending, paying, or deleting.

What is a guardrail?

A check outside the model, such as an input filter, output validator, or policy rule, that blocks unsafe or invalid actions.

Why is retrieved or tool text a risk?

It can contain instructions, and the model may follow them. Outside text should be treated as data, not commands.

Why separate read tools from write tools?

Reads are low risk and can run freely. Writes change state and may need checks, confirmation, or narrower permissions.

What is a dry-run mode for an agent?

The agent plans and proposes write actions, but the harness logs them instead of executing them.

What is a handoff to a human?

The agent stops and passes the case, with its context, to a person when it is unsure or out of scope.

What is a router in an agent system?

A step that classifies the request and sends it to the right prompt, tool set, or specialist agent.

Why use a subagent for a side task?

It works in its own fresh context and returns a short result, so the main context does not fill with the side task's details.

How does agent memory work across sessions?

The harness saves notes or facts to a store and inserts relevant ones into later contexts. The weights do not change.

If the final answer contradicts a tool result, which should be trusted?

The tool result. The answer should be corrected or flagged, since the model misread or ignored the evidence.

What should an agent trace record?

Each model input and output, each tool call with arguments and result, timing, token counts, and why the run stopped.

What can you not establish about a run that left no trace?

Which steps it took, what each tool returned, and why it ended, so failures cannot be diagnosed.

Why re-evaluate when the underlying model changes?

A new model may use tools differently, follow instructions differently, or fail in new ways, even if benchmark scores rise.

What online signals show an agent is helping users?

Task completion, how often users correct or abandon it, escalations to humans, and time saved, alongside cost.

Why define what an agent should refuse or hand off?

Clear limits keep it from attempting tasks it cannot do well, which is where costly errors happen.