Card 1 of 50
Front. Agent. In research terms, what is an LLM agent?
These cards are original study material. They are not questions from a named employer or a live hiring loop.
All 50 cards in this deck
1.Agent
In research terms, what is an LLM agent?
A policy, usually a language model, that repeatedly picks actions in an environment and receives observations back until the episode ends.
2.Policy
What does an agent policy select from a state?
The next action, or a probability distribution over possible next actions.
3.State
What is the state an LLM agent conditions on?
In practice, everything in its context: the task, earlier actions, observations, and any memory that was inserted.
4.Observation
What is an observation?
What the environment returns after an action, such as a tool result, page text, or error message.
5.Action-space
What is the action space?
The set of moves the agent may make, such as calling a listed tool, writing text, or ending the episode.
6.Trajectory
What is a trajectory?
The ordered sequence of states, actions, and observations from one run of the agent.
7.Episode
What ends an episode?
The agent declaring it is done, the environment reaching a terminal state, or a limit such as a step or token budget.
8.Step-limit
A step limit stops a trajectory early. What happened to the later steps?
They never ran. The episode is truncated, which is different from the agent choosing to finish.
9.Mdp
What is a Markov decision process?
A model of sequential decisions with states, actions, transition probabilities, and rewards, where the next state depends only on the current state and action.
10.Pomdp
Why are most agent tasks partially observable?
The agent sees only observations, not the full environment state, so it must keep track of what it has learned.
11.Reactive
What is a reactive policy?
One that maps the current observation to an action with no explicit plan or long-term memory.
12.Horizon
What is the horizon of a task?
How many steps a task takes. Long horizons make errors compound and credit assignment harder.
13.Credit-assignment
What is the credit assignment problem?
Working out which earlier actions caused a reward that only arrives at the end of a long trajectory.
14.Cot
What is chain-of-thought prompting?
Asking the model to write intermediate reasoning steps before its answer, which often helps on multi-step problems.
15.React
What is the ReAct pattern?
Interleaving reasoning text with actions, so the model thinks, calls a tool, reads the result, and thinks again.
16.Self-consistency
What is self-consistency?
Sampling several reasoning paths and taking the most common final answer.
17.Tree-search
What is tree-of-thought or tree search for agents?
Expanding several candidate next steps, scoring them, and exploring the most promising branches instead of one linear path.
18.Reflexion
What is a Reflexion-style agent?
One that writes a short critique after a failed attempt and puts it in context for the next attempt, with no weight update.
19.Plan-execute
What is plan-and-execute?
First write a plan of steps, then carry them out, possibly revising the plan when observations disagree with it.
20.Toolformer
What did Toolformer show?
A model can learn when to insert API calls by training on its own calls that were kept only if they lowered loss on the following text.
21.Tool-learning
What can a tool-use training step change?
The model's weights, and so its tendency to call tools. It does not change what the tools themselves do.
22.Sampling
What does sampling do with the policy's distribution over the next action?
Draws one action at random according to those probabilities, so repeated runs can take different paths.
23.Outcome-reward
What is an outcome reward?
A score given only for the final result, such as whether tests pass or the answer is correct.
24.Process-reward
What is a process reward?
A score for each intermediate step, judging whether that step is valid, not just whether the end is right.
25.Right-wrong-path
The final answer is right but the path is flawed. How does a final-answer metric treat it?
As a full success. Only a process check would notice the flawed steps.
26.Verifier
What is a verifier?
A model or program that checks a candidate output, such as a unit test suite or a trained correctness classifier.
27.Best-of-n
What is best-of-n sampling?
Generate n candidates and keep the one a verifier or reward model scores highest.
28.Rejection-sampling
What is rejection-sampling fine-tuning?
Sample many trajectories, keep only the successful ones, and fine-tune on them.
29.Behavior-cloning
What is behavior cloning?
Supervised training on expert trajectories so the policy imitates the expert's action at each state.
30.Compounding
Why does behavior cloning suffer from compounding errors?
One mistake moves the agent into states the expert never visited, where the policy has no good examples.
31.Rl-env
What does RL from environment feedback add over imitation?
The policy learns from rewards on its own trajectories, so it can improve past the demonstrations and recover from its own mistakes.
32.Exploration
What is the exploration problem?
The agent must sometimes try uncertain actions to discover better strategies, at the cost of short-term reward.
33.Reward-hacking
What is reward hacking?
The policy finds a way to score high on the reward that does not achieve the intended goal.
34.Memory-weights
Must memory that persists across episodes be a weight update?
No. It can be stored text or embeddings that are retrieved and inserted into later contexts.
35.Episodic-memory
What is episodic memory for an agent?
A store of past events or trajectories that can be retrieved later.
36.Multi-agent
What is a multi-agent setup?
Several policies, or several roles of one model, that act and exchange messages in a shared task.
37.Debate
What is AI debate?
Two models argue for different answers and a judge picks the more convincing one, as a way to supervise hard questions.
38.Debate-state
If debate transcripts are not merged, what does each debater condition on?
Only its own context: the question and the messages actually given to it.
39.Self-play
What is self-play?
Training a policy against copies of itself, so the opponent improves as the policy does.
40.Pass-at-k
What does pass@k measure?
The probability that at least one of k sampled attempts succeeds.
41.Success-rate
What is task success rate?
The fraction of tasks in a benchmark that an agent completes, under a stated budget and checker.
42.Swe-bench
What does SWE-bench measure?
Whether an agent can resolve real GitHub issues so that the repository's held-out tests pass.
43.Web-env
What are web-agent benchmarks such as WebArena?
Simulated websites where an agent must complete tasks through browser actions, checked by the final page or database state.
44.Seed-variance
Why report several seeds for agent results?
Sampling makes runs differ, and long trajectories amplify that, so one run can over- or understate a method.
45.Budget-fair
Why must agent comparisons fix the budget?
More steps, tokens, or samples usually raise success, so a gain may come from spending more, not a better method.
46.Cost-per-task
Why report cost per task alongside success?
Two methods with equal success can differ greatly in tokens, calls, or time spent.
47.Llm-judge
What is LLM-as-judge?
Using a language model to grade outputs. It is cheap but can be biased toward length, style, or its own outputs.
48.Heldout-tasks
Why hold out tasks, not just instances, in agent evaluation?
To test whether the agent generalizes to new kinds of task rather than memorizing variants of seen ones.
49.Benchmark-leak
How can an agent benchmark leak?
Its tasks or solutions, such as public issue fixes, may appear in pretraining data or be reachable by the agent's web tools.
50.Environment-determinism
Why does environment determinism matter for agent studies?
If the environment changes between runs, differences in success may come from the environment, not the agent.