AI agents basics
An MLOps interview on the runtime facts of an agent harness: tools, limits, traces, and state.
These prompts are original practice. They are not questions from a named employer or a live hiring loop.
1.After the model emits a tool call, which component runs the tool?
2.When does a tool's network request or file write become a real side effect?
3.What does the model emit when it wants a tool?
4.The assistant reply states a command in ordinary sentences and contains no tool-call object. What does the harness execute from that reply?
5.How does a tool's return value get into the next model call?
6.A call is missing a required argument. When does the runtime apply the schema?
7.The prompt describes a tool one way and the loaded schema describes it another way. Which contract does the runtime enforce?
8.An argument breaks the schema. What does the runtime do before the tool runs?
9.A call fails schema validation. What is stored, and does the tool run?
10.Which arguments can be stored as the action that was actually dispatched?
11.What does a tool timeout bound?
12.The timeout fires before the tool returns. Which observation is honest?
13.A write tool times out, and the harness cannot tell whether the write committed. What should the record say?
14.The model puts a longer timeout in its message. What happens to the timer?
15.The tool responds after the model has already been given the timeout error. What may that late body do?
16.What does the step limit count?
17.The step budget is already used up, and the model emits another tool call. What does the harness do?
18.A run stops because the step limit was reached. How is that stop stored?
19.Where is the step counter for a run kept?
20.The step limit stops the run after several tools have returned. What happens to those finished observations?
21.A tool creates a charge. When is an automatic retry safe?
22.What does a runtime idempotency key allow the tool to do?
23.The model writes that this is the same request as before. Does that sentence work as an idempotency key?
24.A call times out and the harness retries it. Which idempotency key does the retry carry?
25.A read fails once and a write fails once. How do safe retries differ?
26.Where should a tool credential be kept?
27.When may a runtime secret appear inside the model context?
28.A tool result echoes the raw credential. What does the runtime do before the next model call?
29.The model asks the runtime to print the API key. What is required?
30.Two tools are bound with two different keys. Who attaches the key for a given call?
31.What does a tool sandbox constrain?
32.The model writes a warning to stay out of production systems. Does that sentence sandbox the tool?
33.A tool starts a child process. When is that child inside the sandbox?
34.The model names a host that is not on the tool allowlist. What does the runtime enforce?
35.A tool reads a path outside its filesystem root and returns those bytes. How is that result classified?
36.What does the trace store for one tool step?
37.A finished run left no trace. What can a later reader not establish from that run?
38.A tool step in the trace has an action and a time, and the observation was left out. What is missing?
39.A second run starts after the first has finished. Where does the second run write its tool steps?
40.A tool returns an error. What does the trace keep for that step?
41.Where does durable agent state sit relative to the context window?
42.Who chooses which stored state is placed in the next model call?
43.A fact is in the state store and was not loaded on this call. What can the model use from that fact right now?
44.The model states a new fact in ordinary assistant text. When does the state store change?
45.The next call cannot fit every stored fact. What happens to the facts left out of the payload?
46.What is deployed inside a skill package?
47.The skill package is replaced and the weights stay the same. What can change on the next run?
48.Are the skill instructions part of the weight file?
49.A tool process is healthy, and the loaded skill does not bind that tool. The model names the tool. What does the harness do?
50.What does the run record so a later reader can identify the instructions that were loaded?
51.Two skills expose a tool under the same bare name, and the runtime does not namespace them. What fails?
52.The runtime namespaces tools. Which identity does it use to dispatch?
53.What has to be true of the callable names shown to the model?
54.Tools are namespaced. Which tool identity does the trace store?
55.The model emits a bare name that two namespaces both define. What does the runtime do?
56.The model requests several tool calls together. What limit must the harness enforce on them?
57.The prompt allows 30 tools at once. The harness concurrency limit is 4. Which number is enforced?
58.Several calls that ran together have finished. How does the harness attach their observations?
59.Three calls run together. One returns an error and the other two return results. What should the model receive?
60.Who sets the order in which parallel observations are written back into the run?
61.What is a human approval step inside the agent loop?
62.The process restarts while approval is still pending. What must still be stored?
63.The approver accepts the pending action. Which payload runs?
64.The model writes that the pending action is approved. Does the harness run it?
65.The arguments change after approval was recorded. What has to happen before those new arguments run?
66.Which versions are separate artifacts of one agent episode?
67.A trace records the model version and leaves out the skill and the tool. What can a later attempt not do from that trace?
68.The tool implementation is replaced. The skill name and the weights stay the same. What must a new trace record about that change?
69.Two runs share a model version and use different skill versions. What follows?
70.A replay pins the recorded model version and uses a newer tool build than the trace names. Does that replay match the episode?
71.A tool observation is larger than the context budget. What must the runtime do?
72.The full observation is kept in storage. What does the model call receive from it?
73.An observation was shortened before the model call. What must the trace distinguish?
74.Who cuts an oversized observation down to the context budget?
75.A shortened observation is sent with no record of the cut. What can a later reader not know?
76.How does the runtime tell a final answer from a tool call?
77.A message is marked final and also carries a tool-call payload. What does the harness do with that payload?
78.The final answer names a tool in a sentence. What does the harness launch because of that sentence?
79.The model produces a final answer and the run ends on it. Which status fits?
80.What does a multi-agent handoff start?
81.Does the callee receive the caller's secrets at handoff?
82.What is included in the callee's run at the moment it starts?
83.Which tool bindings does the callee run with after a handoff?
84.How does a callee's result get back to the caller?
85.Where is the retry policy for tool calls stored?
86.Two replicas load the same weights and use different retry policies. Can their tool calls differ?
87.Which clock supplies the time stored on a tool step?
88.An operator points the harness at a different clock source and does not change the checkpoint. What changes on later runs?
89.A tool allows one call per minute. The model requests it twice inside that minute. What is enforced?
90.Where is a tool's rate limit defined?
91.The rate limit stops a call. Which observation matches that stop?
92.The model asks again on the next turn, and the rate-limit window is still open. What happens to that window?
93.Prompt logs and traces can hold user data that a tool handled. What access control does the trace store need?
94.A tool is limited to one team. Who may read a trace that stored that tool's observations?
95.Arguments saved in the trace contain user data. How are those arguments protected?
96.A run has ended and its trace is still on disk. What is true of access to that trace?
97.How do tool-endpoint health and model-server health relate?
98.The tool endpoint is down and the model server is up. How does the loop fail?
99.One tool endpoint fails its health check. What else does that single failure mark down?
100.A health failure is written as an empty success. What does that record get wrong?
101.During a deterministic eval, where does a tool observation come from?
102.During a live eval, the model emits a tool call. What does that run do?
103.A replay eval and a live eval pin the same model version. Why are they still different runs?
104.A deterministic eval has no recorded observation for one call. What would turn it into a different kind of run?
105.A bad skill deploy is undone by restoring the previous skill package. The model deploy is not part of that restore. What does the next run load?
106.A serving replica is pointed at an older model checkpoint. The installed skill package is not rolled back. What does the next run load?
107.The previous skill package is restored. Which other records does that restore leave as they are?
108.An older model checkpoint is restored. Which other artifacts move with it?