AI agents

1.Release 441 mounts skill ledger-close 2.4.1 on the invoice workers and leaves model deployment pay-7b on the 18 September digest. The reviewer marks the agent upgraded because the skill digest changed. A live invoice run still answers from those September weights. Which artifacts does that run pair?

2.Canary pods mount skill digest ab12. Baseline and canary both call model route agent-main, which this rollout did not change. Canary traces plan with the new tool reserve_dock, and the model span's weight digest matches baseline. What did the canary pods change?

3.The rollback restores model image dock-3. Skill object yard-skill stays at v9, and v9's instructions require window_id. After the rollback the model still emits slot, and v9 is still the loaded skill. Which pair is serving the yard agent?

4.The pipeline's skill job is green and its model job is skipped. Registry object returns-skill 1.8.0 was published at 14:02. Object returns-llm is still the 11:00 digest. Mesh traffic shifts at 14:05. What new artifact is in the run after the shift?

5.Green workers mount skill support@6. The model service is a separate deployment, still support-llm build 11, and both colors call it. Green answers follow the v6 refund wording. Blue answers follow v5. What did the green rollout put on its workers?

6.Skill picker@2 rolls out with a manifest line that names model pick-70b-r3. The loader pins weights from env MODEL_DIGEST, which is still r2, and the startup log shows r2. That manifest line is not in the loader's config. What does the line change for the next pick run?

7.Image tag agent:2026.11.02 adds the layer /skills/payout and does not replace the weights volume, which is still PVC weights-2026-08. The deployment rolls, and a payout run follows the new skill text. Which bytes are new on that run?

8.The GPU runner is offline, so artifact model-weights is not pushed. The helm release still bumps skillDigest to the new bundle and leaves modelDigest unchanged. New pods log the new skill and the previous weight digest. What did this release install?

9.A shadow test flips only the skill flag. Both arms call llm-prod revision 40, and the spans agree on that revision. The new arm's tool mix shifts, and a ticket calls it a regression in revision 40. What differs between the arms?

10.Someone edits the model card in git so it says the agent can void invoices. No pipeline runs. Production still has skill billing@4, which binds no void tool, and the model server is unchanged. A void request ends in a refusal that matches billing@4. What did the card edit change in the running agent?

11.Config sync delivers skill tax@3 to eu-central. Weight-bucket replication is behind, so that region still mounts tax-llm@1. A trace from a Paris invoice shows the v3 VAT instructions and model id tax-llm@1. What is that region running?

12.The deploy bot says the model was promoted because git tag agent-2026.11.02 moved. That tag points at the skill repo. Weight-registry tag prod was not moved, and the model server still reports the previous digest. What did the moving tag promote?

13.The charge for order 8841 times out in the harness after the provider has already captured $42. The retry is a new POST with no Idempotency-Key. The provider captures $42 a second time. Why did that retry create another charge?

14.The refund of invoice 220 returns 504 after the ledger has committed the refund. The retry sends a newly minted UUID as Idempotency-Key, and a second refund posts. What did that new UUID mean to the ledger?

15.The model writes "same charge, do not double" into the note field. The harness retries the POST with that note and no idempotency key. Two captures land for the $19 fee. What was the note, as far as the provider is concerned?

16.The seat-grant tool uses the trace id as its idempotency key. The timeout retry starts a new trace, the key changes, and user 442 receives a second seat. Why did the grant API accept the second call?

17.The harness computes a stable idempotency key for a $19 fee. The tool adapter drops headers it does not recognize, the payment service never sees the key, and two captures post. What did the adapter strip that the provider needed?

18.The retry of a card capture reuses the idempotency key and changes the body from $40 to $60. The payment API returns a conflict and leaves the $40 capture. The agent tells finance the charge was corrected to $60. What does that conflict leave standing?

19.The idempotency key for payout batch 77 lives in the worker's memory. The worker dies after the payout API has paid the batch. The replacement pod retries with a key it just minted, and a second payout is created. What did the restart lose that the API uses to recognize a repeat?

20.One helper retries every tool call the same way. Retrying an invoice read is harmless. Retrying apply-credit for account 18, with no idempotency key, posts a second credit. What is missing on the credit call that the helper treated as unnecessary?

21.The payment API returns 409 for cart 55 because that idempotency key is still in flight. The agent reads 409 as a failed charge and sends another POST with a new key. Two charges remain. What did the 409 say about the original key?

22.The shipping-label tool builds its idempotency key by hashing the user's sentence. On the retry the model rephrases the sentence, the hash changes, and a second label is purchased for shipment 90. Why did the new wording buy another label?

23.Support replays a timed-out charge through the production charge tool and supplies a new idempotency key so the call will not collide with the first try. The first try had already captured the card. The replay captures it again. What did the new key guarantee at the provider?

24.Two replicas both take the queue message for one access grant. Each replica mints its own idempotency key and calls grant_access. The user ends up with two seats. What did the replicas fail to share?

25.The lookup for pallet 18 times out. The harness writes status ok and an empty body. The model tells the buyer the warehouse has no such pallet. The lookup is still running, and the pallet is on the shelf. What did that stored observation claim?

26.Card capture for order 300 times out. The stored observation is success with an empty auth code, and the model tells the buyer they were not charged. The processor has captured the card. What is wrong with that observation?

27.A refund lookup times out and throws. The harness catches the exception and closes the span as OK with body "". The error board stays flat while agents tell customers there is nothing to refund. What did the OK span bury?

28.The seat-map call ends in gRPC DEADLINE_EXCEEDED. The adapter stores content null and ok true. The model treats the map as empty and books nothing, while the timed-out call may already be holding a seat. What did the adapter's record erase?

29.The buyer disconnects while a void tool is still running. At disconnect the trace row is written as success with an empty observation, and the run is closed. Minutes later the tool voids invoice 77. What does that closed row fail to establish?

30.A streaming ledger tool is cut at the deadline. The partial body is stored and the step is marked complete. That fragment contains "ok": true, and the error field was in the tail that never arrived. The model continues on ok true. What did the cutoff save as the result?

31.The circuit breaker is open on the invoice tool. The harness returns an empty object with success so the loop will not raise. The model tells the customer the account has no invoices and closes the case. What is that empty object standing in for?

32.The nightly scorer sees observation "" and status success on a fraud-hold check that timed out. It marks the task "no action needed," and the report goes green. The hold service never returned a decision. What did the scorer grade?

33.The client cancels a refund call that has already been sent. The harness stores the step as success with an empty body. The next step reads that row and skips the human refund. The in-flight call later commits the refund. What did the next step take as evidence?

34.The tool timer fires, the harness commits an empty success, and only then does the tool return its real body. The harness leaves the observation as it is. The model never sees the body, and audit later finds the side effect. What does the trace hold for that step?

35.The incident-list endpoint fails its health check. A fail-open adapter writes an empty success so the agent can keep going. The agent then posts "no open incidents" in the customer channel. What did the adapter put in the trace?

36.The parent deadline cancels the reservation RPC. The handler still commits reservation R1. The parent stores success and an empty reservation id. A later run creates R2 for the same pallet. What did the empty id hide from the later run?

37.A disputed refund trace stores model pay-7b@88 and no skill version. Two skills were live during that hour. Repro loads pay-7b@88 with today's skill, and the tool arguments do not match the episode. What did the repro lack because of the trace?

38.Spans on the refund path carry model.version and do not carry skill.digest. Two skill deploys overlap for ten minutes, and the disputed refund sits inside that window. The deploy log has both digests. What can the span not assign to the refund?

39.The trace stores the weight sha and the skill channel name latest, the name the pod resolved at start. A reader the next day still sees latest, and the channel has moved. What does the stored label identify?

40.Arm A and arm B both use model nlu-4. Arm A loads skill copy@1 and arm B loads copy@2. The warehouse table partitions only by model version, and a tool-mix alert fires on nlu-4 as a whole. What is that table combining?

41.The worker logs skill digest d0 at startup and copies d0 onto every span. At 16:10 it hot-loads a new skill and does not change the attribute. A request at 16:20 is traced with d0, and the loaded package at that minute is the new one. Which skill built the 16:20 request?

42.The exporter drops any attribute whose value contains the text /skills/. The skill version was stored as a filesystem path, so it is removed. The model version is a bare sha and stays. What does the exported trace lose?

43.The parent span records the model version. Each child tool span records the tool name and no skill version. The same tool name was bound by different skills in different hours. A tool span from 11:00 cannot be joined to a skill. What is absent from both the parent and the child?

44.Replay pins the model version it finds on the trace. The skill field is null, so the job loads whichever skill is current. The replayed tool arguments differ from the episode. Why is that difference not a drift in the weights?

45.charge-svc moves from image 20 to image 21 during the lunch hour. The skill name and the model version stay the same. Traces record those two names and not the tool image digest. A double-capture bug exists only in image 21. What can a trace from that day not separate?

46.The latency board groups tool-call p99 by model version. A release that changes only the skill goes out, p99 jumps, and the board attributes the jump to the current model. That model's digest did not change. Which release is missing from the board's grouping?

47.Audit asks which instructions produced a collections email. The trace has model collections-13b@4 and a hash of the base system template baked into the image. The skill overlay that added the collections tone is not covered by the hash. What can the hash verify?

48.The trace pins skill notify@5 and the model version. The schema enforced on the send tool came from the registry at call time, and it is not on the trace. The registry later drops a required field. Replay with notify@5 and the same model still misses the episode. What else changed underneath the replay?

49.The vendor balance tool returns its API key inside a debug field. The observation stores that body unchanged, and the next model call receives the observation. The key is then readable in the trace store. What did saving the observation do with the key?

50.Redaction covers the user message before the model call. The tool then fails, and its error string echoes the Authorization header. That string is stored as the observation and sent back into the next model call. Where did the header get past the filter?

51.A tool observation contains a live vendor key. The model's final answer repeats the key, and a support agent sees it in the customer transcript. How did the key get into the model's context for that answer?

52.The trace exporter sends observations to a third-party APM project. A sandbox token is sitting in a tool-error observation from the billing agent. The token now shows up in that APM project. What crossed the cluster boundary?

53.Skill billing-helper tells the model to copy each tool result in full, tokens included, so the next step can reuse them. The model puts a refresh token into the next tool's arguments, and the argument log keeps the call. What did the skill lead the loop to do with the token?

54.The billing-page tool returns a DOM observation that includes the agent's session cookie. The step is inserted into warehouse table agent_steps, whose retention is 400 days. What does that stored row contain for the life of the retention?

55.A prompt template interpolates env var PAY_SECRET into the system prompt, and the trace stores that prompt. A contractor whose only access is trace read opens the run. What did the template place where the contractor can read it?

56.The charge tool's exception printer includes the request headers. The harness stores that exception text as the observation and sets the step status to error. A bearer token is in the text. Does the error status keep the token out of the trace?

57.The loop copies each observation verbatim into the next span's input. Export redaction clears the first span's output and does not clear the next span's input. A vendor key was in the observation. Where does the exported trace still contain the key?

58.The model copies a key out of an observation into a tool argument named api_key. The argument logger redacts any field named password, then writes the call. What does the stored call still contain?

59.Approval for payout batch 14 exists only in the worker's memory. The pod restarts, that memory is gone, and the path for a missing approval record is allow. The payout tool runs. Disk has no approval row for batch 14. What did the restart erase, and what did the worker do next?

60.The approver's POST is handled by pod A and recorded in A's memory. The UI shows approved. Pod A is then terminated by the deploy. Pod B, which takes the next step, has no approval record for that action. What does pod B have?

61.Approvals are kept in process memory, and this worker has just restarted. The action id is unknown. The gate's unknown-id branch skips the check so the queue will not stick, and the wire tool runs. What is an unknown id on this restarted worker?

62.The refund tool for invoice 310 commits at the ledger. The worker crashes before it writes an approval row or a completion row. The new process finds neither row and sends the refund again. What has already happened that the new process cannot see?

63.Finance approves a payout whose stored arguments have amount 40. After the click, the model changes the amount to 400. The harness runs the 400 payout under the same approval id. What payload did finance actually approve?

64.The model writes the sentence "approved by finance" into the assistant message. The approval service is never called. The harness treats the sentence as the decision and sends the wire for vendor 12. What, inside the harness, stood in for the approval?

65.The approver's POST lands on replica A and stays in A's memory. Replica B does not see it, prompts again, and the operator approves on B as well. Each replica runs the payout tool when its own memory shows an approval. The payout runs twice. What was private to each replica?

66.The approval row for a seat grant, arguments included, is committed to the store. The worker restarts before the tool runs. The new process loads that row and executes the stored arguments instead of asking the model for a new completion. The arguments match the approval. What did the new process execute from?

67.The approval row for a funds transfer expires after 15 minutes. The worker restarts at minute 20, reads the row, sees status approved, and sends the transfer. The row's expiry timestamp is in the past. What did the status field fail to account for?

68.A deny on the vendor wire is stored only in the worker's memory. The restart clears that memory. The branch for a missing decision is proceed, and the wire that had been denied goes out. What did the lost deny look like to the new process?

69.Startup registers payments.submit from skill refunds at 09:01 and from skill fees at 09:02. The registry keeps one binding, and a dump shows the fees handler. The model, following the refund skill, calls payments.submit to refund order 70. The customer is charged a fee. Why did a refund instruction result in a fee?

70.During the billing incident, some pods had both skills loaded, and both had registered payments.submit. The tool span records the name payments.submit and no skill id. Finance cannot tell a fee call from a refund call. What is missing on that span?

71.The tool list shows the refunds schema for payments.submit, with fields amount and reason. The handler that actually runs is the fees binding, which requires amount and sku. A refund-shaped call fails the sku check, a fallback fills sku MISC, and a fee posts for order 70. What did the shared name tie together?

72.A canary skill registers the name search and shadows the platform search tool on the canary pods. Baseline pods still run the platform tool. Both groups of traces record the tool name search. Canary answers come from the canary index. Why are the result sets different?

73.Uninstalling skill fees deletes the registry entry for payments.submit. Skill refunds had registered that same name and has no separate entry. Refund calls now fail with unknown tool. What did the uninstall delete?

74.A test skill is loaded in the production worker and binds grant_seat to a stub. The stub returns ok and writes no seat. It shadows the production skill's grant_seat. The model tells the user the seat is granted. What did that ok report?

75.The docs skill and the customer skill both bind the name search. The tool list sent to the model contains search once. The runtime executes whichever binding loaded last, and the model has no second name it can call. What did that single list entry hide?

76.Skill alpha binds lookup to the legacy account store. Skill zeta binds lookup to the new store. Precedence is alphabetical, so alpha keeps the name after zeta's deploy is marked live. A lookup for account 55 runs. Which store answers it?

77.The model emits the bare tool name close. Both billing/close and case/close are registered. The runtime has no rule for a bare name and dispatches the first map entry, billing/close. The billing period closes, and support case 88 stays open. What did the bare name leave for the runtime to guess?

78.A worker that should only charge fees has fees bound to payments.submit. A hot reload of skill refunds registers that same name and replaces the binding. The next call on the worker refunds order 15 instead of charging it. What did the reload replace?

79.max_steps is raised from 8 to 20 in the config file at 11:00. The workers are not restarted. A run at 11:20 stops on step 8 with reason step_limit, and the config API returns 20. Which limit did the 11:20 run enforce?

80.After the step-limit edit, pods the autoscaler creates stop their runs at 20 steps. Pods that were already up still stop at 8. Traces from the same half hour show both endings. What is different about those two sets of pods?

81.Helm values now set maxSteps to 20. The deployment was not restarted, so the running container's command is still --max-steps=8, and that flag overrides the file. Runs keep stopping at 8. What number is the process enforcing?

82.The runbook says SIGHUP reloads the step limit. No handler is wired to that signal. An operator edits the file to 20 and sends HUP to the worker. The next runs still stop at 8. What did that signal leave unchanged?

83.The eval harness reads the edited config and allows 20 steps. The production workers still stop at 8. A scenario that needs 12 tool calls passes in eval and dies in production on step 8. What limit did the eval harness measure?

84.Operators reload the worker so its step limit is 20, and they leave the wall-clock deadline at 30 seconds. Runs now reach step 12 and die with reason deadline. The step-limit graph goes quiet. What is ending those runs?

85.One request carries an overlay that asks for 30 steps. The worker's cap, loaded when the process started, is 8. The run stops at step 8 with reason step_limit. What did the overlay fail to raise?

86.The flag service is flipped so tools_enabled is false. Workers read flags once, at process start, and this worker still has true. A payout tool runs ten minutes after the flip. Which flag value did the payout see?

87.The mounted tool file is edited so the charge endpoint moves from the production billing host to the sandbox host. The worker cached the endpoint when it started and keeps calling production. A test charge from that worker hits the production ledger. What did the file edit leave untouched?

88.The dashboard panel labeled effective config reads the config service and displays max steps 20. The incident run's span attribute step_limit is 8, and the run stopped at step 8. Which number did that worker enforce?

89.The billing tool is started with the sandbox flag set, and it still creates a charge on billing.internal. The network policy on that pod allows egress to the billing namespace. What did the sandbox flag leave open?

90.The sandbox builds its allowlist from the hosts the skill declares. This skill declares billing.internal. The sandbox loads that list, and the charge tool posts to billing.internal successfully. How did the billing host get inside the confinement rule?

91.The network policy selects pods with label sandbox=true. The proxy pod has that label and cannot open a connection to billing. The tool runner is a separate deployment with no such label, and its charge lands on billing.internal. Which deployment made the charge?

92.The sandbox allowlist permits docs.internal and denies billing.internal. The tool posts to docs.internal/hooks/charge. That vhost proxies the body to the billing service, and the card is captured. The sandbox log names only the allowed destination. Where was the card captured?

93.The test namespace carries a policy that blocks the billing hosts. The production agent namespace, where the charge tool runs, has no such policy. A test job fails to reach billing, and the production tool posts a charge. The ticket says the sandbox was proved in test. What did that test actually show?

94.The model writes "do not call production billing" into the scratchpad before the tool call. The tool client has a route to billing.internal and posts a charge there. What effect did the scratchpad sentence have on that post?

95.The egress rule denies the hostname billing.internal. The charge tool dials a billing IP that is hardcoded in the skill, and the rule does not list that address. The charge posts. What did the hostname rule miss?

96.Direct egress from the sandbox to the billing hosts is denied. HTTPS_PROXY inside the sandbox points at a proxy that is allowed to dial billing. The charge is sent to the proxy, and the ledger shows a posted charge. Which path carried it?

97.The sandbox policy blocks new processes, and a check confirms bash cannot start inside it. The billing tool is an HTTP client in the same sandbox, and it still posts a charge to the billing host. What did the process block leave available?

98.gVisor is on for the tool pod, and the pod's network policy is empty. The on-call note treats gVisor as network isolation. The tool opens a TCP connection to the billing host and posts a charge. What decision did gVisor not make?

99.The nightly replay is supposed to answer tool calls from a cassette. The cassette has no entry for a refund of order 501. The harness falls through to the live tool registry, and order 501 is refunded. What did the missing cassette entry turn into?

100.The eval job mocks the model client and leaves the production tool registry in place. The job's name in the scheduler is shadow. The first evidence of the run is a capture on a real customer card. What did that mock actually replace?

101.Dry-run is set on the model client for the replay. The tool client does not look at that flag. The replay emits void_invoice for invoice 440, and the invoice is voided in the billing system. What did the dry-run flag never cover?

102.The cassette lookup matches on the tool name and ignores arguments. The stored charge is $5. The replay asks for a charge of $500, the lookup treats the argument difference as a miss, and the miss path calls the live tool. A $500 capture posts. What did that argument miss cause?

103.The replay attaches its cassette after the first tool call returns. That first call is grant_seat against the production access service, and every later call is served from the cassette. When the run ends, the user has a seat they did not have before. What did the late cassette leave behind?

104.In this replay, search is stubbed and the payment tools are not. Scoring looks only at the final assistant message, and the score is high. During the run a real refund posts to order 18. What did the score fail to count?

105.The cassette's recorded result for cancel is an error. The replay shows the model a synthetic success so later steps can run, and the same code path calls the live cancel tool to refresh the tape. Subscription 62 is cancelled. What did the refresh call hit?

106.Online eval samples a production trace of the payout for batch 12 and re-executes the tools to see whether they still match. The re-execution uses a fresh idempotency key. A second payout for batch 12 leaves the bank. What did that re-execution perform?

107.The golden-set runner treats tool stubs as optional. A new scenario ships with no stub for charge. The runner calls the live charge tool, the final text matches the expected sentence, and the scenario is marked passed. The charge shows up on a production merchant. What did making the stub optional permit?

108.Yesterday's episode called close_day, and the cassette keys that stub by date. The replay builds the key from the wall clock, gets today's date, misses the cassette, and the miss path calls the live close_day. The books close a second time. Why was today's key a miss?