FreeExam

Launches

An AI/ML engineering manager interview on which model ships when the demo, the eval, and the incident want the same week.

These prompts are original practice. They are not questions from a named employer or a live hiring loop.

  1. 1.The director loved the museum-guide demo. The eval set is the twelve prompts from the demo script. What is that set measuring?

  2. 2.The port wants the ferry chatbot on Friday. The team is mid-migration of the retrieval index the answers depend on. What do you put in front of the port?

  3. 3.Sales told a hospital the discharge-summary model ships this month. The team has no eval and has not estimated chart access. What do you do today?

  4. 4.The dosage assistant returns a dangerous schedule. The sprint is full of prompt polish. Two people will spend the week on the failure. What happens to the polish?

  5. 5.During a small prompt fix, an engineer wants to replace the retrieval stack of the library bot while you are here. What is that replacement?

  6. 6.Offline score on the housing bot is up four points. The online rate of correct citations is flat. Which number decides the launch?

  7. 7.A gain appears on the benefits bot after someone copied hard eval cases into training to clean the set. The score jumps. What is the jump?

  8. 8.The only human eval of the tax-help bot is twelve ratings, all positive, by the person who wrote the prompts. They want to ship. What is missing?

  9. 9.The product manager writes the golden answers after reading the model's outputs, then scores the model against those answers. Why is the score soft?

  10. 10.The new summary model is ready, and there is no way to return to last week's model. The owner says they will fix forward. Riders read the summaries. What is missing?

  11. 11.The new agent calls three models per question and would double the monthly bill. That number is not in the launch review. What do you add before anyone ships?

  12. 12.The support bot can refund a ticket if the user asks. A pasted page can instruct the bot to refund. Security's note is unread, and the launch is tomorrow. What blocks tomorrow?

  13. 13.Last week's support tickets, including account numbers, were dropped into a fine-tune of the parking bot because they were real language. What has to happen before any ship?

  14. 14.A vendor model would save a quarter on the clinic bot. Leaving that vendor later is a year of work, and the contract is in front of you this week. What is part of the signature?

  15. 15.The quarter plan assumes two unhired GPU engineers are productive in month one on a pretrain. Neither seat is filled. What do you plan with?

  16. 16.Status is green because training loss fell. The refusal eval has not been run. Last time, loss fell while refusals got worse. What is the status?

  17. 17.In week six you learn the hospital summaries need labels the vendor cannot deliver for three more weeks. Launch is week eight. What do you send today?

  18. 18.Engineers are prompting the enrollment bot before anyone has written what a wrong answer is. They say starting now saves time. What do you stop?

  19. 19.The safety review of the benefits bot is the only item that threatens Friday. Someone suggests cutting the review to hit the date. What is Friday now?

  20. 20.The new search UI reads a fresh embedding column. The backfill of old books was cut so the library demo can happen. What does the demo hide?

  21. 21.The launch checklist for the triage assistant has a pod restart and no steps for the model being up while the advice is wrong. What is not ready?

  22. 22.Staging has one small GPU, so the team skips a load test of the museum guide. The peak day is a school holiday. What do you still require?

  23. 23.The new prompt has been at 100 percent for four months. The old prompt is still in the binary, and nobody will remove it. What is the flag now?

  24. 24.Teachers love a prototype of the enrollment bot. It has no audit log and no reason shown when a family is denied. Sales wants it generally available tomorrow. What is general availability here?

  25. 25.After the parking bot launched, support tickets tripled, and they are not on the roadmap. The next feature is a new voice. What is the work?

  26. 26.The objective is to ship five models. The team can hit it with five prompts that do not change what users get. What do you change the objective to?

  27. 27.The objective is to improve the assistant, with no number. What is missing?

  28. 28.The executive demo of the housing bot uses a canned path that hides a citation failure. The real path fails one time in five. What do you demo?

  29. 29.A notebook that impressed the board is about to be named the production dosage assistant. It has no tests and no overnight owner. What did the board see?

  30. 30.Every time a customer pastes a hard prompt, their account manager escalates, and the roadmap stops. Nothing planned has shipped in a quarter. What do you give escalations?

  31. 31.A quick win to add citations needs a new index, a judge model, and three teams. The slide says one week. What do you call it?

  32. 32.Hack week is the same week as the committed launch of the clinic bot. Someone says hack week is the culture. What moves?

  33. 33.The postmortem on invented clinic hours produced three actions. None are on a quarter, and the same invention returned. What are those actions now?

  34. 34.The month's budget for bad answers is already spent. A new assistant is still scheduled for Monday because it was announced. What is the budget for?

  35. 35.The agent can send email as the user. The security review of that tool is unread, and feature work continues because security is not a feature. What outranks the polish?

  36. 36.Privacy review of the hospital summary starts the week of launch, and it can block the launch. When should it have started?

  37. 37.The school enrollment assistant's voice and screen flows are inaccessible, and that work is called phase two. Families have to use it this month. What is phase one?

  38. 38.Addresses outside the domestic form are out of scope. The first clinic customer is in a country whose addresses do not fit. What is in scope?

  39. 39.The multimodal museum guide would double the GPU bill at the holiday traffic you expect. The launch review has accuracy and no bill. What do you bring to that review?

  40. 40.You plan to turn off model v3 of the parking bot on Monday. One city still calls v3 and was not told. Their citations would fail. What has to be true first?

  41. 41.One director wants a paper-quality reranker. Another wants support deflection this month. You have one team, and both are called the top priority. What do you force?

  42. 42.An acquired clinic's model must be integrated in thirty days. You have not seen their chart schema. When do the thirty days mean anything?

  43. 43.A prompt bash the day before launch finds thirty bad answers, including wrong clinic hours. What decides whether you ship?

  44. 44.Every Friday the library bot retrieves nothing for a class of titles, and someone pastes a hotfix prompt. There is no ticket for the cause. What is the Friday ritual?

  45. 45.Planning week for the next assistant is interrupted by a live bot issuing duplicate refunds. What owns the people it needs?

  46. 46.This morning the judge eval failed on refund refusals. The board readout is in an hour, and the slide says on track. What do you say?

  47. 47.You know on Monday the label vendor will miss, and the launch cannot hold. Steering review is Friday. When do you say it?

  48. 48.An all-hands announced a ship date for the tax bot before an eval existed. Leadership now treats that date as the team's. What do you correct?

  49. 49.An engineer is polishing the tone of a settings blurb while a citation bug on the housing bot is open and assigned to them. What do you redirect?

  50. 50.Model-change reviews take five days. The plan assumes a prompt merges the day it is written. What is the schedule actually made of?

  51. 51.Five assistants are in progress. Each is about 20 percent done, and none can answer a user. What do you finish or stop?

  52. 52.Six months of pretraining have not beaten the vendor model on the eval you trust. The next month would cost the whole team. Someone says the six months cannot be wasted. What is already spent?

  53. 53.The platform team wants your embeddings moved this quarter. The enrollment assistant does not need that move to launch. What do you put on one list?

  54. 54.Two teams call the same embedding service. When it returns stale vectors, each pages the other. What is missing?

  55. 55.Someone offers to retrain over the weekend so a date holds, after scope was added midweek. You are about to thank them at the all-hands. What do you remove along with the thanks?

  56. 56.A vendor API would answer the parking questions well enough. An engineer wants a model you own, at the cost of the quarter. What belongs in the decision?

  57. 57.The tokenizer on the critical path of the newsroom model is unmaintained, and a broken upgrade is the likely next incident. What is the plan?

  58. 58.An internal eval tool would save an hour a day. The live assistant is quoting a wrong price on a common question. What decides which one you do?

  59. 59.The only model card for the dosage assistant is in one person's head, and they go on leave tomorrow. The card is scheduled for after launch. What is the launch missing?

  60. 60.The runbook for a bad answer says to call Priya. Priya goes on leave tomorrow. What does the next person need instead?

  61. 61.Mobile is deferred for the transit bot, and half the riders ask from the app. The sprint changes only the desktop bot. What did you defer?

  62. 62.The assistant misses its latency budget on the question people actually ask, and marketing has a date. What is that slow path?

  63. 63.The weekly update reports GPU hours. It does not say what a rider can do now that they could not do last week. What is the progress?

  64. 64.A law will require you to tell applicants when the housing assistant made a suggestion. The date is real. The quarter is locked around a tone refresh. What outranks the lock?

  65. 65.The team wants a tone rewrite before the legally required disclosure, because the current prompt is ugly. The disclosure date is fixed. What ships on that date?

  66. 66.Mid-quarter, a director adds a second assistant and says to absorb it. The team is already at capacity. What does absorb mean in your reply?

  67. 67.In the steering meeting you say the second assistant fits. Afterward you tell the team it does not. Where did the conflict land?

  68. 68.A partner asks for citation support. You say no, with no constraint and no time when it could be considered. What does a useful no include?

  69. 69.The roadmap slide is a list of model names. It has no owners and no order. What is that slide?

  70. 70.A blog post says the clinic bot is coming next Tuesday. The team did not confirm that date. What do you do the day you see the post?

  71. 71.One estimate for the index rebuild is three weeks. Another is seven. The gap is whether the backfill is included. You are about to tell leadership five weeks. What should you do with the gap?

  72. 72.The team skipped a spike on whether the charts can be retrieved at all. The project is now late for a reason nobody can name. What do you run?

  73. 73.The app is being built before anyone has written what the benefits bot must refuse. Designers are inventing answers the model may not give. What comes first?

  74. 74.The old and new fare models have both been serving for months, with no end date and two on-call paths. What belongs on the calendar?

  75. 75.The bot's response drops a field the city's app requires. The change is shipped as a patch. Callers fail. What should a breaking response have been?

  76. 76.The benefits model can change a payment amount. The only rollback plan is to fix forward. What does a payment change need before it ships?

  77. 77.You freeze all model changes for a week to focus, and the week has no single outcome. What does that week become?

  78. 78.The stakeholder meeting adds three ideas for the museum guide and writes down no decision. What do you leave the meeting with?

  79. 79.The design review of the support agent adds a refund tool and removes nothing. The date stays Friday. What does the new tool require in that same conversation?

  80. 80.You have eight people. Three are on-call, interviewing, or out. The plan counts eight full builders. What do you subtract before you promise a date?

  81. 81.Three people are out the week of the launch eval. The plan does not mention them. Someone suggests cancelling the vacations. What should happen instead?

  82. 82.Your own training run is on the critical path of the clinic launch, and you also owe a hiring loop. The run is blocked on you. What do you get off?

  83. 83.The model review board meets monthly. Your change is due in a week, and you remembered the board today. What do you do?

  84. 84.The backfill of 40 million embeddings is described as a script someone will run. It is not in the estimate. What is that script?

  85. 85.After this sprint, the web assistant and the app would show different transit times, because only the web model is changing. What do you add, or what do you refuse?

  86. 86.You commit to Tuesday in the meeting and privately assume Friday, so you look reliable. The team hears Tuesday. What will leadership learn to do with your dates?

  87. 87.Every model date you give is tripled, including a one-line prompt change. When a date is actually tight, nobody believes you. Where does padding belong?

  88. 88.The partner team misses the label delivery by two weeks and asks you to be flexible, while your launch date stays. What can flex?

  89. 89.This week's status says integration is in progress. It does not mention that the partner's labels slipped. What does that phrase do?

  90. 90.Three teams must turn on one assistant the same morning. Nobody owns the go or the no-go. What do you name before that morning?

  91. 91.The new refusal policy is behind a flag that is off. Marketing already emailed clinics that the policy is live. What was early?

  92. 92.Someone wants the flag on for every clinic so the email is not wrong. The refusal eval has not passed. What do you refuse?

  93. 93.One hospital wants a custom model that would fork the discharge assistant. They will pay for the quarter. What is that fork if you take it quietly?

  94. 94.The quarterly review is a demo of what you built. Nobody chooses what stops. What should that review spend its time on?

  95. 95.Leadership asks for a date on an AI that handles enrollment. There is no success metric and no list of decisions the bot must not make. What do you write before any number?

  96. 96.An engineer says maybe six weeks in a hallway. The next day it is a deadline. What do you label out loud the next time a number comes up?

  97. 97.You split your strongest engineer across four assistants so all of them move. What actually moves?

  98. 98.People say the team just gets prompt requests, and each request feels small. Nothing on the roadmap is finishing. What do you count before you cap anything?

  99. 99.A director messages engineers directly every day for a one-off prompt on the live bot. The sprint changes without you seeing it. Where should those requests go?

  100. 100.You take every side prompt yourself so the team can focus. Your hiring loop and 1:1s slip, and leadership thinks the side prompts are free. What is hidden?

  101. 101.The clinic bot ships early. You assign a new pretrain the same day so nobody is idle. The pretrain cannot finish or stop cleanly this month. What was the early finish?

  102. 102.Two prompts both pass the eval. The team has debated wording for a month. What do you do this week?

  103. 103.A senior missed the review that chose the retrieval depth. They want the decision reopened. There is no new measurement. What do you share with them?

  104. 104.The vendor you chose cannot meet the privacy term you required for clinic notes. The choice was celebrated last month. What is that choice now?

  105. 105.You present three model options and no recommendation, hoping the room will converge. The room leaves with no owner. What did the meeting need from you?

  106. 106.The team votes to ship housing suggestions without the required disclosure, because the disclosure feels clunky. The disclosure is not optional. What should the vote have been about?

  107. 107.You tell executives the team is slow. The quarter was four assistants, plus daily one-off prompts, and no budget for those prompts. What is the diagnosis you owed them?