FreeExam

People

An AI/ML engineering manager interview on hiring, feedback, and what you owe someone before a rating.

These prompts are original practice. They are not questions from a named employer or a live hiring loop.

  1. 1.The hiring debrief for a clinic scheduling assistant is split. Two interviewers say the candidate never finished the retrieval exercise. A third says they were warm and felt like a lead. What decides the hire?

  2. 2.A senior candidate for the pharmacy refill assistant will join only if they never take model-quality pages, because seniors design. What do you tell them?

  3. 3.A promotion packet for the ferry-schedule model lists GPU hours and notebooks opened. It does not say what changed for riders. What do you ask for before you submit it?

  4. 4.Two engineers argue in the eval review of the museum audio guide. One calls the other's test set a toy. They stop speaking. What do you do this week?

  5. 5.The person who built the judge harness for the tax-help bot rarely speaks. The presenter, who did not write it, is the name leadership repeats. Promo notes are due. What do you put in?

  6. 6.You have cancelled three 1:1s because a training run for the rec-center recommender needed the room. What is the 1:1 for?

  7. 7.The new hire's buddy is the only person who can restart the trainer for the blood-drive routing model, and that person is on-call. What do you change?

  8. 8.Your strongest prompt engineer on the dental-office bot leaves review comments like 'did you even look.' Their answers are the ones the clinic keeps. What is the performance issue?

  9. 9.An engineer knew on Tuesday that the housing assistant had started inventing deposit amounts. They mentioned it at Friday's demo. What do you coach?

  10. 10.You used to train the model. You still rewrite people's notebooks for the library search assistant after they go home. What does the team learn?

  11. 11.Your skip-level says the recommendation team is fine. That afternoon a report says they are interviewing, and asks you not to be angry they spoke to your manager. What is the signal?

  12. 12.Two seniors want to own the next base model for the newsroom summary tool. Splitting the files in half would leave neither with a staff-sized problem. What do you decide?

  13. 13.Recruiting says the req for the benefits assistant expires Friday, and asks you to skip the eval exercise just this once. What do you protect?

  14. 14.An interviewer rejects a candidate for the transit-delay bot with 'not a culture fit' and no work example. What do you do with that scorecard?

  15. 15.A teammate sent the actual retrieval questions to a friend who is on tomorrow's loop for the warehouse-pick assistant. What happens to that loop?

  16. 16.The take-home for the veterinary triage assistant is a full fine-tune, so you can see real skill. Candidates say it takes a weekend and a spare GPU. What do you change?

  17. 17.The debrief for the parking-citation assistant is a speech by the most senior interviewer. The written notes disagree with the speech. How do you run the decision?

  18. 18.You want to hire a former report onto the school-lunch menu model. They were strong for you two years ago. The current team has not interviewed them. What is required?

  19. 19.The offer for a strong eval engineer is below their current pay. Compensation says the band is closed. They ask you to promise a raise in six months. What can you say?

  20. 20.An engineer on the dosage-check assistant asks if they will be promoted this cycle if they keep the model green. What do you answer?

  21. 21.In calibration you rank your senior above another team's staff engineer, who took a clinic model from a leaked eval to a monitored launch. Your senior's loss is lower on a private set. What comparison is fair?

  22. 22.You mark every person on the assistant team 'exceeds' so the 'meets' conversation feels kinder. What does that do?

  23. 23.A draft plan for an engineer on the claims model says 'be more innovative and more proactive.' What has to be rewritten?

  24. 24.The plan said the claims-model engineer would cut a specific false-approve rate, and they did. Teammates still will not pair with them because of review comments you never put in the plan. What is fair?

  25. 25.You are ready to dismiss the engineer who shipped a prompt with a live API key to the public demo, this Friday at 4pm. HR has not seen notes. What do you do?

  26. 26.A teammate tells you a senior modeler mocked them in sexual terms in the eval channel. You want both people in your office to clear the air. What is your job?

  27. 27.An engineer asks to shift their on-call week for the triage model because of a medical appointment. You ask what the diagnosis is so you can plan fairly. What should you have asked for?

  28. 28.Someone on the enrollment assistant tells you they are pregnant. A teammate proposes moving them off the launch to be kind, without asking them. What do you do?

  29. 29.A new grad will not page when the school-enrollment bot gives a fabricated deadline, because they are afraid the page is wrong. What do you change?

  30. 30.A senior on the audio-guide model refuses junior reviews because the diffs are slow and the loss is what matters. What do you require?

  31. 31.One engineer is proud that nobody else can reproduce the newsroom model's training recipe. Last month they were out and a retrain was impossible. How do you treat that?

  32. 32.Only one person can approve a model push for the ferry assistant, and they will be on a ship with no signal for two weeks. You notice the calendar today. What is already late?

  33. 33.The prompt for the benefits bot is chosen in a Slack thread after 10pm. People who log off miss the decision and get blamed at standup. What do you move?

  34. 34.The one remote engineer on the eval review is talked over because the room shares a laptop mic. Their notes show up later in the doc, unread. What do you change?

  35. 35.The engineer in another time zone gets every dawn page on the triage model because they are already online. The others have not taken a model-quality page in two months. What is the rotation for?

  36. 36.You inherited the summarization team. They answer in one word and quote the last manager's broken promises. What rebuilds trust?

  37. 37.A reorg note that moves the model team is in a public channel before you have talked to them. What do you tell them today?

  38. 38.A neighboring research group is cut. Your team asks if they are next. You do not know. What do you say?

  39. 39.Your best eval engineer has an offer from a lab. You want to counter with a staff title you have not been given the right to grant. What can you do in the stay conversation?

  40. 40.The engineer says the outside offer pays more, and that the work is boring: they only refresh prompts for the parking bot. Compensation can move a little. What do you talk about first?

  41. 41.Three exit notes this quarter name the same senior on the retrieval team, who dismisses other people's evals in public. You have called that senior your force multiplier. What are the exits?

  42. 42.An engineer wants to move to the platform team. If they leave this month, the clinic launch slips. You can block the transfer in the tool. What do you do?

  43. 43.An internal transfer arrives on the summarization team. A hallway comment says they exaggerate metrics. You have not seen the work. How do you use the comment?

  44. 44.The onboarding plan for the new hire is 'read the model wiki.' The last edit is two years old, and the current assistant uses a different index. What is a first-week plan?

  45. 45.By week three, the intern's reranker cannot beat the baseline on the frozen set you care about. Their review is 'did it ship an improvement.' What do you change while there is still time?

  46. 46.The mentor says the intern is not getting a labeling-quality project in week two, and wants them removed. What do you ask before any move?

  47. 47.Two seniors disagree about chunking for the housing assistant. The review thread is now about who is out of their depth. What do you stop first?

  48. 48.You agree with senior A on the chunking, and you have been quiet while they mock senior B in the thread. What is your silence?

  49. 49.An engineer cries in a 1:1 about a family crisis and asks what you would do at home. What is in bounds?

  50. 50.The engineer who owns the judge model says yes to every new prompt request and is online past midnight. You have praised the yes in two retros. What did the praise reinforce?

  51. 51.A training run was watched at 1am all week because nobody would pause it. You are about to call that dedication in the retro. What should the retro say?

  52. 52.A parent leaves at 4:30 after their model change is in and monitored. A teammate jokes that they are part-time. What do you do in the room?

  53. 53.At the director demo, the presenter of the enrollment bot is thanked for the refusal policy. The policy was written by someone else, who is in the room. What do you say there?

  54. 54.You take an engineer's model-card draft to the director and present the findings as your own. What does the team learn?

  55. 55.The goal sheet for each person on the assistant team has twelve objectives, including 'help AI' and 'be present.' What should a goal sheet be able to do?

  56. 56.An engineer asks for feedback on their eval writeups. You say 'keep doing what you are doing.' What did they not get?

  57. 57.The first time an engineer hears that their test set was contaminated is in the annual review, eight months later. What was the review supposed to be?

  58. 58.You write that a scientist needs more executive presence in the launch review of the dosage model. They ask what to practice. What can they practice?

  59. 59.You paste a harsh note in the team channel about a bad prompt, so everyone learns, and the author is named. Where does that feedback belong?

  60. 60.You start to tell an engineer their judge scores do not match the human labels. They get defensive, and you end with 'it's probably fine.' What did you owe them?

  61. 61.A product manager says the scientist on the triage bot is impossible. You are about to repeat that word in the 1:1. What do you need first?

  62. 62.The scientist says the product manager changes the success metric of the housing bot every morning: citations, then shorter answers, then more human. You tell the scientist to be flexible. What is actually yours?

  63. 63.You and the staff scientist disagree on whether the newsroom model should refuse more often. You say you are the manager, so it will refuse more. What is the title in that argument?

  64. 64.You stopped reading eval notes because you are no longer the modeler. A launch review asks whether the test set was carved from the training data. What do you still owe?

  65. 65.The weekly model meeting is an hour of people reading loss numbers that are already in the tracker. Decisions get cut at the end. What is the meeting for?

  66. 66.Delivery of the clinic bot has been so busy that you have not had a career conversation in six months. The burndown looks fine. What do you put back?

  67. 67.Your most senior modeler wants to become the manager because they are the most senior. They dislike giving feedback. What do you offer before any title?

  68. 68.An engineer is ready for staff-level work. The only staff-sized problem is the base model, and its owner will not share a slice. What is the block?

  69. 69.The promotion committee meets next week. You have stories about the eval engineer and no written record of what their harness caught. What do you submit?

  70. 70.The quiet engineer caught a train-test leak that would have shipped. The presenter, with a thinner change, is who leadership names. Both are up for the same promotion. What do you write?

  71. 71.You review every prompt change to stay sharp, and the queue waits on you for two days. What do you keep?

  72. 72.The team is tired. You promise two openings you have not been given, to give them hope during the assistant rewrite. What should you have said?

  73. 73.Four interviewers for the retrieval role all came from the same lab as the last three hires. A candidate from a product team is scored down for different instincts. What do you change?

  74. 74.An interviewer marks a candidate down for not listening on the trainer design. The notes show the candidate caught a bug in the interviewer's setup, and the interviewer argued. What do you ask?

  75. 75.You need someone who has shipped a trainer. The only interviewer who has is on leave, so a generalist runs the loop and says the candidate seemed smart. What is that loop worth for this seat?

  76. 76.The candidate's reference is a friend who coauthored a workshop paper and did not manage them. They offer no one who saw production work. What is that reference?

  77. 77.Someone messages you 'do not hire them' about tomorrow's candidate and will not say why or stand behind it. Where does that message go in the debrief?

  78. 78.The engineer who improved the judge is regularly late to on-call handoff and leaves no note on which prompts are mid-incident. What is the feedback?

  79. 79.Two engineers ship the same reduction in bad answers on the benefits bot. One worked visibly longer. You are about to score the longer hours as more impact. What is the impact?

  80. 80.The same two people take every 'the model is wrong' page because they are better at traces. The others have never practiced. What are you building?

  81. 81.A new hire ships a prompt that leaks internal policy text. You fix it before they see the ticket, and you say nothing. What do they learn?

  82. 82.The page is a harmful answer from a prompt that has been in production for six months. You assign the redesign to whoever was on-call that night, as the consequence. What do they own?

  83. 83.The retro item 'align research and engineering' has been copied for four quarters. Nobody owns an action. What do you do with it this quarter?

  84. 84.The real choice of which model to ship is made at dinner. A parent and the contractor who labels failure cases are never there, and they hear the choice the next day as a fact. What do you move?

  85. 85.You hear an engineer is interviewing. You take away their GPU allocation so the launch is less distracted. What is that?

  86. 86.An engineer who trained the reranker says they are bored of ticket labeling. You assign them more labeling to build character. What is the boredom telling you?

  87. 87.The same person gets every new assistant, and the others maintain prompts. You cannot say out loud why the split is fair. What do you change?

  88. 88.Feature-store chores land on whoever complains least. The quiet engineer now owns all of them, and their model work has stopped. What is maintenance?

  89. 89.A junior's loss has been NaN for three days, and they did not ask. You are annoyed they stayed stuck. What else is true?

  90. 90.A senior says they should not have to explain why the new summary model is compared with last quarter's model. The review cannot tell if the gain is real. What is the senior work here?

  91. 91.You hold 1:1s with the person whose model is in incident, and you skip the person whose assistant is fine. What does the skip train?

  92. 92.A tech lead manages the retrieval group. You disagree with a chunk size, so you tell their engineer to change it in the hallway and not bother the lead. What did you spend?

  93. 93.An engineer asks you to keep their job search private. You mention it in staff meeting as a delivery risk. What did you owe them?

  94. 94.You want a new team charter. The assistant team has ignored three in two years. What is more useful than a fourth poster?

  95. 95.The team has argued for a year about notebooks versus packages, and every launch pays the argument again. What do you do?

  96. 96.An engineer leaks a test slice into training. You say 'no worries' and move on. At review you cite it as the reason for a low rating. What was unfair?

  97. 97.You keep no notes on expectations because this team is high trust. At calibration you cannot show why one person exceeds and another meets. What is the trust not a substitute for?

  98. 98.A director asks in the hallway who your weakest modeler is, before a meeting. What do you answer?

  99. 99.You are told to fit ten people to a curve that requires two below-expectations ratings. The work does not show two. What do you submit?

  100. 100.A former contractor, now an employee, still gets the labeling leftovers. People call them the labeler in reviews of the judge they now own. What do you correct?

  101. 101.A teammate suggests keeping an engineer off the launch because their paperwork might be complicated. HR has not said there is a work restriction. What do you staff from?

  102. 102.You discover research-track and engineering-track people doing the same launch work are paid on bands you cannot explain. You did not set the bands. What do you still do?

  103. 103.An engineer wants a public writeup of a prompt injection that still works on the support bot. Legal wants it quiet until the mitigation ships. When is the public credit fair?

  104. 104.In the incident channel, your modeler and the data engineer blame each other for a stale index that served wrong prices. What do you require in the channel?

  105. 105.You have been the manager of this assistant team for four months. Nobody has a written note of what a good model change is: which eval, which rollback, which review. What do you write?