FreeExam

Transformers

An AI/ML engineer interview on attention, context, and what a transformer does to a request before you ship it.

These prompts are original practice. They are not questions from a named employer or a live hiring loop.

  1. 1.A support reply quotes HX and 449 for an order the customer wrote as one string, HX-4491-BB. The ticket on screen still shows the full string. The model never received a second copy.

  2. 2.The pharmacy labeler has never seen the brand Zelvori. Tokenization replaces it with the unknown token, and the printed name is that unknown token rather than the brand.

  3. 3.Support training placed a separator token between the subject and the body. Serving joins them with a space. The subject is Account closed, and closed is tagged as the body's resolution.

  4. 4.Warehouse training started every scan with a start-of-scan token. A firmware path sends the bare barcode. The lot, which used to be the field after that token, is read from the barcode's first digits.

  5. 5.Students were told that a lone pipe marks the end of an answer. One student uses pipes as fraction bars in the scratch work. The grader stops at the first pipe and scores only the fragment above it.

  6. 6.A claim lists the member's deductible as 500. The attached policy excerpt includes an example deductible of 250. No separator is placed between them, and the denial states the deductible as 250.

  7. 7.A 40-token legal banner is prepended to every support ticket. The product-name tagger still reads absolute position 6, where the product name used to sit. It now tags a word from the banner.

  8. 8.Two pharmacy directions are joined by a space into one line: take one tablet each morning take one tablet at bedtime. Bedtime is tagged onto the morning drug. Training had split directions on a newline token.

  9. 9.During fare training the chosen row was always the last row. The screen now sorts cheapest first. The highlight still lands on the last row, which is now the most expensive fare.

  10. 10.A pick string is the same lot code twice, A12 A12, meaning the near bin and then overflow. No other tokens are added, and position embeddings are turned off. The picker is given one lot and no order.

  11. 11.Tutoring turns are concatenated, and each turn restarts position ids at 0. A head resolves the first number by reading position 0. Both turns place 7 at position 0, and the tutor mixes the two mentions.

  12. 12.The claims position table was trained through index 1023. The injury date falls at token 1400. The pipeline clamps every later index to 1023, which is still a boilerplate line. The extracted date is a phrase from that line.

  13. 13.Serving divides every position index by 4 so an 8192-token span fits the trained range, and it leaves that division on for a 40-token refund ticket. Yes and refund, 8 positions apart in training, land 2 positions apart. The reply does not treat the yes as an answer to refund.

  14. 14.A crush warning sits several hundred tokens after the drug name. Attention adds a distance penalty, and the warning head keeps the nearby directions instead. The bottle omits do not crush.

  15. 15.A fare prompt lists the total and then a tax line. You read the hidden state at the word total in a causal model to build the price explanation. The explanation never mentions the tax.

  16. 16.A scan reads LOT A12 QTY 40. A bidirectional tagger ties 40 to A12. A causal model is read at A12 and reports the quantity as missing.

  17. 17.The tutoring screen needs the next hint as a sentence. The served model is encoder-only, and the screen shows a pooled score against old hints rather than a new sentence.

  18. 18.A denial letter is written by a decoder that cross-attends to an encoder. Only the claim form was encoded. The exclusion lives only in a policy file that was not part of that input. The letter approves an excluded repair.

  19. 19.Short support tickets are right-padded to the longest ticket in the batch. The intent head always reads the final index. Every short ticket is classified the way the pad token is classified.

  20. 20.Eight vial labels are packed into one sequence. No attention mask separates them. A 10 mg vial is tagged 50 mg, which is the strength of the vial packed just before it.

  21. 21.Fare training treated mask value 1 as attend. The server treats 1 as ignore. Real fare tokens are blocked, and the price is computed from positions that contain only padding.

  22. 22.Fine-tuning the tutor computes a loss on every position, including the pads used to square the batch. The model begins ending hints with the pad token.

  23. 23.A 40-token claim is padded to 2000. The denial vector is the average of every position, pads included. It sits close to the pad embedding, and the real denial phrase barely shows up.

  24. 24.A support window holds 2048 tokens. Standing rules take 400 and example replies take 1200. The 900-token ticket is cut from the left. The order id was in the first customer message, and the reply asks for it again.

  25. 25.Line 1 of a pharmacy request is an allergy to penicillin. Serving keeps the last 512 tokens of a long leaflet and drops the front. The suggested drug is in the penicillin family.

  26. 26.Fare rules are shortened by keeping the first 300 tokens and the last 300. The change fee appears only in the middle. The screen says the fee is not stated.

  27. 27.A warehouse budget cuts each scan at 200 characters and then tokenizes whatever remains. The cut lands inside a barcode. The surviving prefix equals a different SKU that is actually on a shelf.

  28. 28.A tutoring problem sits at the top of a long chat. The window keeps only the latest turns. The student asks what about question 4, and question 4's statement has fallen out of the window. The tutor answers the aside.

  29. 29.A claim header fits in the window and the notes do not, so the notes are dropped. The date of injury is written only in the notes. The queue stores a blank injury date.

  30. 30.A worker finishes support ticket 88421 and the next ticket reuses that KV cache. The new reply mentions 88421's order id. The new ticket text does not contain that id.

  31. 31.A pharmacy cache is built for a prompt that says 10 mg. The pharmacist then changes the middle of the prompt to 5 mg, and the tail of the cache is not rebuilt. Later generated tokens still say 10 mg.

  32. 32.A fare reply is halfway through a long rule sheet when a sliding cache drops the earliest keys. The departure airport, SFO, was in that dropped prefix. The total that follows names the airport in the latest sentence instead.

  33. 33.An hour after a scan, the picker asks which bin that same barcode resolved to. Those scan tokens are still in the cache. The answer matches the hour-ago scan. The model has no connection open to the inventory system.

  34. 34.A student says to forget 6 and use 7. The next hint still multiplies by 6. The earlier turn that contains 6 is still in the KV cache.

  35. 35.A policy lookup returns clause 4.2(b) to the review screen and the logs. The letter model's sequence is still only the claim form. The letter cites no clause.

  36. 36.A support body says the crib arrived broken, but only after a long greeting. A refund decision is read from the first token of a causal model. The score says no refund.

  37. 37.Each pharmacy token needs a tag: drug, strength, route, or frequency. The product does not need a new sentence. A decoder-only model writes a paragraph and invents a route the label never stated.

  38. 38.A long fare rule must become a short JSON card, and each field of the card should be able to look at the whole rule. The decoder in serving never cross-attends. It only sees words it has already written, and the total copies the previous field.

  39. 39.When a warehouse scan disagrees with the expected bin, the product must write one exception sentence. The deployed model returns only the nearest bin id from an encoder embedding. The sentence is blank.

  40. 40.Long tutoring sessions drop a 700-token preface that says to give one hint and then a check question. The student's problem is still in the window. The tutor writes the full solution. Which change keeps that style without spending 700 tokens every session?

  41. 41.Texas denial deadlines were fine-tuned into the claims model. An Ohio claim is sent with no state token. The letter uses the Texas deadline.

  42. 42.Order 18's transit status changes through the day. Last night's status was fine-tuned in, and the status was removed from the prompt. At noon the reply still says the crib is on the truck, though noon shows delivered.

  43. 43.A perfect example label at the front of a pharmacy prompt shows the field order: name, then NDC, then directions. The leaflet fills the window and the example is cut. The NDC digits remain in the leaflet, but the printed label skips the NDC line.

  44. 44.Fare fine-tuning saw itineraries of at most 400 tokens, with the promo always near the start. Serving places a promo at token 2500 of a long schedule. The base model can hold that many tokens, and the promo is ignored.

  45. 45.Yesterday's aisle map was fine-tuned into the warehouse model. This morning the bin moved. The request sends the beep and does not send the new map. The picker is sent to yesterday's bin.

  46. 46.Shortened hints for one student's accommodations are fine-tuned into the shared tutor. The next student has no such note in the prompt and still receives the shortened hints.

  47. 47.Twelve sample denial letters fill the claims window, so this member's exclusion never fits. You fine-tune the letter format, drop the twelve samples, and the exclusion then fits. The letter quotes it.

  48. 48.A support collator trims every ticket in the batch to the shortest ticket's length before encoding. A two-line status question stays whole. A long refund ticket loses the amount at the end and is scored as a greeting.

  49. 49.A pharmacy strength head reads the last index. Training left-pads so the last real token sits on that index. Serving right-pads. Short labels come back with an empty strength.

  50. 50.Fare itinerary A and itinerary B are concatenated for one forward pass. No mask separates them. B's card names A's connection city, Denver, which is not on B's route.

  51. 51.Tutoring sessions are right-padded to the model maximum. The grader reads position max length minus 1. A short session's last real token is near the start, and the score matches the pad embedding.

  52. 52.A claims diagnosis head expects the code at absolute position 20. That was true when claims were left-padded to a long fixed length. In a shorter bucket there are fewer left pads, the code lands at position 8, and position 20 is pad. The head still reads position 20.

  53. 53.Support priority is taken from layer 2 at the subject token, and later layers are not used for this tag. The subject is Question. Two hundred tokens later the body says the child is locked out of the account. The state at the subject is causal. The tag says routine.

  54. 54.The pharmacy check uses one local attention window of 64 tokens. Allergy: sulfa is 400 tokens before the drug being checked. The check approves a sulfa drug.

  55. 55.A fare price head receives a pool of only the first 16 token states. The cabin code is token 40. The screen prices economy for a business cabin.

  56. 56.A scanner dashboard shows 8 C for a cold-chain pallet. That temperature is not added to the model input. Spoilage instructions depend on it, and no attention head mentions a cold-chain warning.

  57. 57.A tutoring preprocessor deletes numeric tokens as noise. The student answered 0.9. The grader then sees the surrounding words and has no numeric answer left to grade.

  58. 58.Every claim ends with the same 800-token legal footer. You average all non-pad tokens for an approve-or-deny head. The exclusion is one sentence. The decision follows the footer and misses the exclusion.

  59. 59.A support ticket says cancel and, later, the subscription, not the order. You classify at the token cancel in a causal model. The action taken cancels the order.

  60. 60.A sig reads take 1 tablet by mouth each morning. The route should be read only once the whole sig is in view. The route head instead reads the causal state at tablet, and the route comes back blank.

  61. 61.A fare card decoder used to cross-attend to a rule encoder. Serving now leaves only the decoder's self-attention. The total repeats the previous JSON value and ignores a tax rule that is present in the encoder input.

  62. 62.The warehouse note is generated after a boundary token, and it can look back only through tokens already placed before that boundary. Serving puts the boundary before the bin code and starts the note there, leaving the bin out. The note says the bin is unknown.

  63. 63.The tutoring product needs a tutor turn after the student. Serving uses a bidirectional denoiser. It rewrites the student's equation in place and never adds a tutor turn.

  64. 64.A claims template reads reason, then a blank, then clause: excluded if the injury predates the policy. The blank is filled from the causal state at the blank. The reason comes out generic and never mentions the timing.

  65. 65.A support gate allows 512 tokens but counts 512 characters of SKU-heavy text and lets the ticket through. The text is about 900 tokens. Error code E-17 is at the end and is cut by the real window.

  66. 66.Pharmacy training keeps 10mg as one token. Serving tokenization yields 10 and mg as two tokens. A rule that steps one token past the drug name used to land on the unit and now lands on 10. The unit is recorded as missing.

  67. 67.The fare amount $1,284.50 is split across several tokens. A span rule that accepts only a single amount token finds no match. The screen shows a blank price.

  68. 68.A control character before a warehouse barcode is mapped to the unknown token. Field indexes were hard-coded on the assumption that the barcode starts at index 0. The lot is read from the quantity's tokens.

  69. 69.A student writes 8 × 7. The multiplication sign is not in the vocabulary and becomes the unknown token. The tutor explains a missing operation and adds the two numbers.

  70. 70.The diagnosis code E11.9 is split into several tokens. A lookup embeds one token per whole code and finds no neighbor. The field stays blank even though the pieces sit in the claim.

  71. 71.A CLS token is prepended to each support ticket for intent. The serving mask lets CLS attend only to itself. A ticket that clearly asks for a replacement is scored as empty.

  72. 72.Pharmacy fine-tuning inserts one DRUG marker token before each name. The serving tokenizer has no such token and splits the brackets into ordinary characters. The strength head searches for the marker id and finds none.

  73. 73.Fare training marked the cities with an origin token and a destination token. Serving passes SFO JFK with neither marker. The card swaps the two cities.

  74. 74.A double beep inserts an end-of-scan token in the middle of a warehouse barcode. The decoder is set to stop on that token. The bin, which follows the barcode, is never generated.

  75. 75.A worksheet used to fill the blank in the sentence 'the slope is [MASK]' with a masked model. Serving now uses a causal tutor. The product still reads that mask position for the slope, and the blank is unchanged.

  76. 76.The billed amount 4200 is on the claim form. The pipeline still places a mask token over it, the way it did when training a filler. The model writes 3000, a common training amount, and the queue shows 3000.

  77. 77.A support budget keeps the start of the ticket and cuts the end. What remains is Hello, I hope you are well, and the question is past the cut. The reply wishes the customer a nice day.

  78. 78.A pharmacy NDC is kept from the left under a tight token budget. The budget ends in the middle of the NDC's tokens. The printed code is that surviving prefix, and it matches a different real NDC.

  79. 79.A fare session holds a system line, ten earlier searches, and one new search. The window keeps the system line and the earlier searches. The new search's date does not fit, and the screen prices an earlier date.

  80. 80.Token 0 of a warehouse scan is a hazard flag. The keeper retains the latest scanner chatter and cuts from the left. The picker is not warned about the hazard.

  81. 81.A worked tutoring example that ends at 42 stays fully inside the window. The student's own problem is cut in half. The tutor answers 42.

  82. 82.An exclusion sits on page 40 of a 100-page policy. The request keeps the first 1000 tokens and the last 1000. Page 40 falls in neither slice. The letter says the policy states no exclusion.

  83. 83.A support system prefix says the return window is 30 days, and that prefix is cached. The prefix text is edited to 14 days. The next ticket reuses the cached prefix. The reply quotes 30 days.

  84. 84.Generation of one vial's label is halfway done. The cache was created for this request and is not shared. Which earlier tokens can the next token attend to?

  85. 85.Traveler A's cached prefix includes the promo SAVE20. Traveler B reuses that prefix and did not enter a promo. B's fare screen applies SAVE20.

  86. 86.A warehouse tool returns bin C-14, and the transcript log shows it. The cache used for the next spoken word still ends at go to, with no keys appended for C-14. The spoken bin is wrong.

  87. 87.Tutoring turns 1 through 3 are cached. The equation in turn 2 is corrected. Turn 3's keys were computed from the old equation. The turn-4 hint follows the uncorrected equation.

  88. 88.A summary model truncates a claim's exclusion. The letter model cross-attends only to that summary. The letter cannot cite an exclusion the summary does not contain.

  89. 89.Reply openings were fine-tuned on store 14, which always starts with a greeting. A store 15 ticket says skip the greeting, and that line is inside the window. The draft still greets.

  90. 90.A one-line formulary flag would say a drug is discontinued. The pharmacy request instead pastes a 2000-token March monograph to carry that flag. The directions at the end no longer fit, and the label has no sig.

  91. 91.Last month's sale price was fine-tuned into the fare model. Today's price is in the prompt and inside the window. The screen still shows the sale price. Which check belongs on the screen before the price is shown?

  92. 92.Today's aisle row is inside the warehouse window, 200 tokens after the beep. The decision layer's mask lets it attend only to the beep span. The picker is sent to a fine-tuned bin, not to the bin in the aisle row.

  93. 93.Grade-6 hint style was fine-tuned and still appears. The tutoring problem was cut from 2000 tokens to 300. The tutor writes a grade-6 hint about the remaining fragment and never addresses the cut steps.

  94. 94.A claim's state code is token 4, inside the window. The deadline sentence is generated by a head masked to positions 100 and later, where the boilerplate begins. The letter uses a generic deadline.

  95. 95.Support training places the end-of-ticket token right after the last real token. An attachment head looks only at position 511, the old fixed length. A short ticket has its end token at position 40 and pads after that. The head says the attachment marker is absent.

  96. 96.Pharmacy training puts the target sig in the same sequence as the source and uses an all-ones attention mask. The position that should predict the strength can see the strength later in the target. At serving, generation is causal and the strength comes out wrong.

  97. 97.A fare model was trained with positions through 1024. Serving accepts 4096 tokens and feeds the extra positions through embedding rows that were never trained. A bag fee sits in that untrained tail and comes back as nonsense.

  98. 98.A warehouse pick list has a global token that was supposed to see every bin. The decision is wired to a local layer instead. The right bin is 500 tokens away, and the decision says there is no matching bin.

  99. 99.An answer key is prepended to a tutoring attempt. The grade is taken from a local head on the student's last line, with a window of 32 tokens. The key is 800 tokens earlier. The grade uses the scratch work only.

  100. 100.The claims model is served with a smaller set of heads. The heads that used to attend from the decision token to dollar amounts are not in the set you kept. Dates, which the kept heads do attend to, come out right. The dollar amount is wrong.

  101. 101.A customer sends a screenshot of error E-17. The support model is a text transformer and receives the caption screenshot, with no OCR text. The reply asks what the error says.

  102. 102.A pharmacy dose head was trained on 0.5 as one token. A label writes 0,5, and that string splits into separate tokens. The printed dose is 5.

  103. 103.Sunday's requested flight and an example Saturday itinerary are concatenated with no separator token. Segment ids were the only marker between them, and serving drops those ids. The card offers the Saturday flight.

  104. 104.Warehouse scan 2 is packed immediately after scan 1, and positions keep counting. Scans were trained to start at position 0. Scan 2's lot sits at position 80. Scan 1 is read correctly, and scan 2's lot comes back empty.

  105. 105.The tutor's next token comes from one attention layer with a 256-token window. No earlier layer has brought the original equation into that window. The equation is 1500 tokens back. The student says that equation, and the reply uses the numbers in the latest message.

  106. 106.Claims training always had a 100-token boilerplate header, and the member id sat at position 100. One claim arrives with no header. Position 100 is now a billing code. The id field shows that billing code.

  107. 107.The support chat template needs a beginning token before the agent reply. A retry path drops that token. The model continues the customer's last sentence instead of starting the reply.

  108. 108.A fare screen only needs change allowed or change denied for one rule paragraph. The deciding words are a negation at the start and an exception at the far end, and no new sentence is required. The class is read from the first token of a causal model. The paragraph 'not changeable, except same-day' is scored as a flat denial.