FreeExam

ML platform code

Write the code that ships and watches models: registry stage rules, canary and weighted routing, shadow diffs, rollback triggers, PSI and drift checks, point-in-time feature joins, lineage, cache keys, config diffs, GPU scheduling, batching, latency percentiles, error budgets, guardrails, and autoscaling.

These problems are original practice. They are not questions from a named employer or a live hiring loop.

  1. 1.Registry stage transitions

    Write apply_transitions(requests); each is [version, target], applied in order. Versions start in "None". Legal: None→Staging, Staging→Production, Staging→Archived, Production→Archived, Archived→Staging. Any other move (even to the same stage) is rejected and changes nothing. A new Production version archives the old one. Return {"stages": {each requested version: stage}, "rejected": [indices]}.

  2. 2.Promote best candidate

    Write promote_best(candidates, production_score, metric, min_gap). Each candidate is {"version": str, "metrics": {name: value}}; higher is better. Skip candidates without the metric. The best is the highest value; on a tie the earliest in the list wins. Return its version if production_score is None or best − production_score >= min_gap; otherwise, or with no usable candidate, return None.

  3. 3.Latest semantic version

    Write latest_version(versions). Each string is MAJOR.MINOR.PATCH with an optional leading "v" and an optional "-rcN" suffix (N an integer), e.g. "v1.10.0-rc2". Compare the three numbers numerically, in order; a release ranks above any of its rc builds, and rc builds compare by N. Return the original string of the latest; if several are equal, the first one; for an empty list, None.

  4. 4.Deterministic canary routing

    Write canary_route(user_ids, percent). A user's bucket is the sum of the character codes (ord) of their id, mod 100. The user goes to "canary" if bucket < percent, else "stable". percent is an integer from 0 to 100. Return the list of routes in input order.

  5. 5.Weighted traffic split

    Write split_traffic(request_ids, weights). weights maps arm name to a non-negative integer; total = their sum (> 0). Sort arms by name; they own consecutive slot ranges from 0 sized by weight (weights {"a": 2, "b": 1} give a slots 0–1, b slot 2). A request's slot is the sum of its id's character codes mod total. Return {arm: [request ids in input order]} for every arm, even empty ones.

  6. 6.Shadow deployment diff

    Write shadow_diff(primary, shadow, tol) for two equal-length lists of predictions from the live and shadow model. Position i matches if abs(primary[i] − shadow[i]) <= tol. Return {"mismatches": [indices that do not match, ascending], "match_rate": matched / length}. For empty lists match_rate is 1.0.

  7. 7.Rollback from error windows

    Write rollback_at(baseline, windows, factor, k). windows lists the new version's error rate per time window, in order. A window breaches if its rate is strictly greater than baseline × factor. Return the index of the window at which k consecutive breaching windows are first reached, or -1 if that never happens.

  8. 8.Population stability index

    Write psi(expected, actual, floor). Both are bin count lists of equal length with positive totals. Turn each into proportions (count / its total), then replace any proportion below floor with floor (no renormalizing). Return PSI = sum over bins of (a − e) × ln(a / e), where e is the expected and a the actual proportion.

  9. 9.Feature drift by mean shift

    Write drift_flags(reference, current, threshold). Both map feature name to a list of numbers. For each feature in reference whose current list exists and is non-empty, shift = abs(mean(current) − mean(reference)) / std(reference), with population std (divide by n). If std is 0, shift is 0 when the means are equal, else infinite. Flag when shift > threshold. Return flagged names sorted.

  10. 10.Point-in-time feature join

    Write point_in_time_join(events, feature_rows). events are [entity, ts]; feature_rows are [entity, ts, value] in any order. For each event, return the value of the row with the same entity and the largest ts at or before the event ts. If several rows share that ts, the one later in feature_rows wins. With no such row, use None. Return the values in event order.

  11. 11.Feature freshness check

    Write stale_features(last_updated, now, max_age). last_updated maps feature name to its last update time in seconds, or None if never updated. max_age maps feature name to its allowed age in seconds and always has a "default" key used for features not listed. A feature is stale if its time is None or now − time > its max age. Return stale names sorted.

  12. 12.Training data lineage

    Write upstream_datasets(edges, model). Each edge [src, dst] means src feeds dst; the graph has no cycles but nodes may be shared. A raw dataset is a node with no incoming edges. Return the sorted list of raw datasets that feed model directly or through other nodes, each once. If model has no inputs, return [].

  13. 13.Artifact cache key

    Write cache_key(config) for a training config dict. Serialize it canonically with json.dumps(config, sort_keys=True, separators=(",", ":")), encode as UTF-8, take the hashlib.sha256 hex digest, and return its first 12 characters.

  14. 14.Nested config diff

    Write config_diff(old, new) for nested dicts. Walk both together, joining keys with ".". Where a key exists in both and both values are dicts, recurse. A key only in new is added; only in old is removed (report that key's path, not its children). Otherwise, unequal values are changed. Return {"added": [...], "removed": [...], "changed": [...]}, each list of paths sorted.

  15. 15.Retry with backoff and jitter

    Write retry_run(outcomes, base, cap, jitter). outcomes[i] is True if attempt i succeeds; at most len(outcomes) attempts are made, stopping at the first success. After the n-th failure (n from 0), if another attempt remains, wait min(cap, base × 2^n) + jitter[n] seconds. Return {"attempts": attempts made, "succeeded": bool, "waits": [waits in order]}.

  16. 16.GPU job first-fit scheduler

    Write first_fit_gpus(jobs, machines). jobs is a list of [name, gpus] handled in order; machines lists each machine's free GPU count. Put each job on the lowest-index machine that still has at least gpus free and subtract them; a job that fits nowhere is unplaced. Return {"placement": {name: machine index}, "unplaced": [names in job order]}.

  17. 17.Training queue wait times

    Write queue_waits(jobs, workers). jobs is a list of [submit_time, duration] sorted by submit_time and served first come first served. Each job starts at the later of its submit time and the earliest time any of the workers is free, then holds that worker for duration. Return each job's wait (start − submit) in input order.

  18. 18.Batch inference chunking

    Write make_batches(token_counts, max_batch, max_tokens). Go through items in order, adding each to the current batch unless that would exceed max_batch items or max_tokens total tokens; then start a new batch with it. An item over max_tokens on its own still goes in a batch, alone. Return the batches as lists of item indices; no items gives [].

  19. 19.Nearest-rank latency percentiles

    Write latency_percentiles(latencies, percentiles). Use nearest rank: sort latencies ascending; for percentile p, rank = ceiling(p × n / 100) with a minimum of 1, and the value is the rank-th smallest (1-based). Return {"p<p>": value} for each p, e.g. {"p50": 12, "p95": 40}. If latencies is empty, every value is None.

  20. 20.SLO error budget remaining

    Write budget_remaining(slo, days). days is a list of [requests, failed] per day. After each day, using totals so far: allowed = (1 − slo) × requests so far, remaining = (allowed − failed so far) / allowed. If allowed is 0, remaining is 1.0. Do not clamp: overspending gives a negative value. Return the remaining fraction after each day.

  21. 21.Alert dedupe window

    Write dedupe_alerts(alerts, window). alerts is a list of [ts, key] sorted by ts. Keep an alert if no alert with the same key has been kept yet, or ts − (that key's last kept ts) >= window. Suppressed alerts do not restart the window. Return the kept alerts in input order, as [ts, key] lists.

  22. 22.Weekly performance decay

    Write detect_decay(weekly, w, drop). weekly is a model metric per week (higher is better). baseline = mean of the first w weeks. For each week index i from 2w − 1 upward, trailing = mean of weeks i − w + 1 through i. Return the first i where baseline − trailing >= drop, or -1 if none.

  23. 23.A/B guardrail check

    Write guardrail_check(control, treatment, guardrails). Both map metric to a positive number. Each guardrail is [metric, direction, max_regression]: "up" if higher is better, "down" if lower is. Regression is (c − t) / c for up, (t − c) / c for down. It fails if the metric is missing on either side or regression > max_regression. Return {"passed": bool, "failed": [metrics in guardrail order]}.

  24. 24.Parse container resource requests

    Write total_resources(containers). Each container dict may have "cpu" and "memory" strings; a missing key counts as 0. CPU: "500m" is millicores (0.5 cores), otherwise a number of cores like "2" or "0.25". Memory: integer with optional suffix Ki, Mi, Gi (1024, 1024², 1024³), K, M, G (1000, 1000², 1000³), or none for bytes. Return {"cpu": total cores as float, "memory": total bytes as int}.

  25. 25.Autoscaling replica count

    Write desired_replicas(current, metric, target, min_r, max_r) with positive integers. If abs(metric − target) <= target / 10, keep current. Otherwise desired = ceiling(current × metric / target). In both cases clamp the result to [min_r, max_r]. Return the replica count as an int.

  26. 26.Prediction cache with TTL

    Write cache_results(requests, ttl). requests is a list of [ts, key] sorted by ts. A request is a hit if its key was stored at time s and ts − s < ttl; otherwise it is a miss and the key is stored at ts. Hits do not refresh the stored time. Return {"results": ["hit" or "miss" per request], "hit_rate": hits / requests}, with hit_rate 0.0 for no requests.

  27. 27.Pipeline step order

    Write pipeline_order(steps). steps maps each step name to the list of steps it depends on; every dependency is also a key. Return an order where each step comes after all its dependencies, choosing the alphabetically smallest ready step whenever several are ready. If the dependencies contain a cycle, return None.

  28. 28.GPU node busy time

    Write busy_seconds(jobs). Each job is [machine, start, end] with start < end, in any order. A machine is busy whenever at least one of its jobs runs; overlapping or touching intervals count once. Return {machine: total busy seconds} for every machine that appears.