Applied AI Scientist

1 topic · 28 questions

Write the evaluation and data code around language models, from scoring answers to parsing model output.

Mid level

LLM evaluation code
Write the code around language-model evals in plain Python: answer normalization, token F1, ROUGE-L, n-gram precision, retrieval metrics, pass@k, calibration, win rates and Elo, prompt templates, JSON and tool-call parsing, dedup, contamination checks, and context truncation.
28 questions · about 14 minutes