ML fundamentals

Question 1

A gradient-boosted model scores 0.99 accuracy on its training rows and 0.81 on a held-out week. A smaller model scores 0.84 on both. What does the gap in the first model mostly show?

Accepted Answer

High variance: it fit details of the training rows that did not carry over.

Question 2

A linear model for house prices has a training RMSE of 92,000 dollars and a validation RMSE of 94,000. The business needs under 40,000. Where is the problem?

Accepted Answer

Bias: the model cannot fit even the rows it was trained on.

Question 3

Which change usually lowers variance and raises bias?

Accepted Answer

Raising the L2 penalty.

Question 4

As you add training rows, training error rises slowly and validation error falls. The gap between them keeps shrinking but has not closed. What is the next step most likely to help?

Accepted Answer

Collect more rows, since variance is still what limits the score.

Question 5

Training and validation curves both flattened at the same high error after 20,000 rows. Will going to 200,000 rows help much?

Accepted Answer

Probably not. The curves have met, so the model needs more capacity or better features.

Question 6

For squared error, the expected error on a new point splits into which three parts?

Accepted Answer

Bias squared, variance, and irreducible noise.

Question 7

Where does the irreducible error in that split come from?

Accepted Answer

Variation in the target that no function of the given features can predict.

Question 8

In k-nearest neighbors, what happens when you move k from 1 to 50?

Accepted Answer

Bias rises and variance falls, because each prediction averages more neighbors.

Question 9

A decision tree grown until every leaf is pure tends toward which failure?

Accepted Answer

High variance: a small change in the rows produces a very different tree.

Question 10

Why does a random forest usually generalize better than one deep tree?

Accepted Answer

Averaging many decorrelated trees cuts variance while keeping each tree's low bias.

Question 11

Boosting fits trees one after another, and each tree fits the errors the earlier ones left. Which error does that mainly attack?

Accepted Answer

Bias, by adding capacity a little at a time.

Question 12

A teammate says the model is overfit because validation accuracy is 3 points below training accuracy. What would you want to see before agreeing?

Accepted Answer

Whether a smaller or more regularized model actually scores higher on validation.

Question 13

A very large network keeps improving on validation after it reaches zero training error. Does that break the bias and variance picture?

Accepted Answer

Not exactly. In heavily overparameterized models, test error can fall again past the point where training error hits zero.

Question 14

Which symptom points to underfitting?

Accepted Answer

Training loss stays high and close to validation loss.

Question 15

Validation loss is lowest at epoch 14 and climbs after that while training loss keeps falling. What does early stopping do here?

Accepted Answer

Keeps the weights from about epoch 14, using training length as a regularizer.

Question 16

Why must the early-stopping epoch be chosen on validation data and not on the test set?

Accepted Answer

Choosing on the test set makes it part of training decisions, so its score is no longer an unbiased estimate.

Question 17

What does L2 regularization add to the loss?

Accepted Answer

A penalty proportional to the sum of squared weights.

Question 18

L1 regularization tends to produce something that L2 does not. What?

Accepted Answer

Weights that are exactly zero, so it also selects features.

Question 19

Why does L1 push small weights to exactly zero while L2 only shrinks them?

Accepted Answer

The L1 penalty has a constant slope near zero, while the L2 slope shrinks to nothing as a weight approaches zero.

Question 20

Two features are near copies of each other. How do lasso and ridge tend to treat them?

Accepted Answer

Lasso often keeps one and zeroes the other; ridge spreads weight across both.

Question 21

Elastic net mixes the L1 and L2 penalties. When is that worth it over plain lasso?

Accepted Answer

When groups of correlated features matter and you still want some sparsity.

Question 22

What happens to the coefficients when ridge regression's lambda goes from 0.01 to 100?

Accepted Answer

They shrink toward zero, trading some bias for lower variance.

Question 23

Why should features be standardized before an L2 penalty is applied?

Accepted Answer

The penalty is on raw weight size, so a feature in large units needs a small weight and gets penalized less than one in small units.

Question 24

Is the intercept term usually included in the L2 penalty?

Accepted Answer

No. Shrinking it would pull predictions toward zero rather than toward the average.

Question 25

What does dropout do during training?

Accepted Answer

Zeroes a random subset of activations on each step, so units cannot depend on specific partners.

Question 26

With the common inverted-dropout implementation, what happens to dropout at inference?

Accepted Answer

It is turned off. Activations were already scaled up during training, so nothing is rescaled at inference.

Question 27

A model's validation score changes every time you run evaluation on the same rows. You find dropout was still on. What went wrong?

Accepted Answer

The model was left in training mode, so each pass used a different random subnetwork.

Question 28

With the Adam optimizer, why do people prefer decoupled weight decay (AdamW) over adding an L2 term to the loss?

Accepted Answer

Adam rescales each gradient, so the L2 part gets rescaled too and no longer shrinks weights evenly. AdamW applies the decay directly.

Question 29

Why is flipping and cropping training images a form of regularization?

Accepted Answer

It teaches the model that those changes should not change the label, which narrows the functions that fit.

Question 30

For a model that reads handwritten digits, why is a horizontal flip a poor augmentation?

Accepted Answer

A flipped digit can look like a different symbol or none at all, so the label is no longer guaranteed.

Question 31

Label smoothing replaces a one-hot target of 1.0 with something like 0.9. What does it discourage?

Accepted Answer

Logits that keep growing to push training loss toward zero, which makes the model overconfident.

Question 32

Which of these is not a regularizer?

Accepted Answer

Adding more hidden layers.

Question 33

A model with 2 million parameters is trained from scratch on 5,000 labeled rows. Which plan is most reasonable first?

Accepted Answer

Start from a pretrained model or a much smaller one, and add regularization.

Question 34

Dropout, a strong L2 penalty, and a max-norm limit were added together. Validation loss is now worse than with none of them. What is the likely reading?

Accepted Answer

The model now underfits. Check training loss and ease off the strength.

Question 35

What does 5-fold cross-validation give you that one 80/20 split does not?

Accepted Answer

Five estimates from different held-out slices, so you see the spread as well as the average.

Question 36

When is k-fold cross-validation with random folds the wrong tool?

Accepted Answer

When rows are ordered in time and the model will predict the future.

Question 37

For time-ordered data, which validation scheme fits?

Accepted Answer

Rolling or expanding windows where each validation block comes after its training block.

Question 38

A dataset has 40 photos per patient across 300 patients. How should folds be made?

Accepted Answer

Group by patient, so no patient appears on both sides.

Question 39

You tune hyperparameters with 5-fold cross-validation and report the best fold-averaged score as the expected production score. What is wrong with that?

Accepted Answer

Picking the best of many settings on the same folds makes that score optimistic. Nested cross-validation or a separate test set fixes it.

Question 40

What does stratified k-fold keep about the same in every fold?

Accepted Answer

The share of each class.

Question 41

The positive class is 2 percent of 3,000 rows. Why stratify the folds?

Accepted Answer

With only about 60 positives, random folds can get very few of them, and fold scores swing.

Question 42

What is a drawback of leave-one-out cross-validation?

Accepted Answer

It needs n separate fits, and because the training sets are nearly identical, the averaged estimate can still have high variance.

Question 43

A scaler is fit on the full dataset before the cross-validation folds are made. What does that do?

Accepted Answer

Leaks validation statistics into training, so scores come out somewhat optimistic.

Question 44

What is the clean way to avoid that leak?

Accepted Answer

Put the scaler inside a pipeline, so each fold fits it on that fold's training rows only.

Question 45

Feature selection keeps the 50 features most correlated with the label, using all 1,000 rows. Cross-validation then runs on those 50. Why is the score inflated?

Accepted Answer

The selection already looked at the validation rows' labels.

Question 46

What is the test set for?

Accepted Answer

One final estimate after every choice is made, looked at as rarely as possible.

Question 47

A fraud model on data with 0.5 percent fraud reports 99.5 percent accuracy. What trivial model matches it?

Accepted Answer

One that predicts 'not fraud' for every transaction.

Question 48

On 1,000 cases, a model gets 80 true positives, 20 false positives, 40 false negatives, and 860 true negatives. What is precision?

Accepted Answer

0.80

Question 49

With 80 true positives, 20 false positives, 40 false negatives, and 860 true negatives, what is recall?

Accepted Answer

About 0.67

Question 50

With precision 0.80 and recall about 0.67, what is F1?

Accepted Answer

About 0.73

Question 51

With 20 false positives and 860 true negatives, what is the false positive rate?

Accepted Answer

About 0.023

Question 52

When does recall matter more than precision?

Accepted Answer

When a miss costs far more than a false alarm, as in screening for a serious disease.

Question 53

A spam filter that moves real mail into spam annoys users far more than missed spam does. Which metric should the team watch most closely at the chosen threshold?

Accepted Answer

Precision on spam.

Question 54

A false positive costs 1 dollar and a false negative costs 500 dollars. Why is F1 a poor single metric here?

Accepted Answer

F1 weights precision and recall equally, so it ignores the real cost ratio.

Question 55

What does ROC-AUC measure?

Accepted Answer

The chance that a random positive gets a higher score than a random negative.

Question 56

A model's ROC-AUC is 0.5. What does that mean?

Accepted Answer

Its ranking is no better than random.

Question 57

With 1 positive per 1,000 rows, why can ROC-AUC look excellent while the model is useless at the top of its list?

Accepted Answer

The false positive rate divides by a huge number of negatives, so many false alarms barely move it. Precision shows them.

Question 58

What is the baseline average precision (PR-AUC) for a model that scores at random?

Accepted Answer

The share of positives in the data.

Question 59

ROC-AUC stays the same when you raise the decision threshold from 0.5 to 0.8. Why?

Accepted Answer

AUC summarizes ranking across all thresholds, so no single threshold changes it.

Question 60

Who should choose the operating threshold, and with what?

Accepted Answer

The team, using validation data and either the cost of each error or a required precision or recall.

Question 61

Model A says 0.99 for a case that turns out negative. Model B says 0.6 for it. How does log loss treat them?

Accepted Answer

A pays far more: about -ln(0.01) ≈ 4.6 versus -ln(0.4) ≈ 0.92.

Question 62

What does it mean for a classifier to be well calibrated?

Accepted Answer

Among the cases it scores 0.7, about 70 percent are positive.

Question 63

A model ranks well, with an AUC of 0.92, but only 60 percent of the cases it scores 0.9 are positive. What can fix that without retraining?

Accepted Answer

Fit a calibration map, such as Platt scaling or isotonic regression, on held-out data.

Question 64

A model's score feeds an expected-cost calculation. Why does calibration matter here?

Accepted Answer

The cost math multiplies by the probability, so a correct ranking with wrong probabilities still gives wrong decisions.