Card 1 of 50
Front. Bias. What is bias, in the bias-variance sense?
These cards are original study material. They are not questions from a named employer or a live hiring loop.
All 50 cards in this deck
1.Bias
What is bias, in the bias-variance sense?
Error from a model being too simple or too constrained to capture the real pattern. A high-bias model is wrong in a consistent way, even on its own training data.
2.Variance
What is variance, in the bias-variance sense?
How much the fitted model changes when it is trained on a different sample of data. A high-variance model fits noise in its training rows.
3.Tradeoff
What is the bias-variance tradeoff?
Adding capacity usually lowers bias and raises variance, and constraining a model does the reverse. The goal is the balance with the lowest error on new data.
4.Decomposition
Expected squared error on a new point splits into which three parts?
Bias squared, variance, and irreducible noise. Noise is the part of the target that no model using these features can predict.
5.Overfitting
What is overfitting?
Fitting the training data so closely, noise included, that performance on new data gets worse. The sign is low training error with much higher validation error.
6.Underfitting
What is underfitting?
A model too simple to capture the pattern. Training and validation error are both high and close together.
7.Learning-curve
What is a learning curve?
A plot of training and validation error as the amount of training data grows. A gap that is still closing means more data would help. Curves that have met at a high error mean the model needs more capacity or better features.
8.Generalization
What is generalization?
How well a model performs on data it was not trained on, drawn from the same setting it will be used in.
9.Regularization
What is regularization?
Any technique that limits how closely a model can fit its training data, trading a little bias for lower variance. Examples are weight penalties, dropout, and early stopping.
10.L2
What does L2 regularization (ridge) do?
Adds a penalty proportional to the sum of squared weights. Weights shrink toward zero but rarely become exactly zero.
11.L1
What does L1 regularization (lasso) do?
Adds a penalty proportional to the sum of absolute weights. It drives many weights to exactly zero, so it also selects features.
12.Elastic-net
What is elastic net?
A mix of L1 and L2 penalties. It keeps some sparsity while handling groups of correlated features better than lasso alone.
13.Lambda
What does the regularization strength (lambda) control?
How much the penalty counts against training error. Larger values give simpler, flatter models; too large leads to underfitting.
14.Dropout
What is dropout?
During training, randomly setting a fraction of activations to zero on each step. It stops units from relying on specific partners. It is turned off at inference.
15.Early-stopping
What is early stopping?
Stopping training when the validation score stops improving and keeping the best checkpoint. Training length acts as a regularizer.
16.Augmentation
What is data augmentation?
Creating extra training examples by applying changes that should not alter the label, such as crops or flips of an image.
17.Train-val-test
What are the roles of the training, validation, and test sets?
Training data fits the weights. Validation data guides choices such as hyperparameters and thresholds. The test set gives one final, unbiased estimate after every choice is made.
18.Cross-validation
What is k-fold cross-validation?
Splitting data into k parts, training on k minus 1 parts and validating on the remaining one, rotating through all k. It gives an average score and its spread.
19.Stratified
What is stratified k-fold?
Cross-validation that keeps each class's share about the same in every fold. It matters when a class is rare.
20.Group-split
What is a group split?
A split that keeps all rows from the same entity, such as a user or patient, on the same side. It stops the model from scoring well by recognizing the entity.
21.Time-split
What is a time-based split?
Training on earlier data and validating on later data. It matches how the model will be used when it predicts the future.
22.Nested-cv
What is nested cross-validation?
An outer loop that estimates performance and an inner loop that tunes hyperparameters. It avoids the optimistic score you get by tuning and reporting on the same folds.
23.Leakage
What is data leakage?
Information that would not be available at prediction time getting into training or features. It inflates offline scores and fails in production.
24.Target-leakage
What is target leakage?
A feature that is caused by, or only filled in after, the outcome being predicted. An example is 'account closed date' used to predict churn.
25.Confusion-matrix
What is a confusion matrix?
A table of counts of true positives, false positives, false negatives, and true negatives at a chosen threshold.
26.Accuracy
What is accuracy, and when does it mislead?
The share of predictions that are correct. With a rare class it misleads, because always predicting the common class scores high.
27.Precision
What is precision?
True positives divided by all predicted positives. Of the cases the model flagged, how many were really positive.
28.Recall
What is recall?
True positives divided by all real positives. Of the real positive cases, how many the model caught. Also called sensitivity or true positive rate.
29.F1
What is the F1 score?
The harmonic mean of precision and recall: 2PR / (P + R). It leans toward the weaker of the two and weights them equally.
30.Fpr
What is the false positive rate?
False positives divided by all real negatives. It is the x-axis of a ROC curve.
31.Roc-auc
What does ROC-AUC measure?
The chance that a random positive gets a higher score than a random negative. It measures ranking across all thresholds; 0.5 is random and 1.0 is perfect.
32.Pr-auc
What is PR-AUC (average precision), and when is it preferred?
The area under the precision-recall curve. It is more informative than ROC-AUC when positives are rare. A random model scores about the positive rate.
33.Threshold
What does the decision threshold do?
Turns a score into a yes-or-no decision. Raising it usually raises precision and lowers recall. It should be chosen from error costs on validation data.
34.Log-loss
What is log loss (cross-entropy)?
The average of minus the log of the probability the model gave to the true label. Confident wrong predictions are punished heavily.
35.Calibration
What does it mean for a model to be calibrated?
Its scores match observed rates. Among cases scored 0.7, about 70 percent are positive.
36.Platt-isotonic
What are Platt scaling and isotonic regression?
Ways to recalibrate a model's scores after training, fit on held-out data. They change the scores without changing their order.
37.Brier
What is the Brier score?
The mean squared difference between the predicted probability and the 0 or 1 outcome. Lower is better; it rewards both calibration and ranking.
38.Macro-micro
What is the difference between macro and micro averaging?
Macro averages a metric across classes equally, so small classes count as much as large ones. Micro pools all predictions, so large classes dominate.
39.Mae
What is mean absolute error (MAE)?
The average of absolute errors. It is in the target's units and is less sensitive to outliers than RMSE. Minimizing it predicts the median.
40.Rmse
What is root mean squared error (RMSE)?
The square root of the average squared error. Large errors count more. Minimizing squared error predicts the mean.
41.R-squared
What is R²?
1 minus the model's squared error divided by the squared error of predicting the mean. 0 means no better than the mean; on new data it can be negative.
42.Mape
What is MAPE, and what is its weakness?
Mean absolute percentage error. It blows up when true values are near zero and treats over- and under-prediction unevenly.
43.Class-imbalance
What are common ways to handle class imbalance?
Use precision-recall metrics, tune the threshold, apply class weights, or resample the training data only. Keep validation at the real class rate.
44.Smote
What is SMOTE?
An oversampling method that creates synthetic minority examples between nearby real ones. Apply it only inside the training fold.
45.Scaling
Why scale features, and which models need it?
Scaling puts features on comparable ranges. Distance-based and gradient-based models, and weight penalties, need it. Tree models mostly do not.
46.One-hot
What is one-hot encoding?
Turning a categorical value into a vector with a 1 for its category and 0 elsewhere. It gets unwieldy with very many categories.
47.Target-encoding
What is target encoding?
Replacing a category with the average label for that category. It must be computed out of fold and smoothed, or it leaks the label.
48.Curse-dimensionality
What is the curse of dimensionality?
In many dimensions, data becomes sparse and distances between points become similar, so methods based on nearness work poorly.
49.Pca
What is PCA?
Principal component analysis finds perpendicular directions of greatest variance and projects data onto the top few. It ignores the label.
50.Bagging-boosting
What is the difference between bagging and boosting?
Bagging trains models independently on resampled data and averages them, which mainly cuts variance; random forests are an example. Boosting trains models in sequence, each fixing the last one's errors, which mainly cuts bias.