Statistical inference

How does a population parameter differ from a statistic?

A parameter describes the target population, such as its mean. A statistic is computed from the observed sample and varies across samples; inference connects the statistic to the parameter under explicit sampling assumptions.

Why can a million voluntary survey responses still give a biased estimate?

People who respond may differ systematically from the target population. More observations reduce random error around the selected population's result, not the selection bias; assess coverage and response mechanisms.

What does a probability sample contribute to inference?

Known, nonzero selection probabilities support design-based estimation and uncertainty for the sampled population. Nonresponse and incomplete sampling frames can still invalidate representativeness unless addressed.

A group is sampled at twice another group's rate. How can a population mean be estimated?

Use weights proportional to inverse inclusion probabilities and normalize the weighted sum. Verify the sampling design and account for weights in uncertainty; treating weighted rows as independent replicas understates variance.

Why can a weighted sample of 10,000 have much less information than 10,000 equal-weight observations?

A few large weights can dominate the estimate. The diagnostic (sum w)^2 / sum(w^2) measures weight concentration, but clustering and the actual design also affect effective information.

What does mean squared error combine?

For an estimator, MSE equals variance plus squared bias relative to the target parameter. A slightly biased shrinkage estimator can have lower MSE than an unbiased but unstable one.

Does an unbiased estimator have to be consistent?

No. Unbiasedness concerns its expectation at each sample size; consistency concerns convergence to the target as sample size grows. A statistic that always uses only the first observation can be unbiased without becoming more precise.

How does standard error differ from standard deviation?

Standard deviation describes variation among observations. Standard error describes sampling variation of an estimator; for an IID mean with finite variance it is estimated by s / sqrt(n).

How much more IID data is needed to halve a mean's standard error?

About four times as much, because standard error scales as 1 / sqrt(n) when variability and the sampling design remain comparable. Doubling the sample reduces it only by a factor of sqrt(2).

What does the central limit theorem approximate?

Under suitable conditions, the standardized sample mean approaches a normal distribution as sample size grows. It does not make the raw observations normal, and dependence or very heavy tails can make the usual approximation inappropriate.

Why can mean revenue stabilize slowly even with many users?

A small number of very large purchases can dominate variance and convergence. Inspect the tail and uncertainty; trimming or winsorizing may improve stability but changes the quantity being estimated unless justified by the estimand.

When does a median answer a different business question from a mean?

The median describes the middle observation, while the mean determines total divided by count. Median order value can stay flat while total revenue rises through rare large orders; report the statistic relevant to the decision.

Why use the median absolute deviation as a scale diagnostic?

It is less sensitive to extreme observations than standard deviation. Its normal-consistency scaling is an assumption-dependent conversion; it does not by itself describe tail risk or make outliers invalid.

What does 95% confidence refer to in a frequentist interval?

The long-run coverage of the interval-building procedure under its assumptions. It is not a 95% probability assigned to the fixed parameter after this particular interval has been observed.

How does a prediction interval differ from a confidence interval for a mean response?

A prediction interval includes variation of a new outcome as well as uncertainty in the estimated mean. It is generally wider; a precise average forecast does not imply precise predictions for individual cases.

Why does a small-sample mean interval use a t critical value when variance is estimated?

Estimating variance adds uncertainty. The classical exact t result assumes IID normal observations; approximate use for non-normal samples needs adequate behavior of the mean, especially with skew and outliers.

Why might Welch's t-test be preferred to the pooled two-sample t-test?

It allows unequal population variances and uses adjusted degrees of freedom. Both require an appropriate independent-sample design; Welch's adjustment does not solve clustering, selection bias, or severe outlier problems.

Before-and-after measurements come from the same people. What changes in the analysis?

Analyze within-person differences or an equivalent paired model rather than treating the samples as independent. Pairing can remove stable between-person variation, but a before/after difference is not automatically a causal treatment effect.

Why are ordinary row-level standard errors risky for repeated measurements per user?

Observations within a user may be correlated, so row count exaggerates independent information. Use an appropriate clustered, hierarchical, or repeated-measures method and check that the number of independent clusters is adequate.

What should be resampled in a bootstrap of customer-level spending?

Resample independent customers with all their relevant records, not individual transactions, when customers are the sampling units. Preserve pairing, clustering, or stratification required by the design.

Why use a block bootstrap for a time series?

Resampling contiguous blocks can retain short-range dependence that IID resampling destroys. Block length affects bias and variance, and strong nonstationarity may require a different model or resampling design.

Why does bootstrapping not fix a biased or incomplete dataset?

It resamples the observed empirical distribution. Missing populations, systematic measurement error, and unobserved extreme events are not created by drawing the same records repeatedly.

How is a simple percentile bootstrap interval constructed?

Recompute the statistic on many appropriate resamples and take empirical quantiles, such as 2.5% and 97.5%. Coverage can be poor for biased, skewed, boundary, or nonsmooth estimators; inspect whether a different interval method is needed.

What makes a permutation test valid?

The chosen rearrangements must be justified under the null, such as by randomized assignment or exchangeability. Shuffling arbitrary time-series rows or breaking matched pairs can produce an invalid reference distribution.

Why must the null hypothesis be written precisely?

A test of zero mean difference, identical distributions, and no individual treatment effect are different hypotheses. The statistic, reference distribution, and interpretation must match the claim being tested.

What does a tiny p-value fail to establish?

It does not establish a large effect, causal identification, valid measurement, or a low probability that the null is true. It measures incompatibility with the specified null model, including its assumptions.

When is a one-sided test appropriate?

When a directional hypothesis and decision rule are specified before seeing results and effects in the other direction do not count as evidence for that claim. Choosing the direction after inspecting the data invalidates the nominal error rate.

When does excluding zero from a 95% confidence interval correspond to p < 0.05?

When the interval and two-sided test are compatible inversions of the same method and assumptions. Mixing bootstrap intervals, different standard errors, or adjusted and unadjusted procedures can break that equivalence.

What does family-wise error control protect?

It limits the probability of at least one false rejection in a defined family of hypotheses. Define the family before interpretation; splitting a search into many convenient families can defeat the intended protection.

How does false discovery rate differ from the probability a specific finding is false?

FDR controls an expected proportion of false rejections among discoveries under the procedure's assumptions. It does not assign each reported finding that same posterior probability of being false.

How does Bonferroni control error across m tests?

Test each hypothesis at alpha / m to bound family-wise error by alpha using the union bound. It does not require independent tests, but can be conservative when many tests are correlated.

What does a coefficient in a multiple linear regression describe?

The fitted change in conditional mean outcome for a one-unit predictor change holding the included predictors fixed. Its meaning depends on transformations and interactions, and it is not causal without identification assumptions.

In y = b0 + b1 x + b2 z + b3 xz, what is the effect of x on the fitted mean?

The slope is b1 + b3 z. Reporting b1 alone describes the slope at z = 0; centering z can make that reference value more meaningful without removing the interaction.

What does a widening residual funnel suggest?

Error variance may change with fitted values or predictors. Heteroskedasticity-robust standard errors can improve inference under suitable conditions, but do not repair a misspecified mean function or create constant prediction uncertainty.

A residual plot shows a curved pattern. What should be investigated?

Missing nonlinear structure, interactions, or an unsuitable outcome scale. Increasing sample size will not repair a systematically wrong conditional mean; revise and validate the model rather than only adjusting standard errors.

Why can coefficients swing while predictions remain stable?

Highly correlated predictors allow several coefficient combinations to fit similarly. Inspect uncertainty and conditioning, consider regularization or meaningful reparameterization, and avoid treating unstable individual coefficients as reliable explanations.

How is an influential observation different from an outlier?

An outlier has an unusual outcome relative to the model; an influential point materially changes the fit when removed. High-leverage predictor values can be influential even with modest residuals. Investigate data quality and sensitivity before excluding it.

What can Pearson correlation miss?

Nonlinear dependence, subgroup structure, outliers, and causal direction. A near-zero correlation does not imply independence except under special distributional assumptions; inspect plots and domain mechanisms.

When is Spearman correlation useful, and what does it not solve?

It measures monotonic association using ranks and is less tied to linear scaling. It still does not establish causality or detect every nonmonotonic relationship, and ties affect the calculation and inference.

What does missing completely at random mean?

Missingness is independent of both observed and unobserved values in the relevant data model. Complete-case analysis can then avoid selection bias for many estimands, but loses precision; the assumption is stronger than a nonsignificant missingness check.

What does missing at random allow?

Missingness may depend on observed variables, but not on the missing value after conditioning on the observed information. Appropriate imputation or weighting can help under this assumption; it cannot generally be verified from observed data alone.

Why is missing-not-at-random data difficult to identify from the observed sample?

Missingness still depends on unobserved values after conditioning on observed data. Several unseen-data mechanisms can fit the same observations, so use substantive assumptions, external information, and sensitivity analysis.

Why should imputed datasets not be treated as extra independent observations?

They represent uncertainty about the same missing values. Analyze each completed dataset and combine estimates with within- and between-imputation uncertainty using a suitable pooling method, rather than concatenating them to inflate sample size.

What are the roles of prior, likelihood, and posterior?

The prior expresses parameter assumptions before the current evidence; the likelihood describes data conditional on parameters; their product, normalized, yields the posterior. Conclusions remain conditional on the model and prior choices.

What does a 95% Bayesian credible interval state?

Under the specified prior and likelihood, the interval contains 95% posterior probability for the parameter. It need not have 95% frequentist coverage, especially under model misspecification.

What does a posterior predictive check compare?

Observed data features with replicated data generated from the fitted model's posterior predictive distribution. Mismatched tails, zero counts, or subgroup patterns reveal model inadequacy; passing a few checks does not prove the model correct.

Why partially pool estimates across small groups?

A hierarchical model can pull noisy group estimates toward a shared distribution, reducing extreme estimates driven by small samples. The amount of pooling should reflect uncertainty and genuine between-group variation, not force all groups to be equal.

No failures occur in 100 independent trials. Does that establish a zero failure rate?

No. Under a binomial model, the one-sided 95% upper bound is about 1 - 0.05^(1/100), roughly 3%. The rule of three approximates it as 3/n; dependence or selective sampling changes the interpretation.

What should accompany an inference that depends on a debatable modeling choice?

Show how conclusions change under plausible alternatives for exclusions, missingness, dependence, transformations, and prior choices. Distinguish a stable decision from a result that flips under small reasonable changes.