Causal inference

Why is an individual causal effect usually unobservable?

It compares the same unit's outcome under two alternative interventions at the same relevant time. Only one is observed, so identifying average effects requires a design and assumptions connecting different units or periods.

Why is 'the effect of engagement' an underspecified causal question?

Engagement can be changed by many interventions with different consequences. Specify the action, comparator, eligible users, timing, and outcome horizon before choosing a dataset or estimator.

What population does the average treatment effect describe?

The expected difference between treated and untreated potential outcomes in a stated target population. Changing eligibility or trimming poor-overlap units changes that population and potentially the estimand.

How can the ATT differ from the ATE?

ATT averages effects among those actually treated; ATE averages over the target population. If treatment effects vary and treated users differ from others, the two averages need not agree.

What does consistency require in a potential-outcomes analysis?

A unit receiving the specified treatment has the corresponding potential outcome. Hidden treatment versions, different doses, or poorly defined timing can make a single treatment label inadequate.

What is conditional exchangeability?

After conditioning on the chosen covariates, treatment assignment is independent of the potential outcomes. It is often called no unmeasured confounding; fitting a flexible model does not establish it.

Why is overlap necessary for covariate-adjusted causal comparisons?

Each treatment option must be possible for the covariate profiles in the target population. If some users always receive treatment, their untreated outcome requires unsupported extrapolation unless the target or design changes.

What breaks when one user's treatment changes another user's outcome?

The outcome may depend on others' assignments as well as the user's own treatment. Define exposure mappings or cluster/network interventions; an ordinary individual-treatment estimand may no longer capture the policy effect.

What does an arrow in a causal DAG represent?

An assumed direct causal relationship relative to the included variables. The graph encodes substantive assumptions, not merely correlations, and cannot generally be recovered uniquely from observational associations alone.

A loyalty score affects both discount targeting and later spending. Why adjust for it?

It can create a backdoor association between receiving a discount and spending. Adjusting for a suitable pretreatment score can block that path, provided it is measured well and the full adjustment set is sufficient.

Why can adjusting for post-discount purchases distort the total effect on annual revenue?

Those purchases may be part of the pathway through which the discount changes revenue. Conditioning on a mediator removes part of the total effect and can introduce additional bias; direct effects require their own identification assumptions.

Hiring depends on both skill and a referral. Why can studying only hired people create a misleading association?

Hiring is a common effect, or collider. Conditioning on being hired can associate skill with referral status even if they were unrelated in the applicant pool; selection can open a noncausal path.

What makes an adjustment set suitable for identifying a total effect by the backdoor criterion?

It blocks all backdoor paths from treatment to outcome without including descendants of treatment. Use the causal graph to justify the set rather than selecting variables solely by predictive importance.

Why is controlling for every available variable not automatically safer?

The set may include mediators, colliders, post-treatment measurements, or noisy proxies. More controls can increase bias or instability; their causal roles matter more than their count.

Does controlling for a noisy proxy fully remove confounding?

Not generally. Residual variation in the true confounder can still affect assignment and outcome. Assess measurement quality and sensitivity rather than assuming the proxy's inclusion resolves the problem.

What does a propensity score estimate?

The probability of treatment conditional on pretreatment covariates. It models assignment, not the outcome or the probability treatment helps; its value is in supporting balance and overlap under the identification assumptions.

What should be checked after propensity-score matching?

Covariate balance, overlap, discarded units, matching quality, and uncertainty that reflects the procedure. A similar propensity-score distribution alone does not guarantee balance on every important covariate.

Why are standardized differences useful for balance diagnostics?

They measure covariate differences on a scale less driven by sample size than significance tests. Inspect distributions and important interactions too; a conventional cutoff is a diagnostic convention, not proof of no confounding.

How do basic inverse-probability weights estimate an ATE?

Treated units receive weight 1/e(X), controls 1/(1-e(X)), reweighting observed groups toward a common covariate population. Identification still requires exchangeability, positivity, consistency, and an appropriate assignment model.

What does a few enormous inverse-probability weights warn about?

Near-zero probability of the observed treatment can make the estimate dominated by a few units. Inspect overlap and model fit; trimming or clipping reduces instability but may change the target or introduce bias.

How does outcome-model standardization estimate a treatment effect?

Fit an outcome model conditional on treatment and confounders, predict each target unit under both treatments, and average the prediction differences. This requires identification assumptions and a sufficiently correct outcome model.

What does 'doubly robust' mean for a suitable AIPW estimator?

Under the causal assumptions and regularity conditions, consistency can hold if either the propensity model or the outcome model is correctly specified. It does not protect against unmeasured confounding, poor overlap, or both models being wrong.

Why use cross-fitting with flexible nuisance models?

Estimate nuisance functions on other folds and evaluate their contributions on held-out observations to reduce overfitting bias. It supports certain orthogonal estimators under rate and regularity conditions, but does not create causal identification.

Discount users spend twice as much as nonusers. What prevents a causal conclusion?

High-intent or high-value customers may be more likely to receive or redeem discounts. Define assignment versus redemption, identify confounders, and seek randomized or otherwise credible variation rather than interpreting the raw gap as lift.

Treated revenue rises from 40 to 55; control rises from 30 to 36. What is the two-period DiD estimate?

The treated change is 15 and the control change is 6, so the estimate is 9 revenue units. Its causal interpretation depends on untreated parallel trends and the other design assumptions.

Does parallel trends require equal outcome levels before treatment?

No. It requires the treated group's counterfactual untreated change to follow the control group's change, possibly conditional on covariates. Baseline levels can differ substantially.

Why does a nonsignificant pretrend test not prove a DiD design valid?

The test may have little power, and post-treatment counterfactual trends remain unobserved. Inspect plausible differential shocks, anticipation, composition changes, and sensitivity to violations rather than treating p > 0.05 as approval.

How can an announced policy contaminate the pre-treatment period?

People may alter behavior before the formal start date. Then the nominal baseline already contains treatment effects; redefine exposure timing or exclude the anticipation window with a justified design.

Why can a two-way fixed-effects regression mislead with staggered adoption?

When effects vary by cohort or time since adoption, already-treated groups can become unsuitable controls and the coefficient can mix effects with problematic weights. Use an estimator aligned with explicit cohort-time comparisons and assumptions.

What should an event-study plot make explicit?

The reference period, treatment cohorts, comparison groups, support at each relative time, uncertainty, and estimator. Long-run points may represent only early adopters, so apparent dynamics can reflect changing composition.

A new store draws customers away from control stores. What happens to a store-level DiD?

The controls are affected by the intervention, so their outcomes no longer represent untreated trends. The estimated contrast can combine gains and displacement; choose a larger market unit or model spillovers explicitly.

What is the role of donor weights in synthetic control?

They combine untreated units to approximate the treated unit's pre-intervention outcomes and predictors. Credibility depends on pre-fit, donor relevance, absence of contamination, and a plausible continuation of the untreated relationship.

What can placebo interventions reveal in synthetic-control analysis?

Applying the procedure to untreated units or earlier times shows whether similarly large gaps commonly arise without the intervention. Interpret gaps relative to pre-fit and comparability; placebo ranks are not automatically exact randomization p-values.

What identifies an effect in a regression discontinuity design?

A treatment change at a known assignment threshold, combined with smooth potential-outcome behavior around that threshold. The comparison is local; units just above and below must not differ discontinuously for other causal reasons.

Why investigate sorting near an eligibility cutoff?

Precise manipulation can make units on either side systematically different, undermining local comparability. Examine assignment rules, density patterns, and baseline covariates; a clean density test alone does not prove validity.

What is the bandwidth tradeoff in regression discontinuity?

A narrow window improves local comparability but reduces precision; a wider window adds observations and stronger functional-form assumptions. Report justified selection and sensitivity rather than choosing the window with the preferred result.

What changes when crossing a cutoff raises treatment probability but does not determine treatment?

A fuzzy RDD uses the discontinuity in assignment as an instrument. Under additional assumptions, the outcome jump divided by the treatment-probability jump identifies a local effect for threshold-induced compliers.

What must an instrumental variable change?

It must meaningfully affect treatment assignment or uptake. A weak first stage can produce unstable estimates and misleading conventional inference; relevance alone does not establish instrument validity.

What does the exclusion restriction rule out?

The instrument must affect the outcome only through the specified treatment, not through another pathway. It is a substantive assumption that cannot generally be proved by showing a strong first stage.

Why might distance to a service be a problematic instrument for usage?

Distance may also reflect neighborhood, income, access, or other determinants of the outcome. The instrument must be independent of relevant potential outcomes under the design, not merely correlated with treatment.

Which effect does a binary instrument identify under standard LATE assumptions?

The average treatment effect among compliers whose treatment changes with the instrument, assuming relevance, independence, exclusion, and monotonicity. It need not equal the effect for all users or always-takers.

What does monotonicity mean in the usual binary-instrument setup?

The instrument does not cause some units to take treatment while causing others to stop, holding potential treatment responses fixed. It excludes defiers relative to the instrument direction; it does not require everyone to respond.

What is a negative-control outcome used for?

An outcome the intervention should not causally affect can reveal confounding or measurement artifacts if an association appears. A null result does not prove the main analysis unconfounded, because bias pathways may differ.

How can sensitivity analysis address unmeasured confounding?

Quantify how strong an omitted relationship or assumption violation would need to be to alter the conclusion. Compare that strength with domain knowledge or observed covariates, while avoiding claims that sensitivity analysis removes the confounder.

What is wrong with classifying subscribers as treated from signup when subscription happens months later?

To enter the treated group, users had to remain observable until subscription, creating a guaranteed pre-treatment period. Align eligibility, treatment assignment, and follow-up start, or use a suitable time-varying treatment design.

What should a target-trial specification define before observational analysis?

Eligibility, treatment strategies, assignment mechanism, time zero, follow-up, outcome, causal contrast, and analysis plan. Mapping data to this protocol exposes timing and selection problems before modeling.

Why can ordinary adjustment fail for a confounder affected by earlier treatment?

The variable can both mediate earlier effects and confound later treatment. Conditioning on it conventionally can block part of the effect or induce bias; longitudinal g-methods may be appropriate under their assumptions.

Why might an internally valid experiment not generalize to all customers?

Trial participants, context, treatment implementation, or effect modifiers may differ from the deployment population. Define the target and assess support before reweighting or extrapolating results.

What should a causal result report beyond a point estimate?

The intervention and target population, identification strategy, assumptions, uncertainty, diagnostics, and sensitivity results. Separate observed associations from identified effects and explain which conclusions remain unsupported.