Purpose

Public preview: This page shows the analysis output, figures, diagnostics, and reporting guidance without exposing the R code. The paid version includes the complete editable R Markdown source, reusable functions, all analysis code, synthetic data, documentation, and exported results.

This workflow teaches design-led evaluation when educational programs were not randomly assigned. Four separate synthetic examples cover propensity-score adjustment, difference-in-differences, regression discontinuity, and interrupted time series. Each module defines its own estimand and assumptions; no adjustment is described as recreating a randomized experiment.

User settings

Generate the examples

Design before analysis

Before modeling, state the intervention, assignment process, target population, potential outcomes, estimand, comparison condition, eligibility date, follow-up window, interference risks, attrition, and clustering. A plausible design requires knowledge that cannot be recovered from the dataset alone.

Design logic and identifying assumptions
design assignment estimand central_assumption
Propensity weighting/matching Observed covariates ATE or ATT No important unmeasured confounding plus overlap
Difference-in-differences Group and time Change attributable to adoption Parallel untreated trends
Regression discontinuity Threshold rule Local effect at cutoff Continuity and no precise manipulation
Interrupted time series Known intervention time Level/slope change No concurrent cause with same timing

Propensity-score adjustment

Overlap and balance

Standardized mean differences before and after weighting
variable unweighted weighted
prior 0.669 0.058
ses -0.425 -0.037
attendance 0.072 0.008

## Weighted and matched estimates

Adjusted program-effect estimates
method estimate std_error
ATE weighting 2.919 0.507
Nearest-neighbor ATT matching 3.168 0.519

Overlap, balance, weight concentration, and sensitivity to specifications matter more than propensity-model classification accuracy. The estimate remains conditional on no consequential unmeasured confounding.

Difference-in-differences

Regression discontinuity

Local linear estimate

Local linear discontinuity estimate
bandwidth estimate std_error p_value
8 4.9 1.288 0

Bandwidth sensitivity
bandwidth estimate std_error
5 4.997 1.731
8 4.900 1.288
12 4.681 1.007

The effect is local to the cutoff. Investigate sorting, other rules at the same threshold, functional form, bandwidth, and covariate continuity before making a causal claim.

Interrupted time series

Interrupted time-series model with Newey-West uncertainty
term Estimate Std. Error t value Pr(>|t|)
(Intercept) 70.741 0.423 167.102 0.000
month 0.133 0.027 4.872 0.000
intervention 1.753 0.855 2.051 0.045
time_after 0.044 0.037 1.185 0.241

Check seasonality, autocorrelation, changes in measurement, concurrent events, anticipation, and sufficient pre/post observations. A single series offers weak protection against coincident shocks; add a comparison series when possible.

Dynamic reporting example

The weighted synthetic comparison estimated an average program effect of 2.92 points. The difference-in-differences estimate was 2.45 points (p < .001), and the local regression-discontinuity estimate at the eligibility threshold was 4.90 points (p < .001). These estimates answer different causal questions and should not be treated as interchangeable replications.

Reporting checklist

  • Name the estimand, assignment process, comparison condition, timing, and analytic population.
  • Show overlap and balance, pretrends, cutoff diagnostics, or time-series diagnostics before the focal estimate.
  • Report uncertainty that respects clustering, serial dependence, and repeated observations.
  • Present specification and bandwidth sensitivity without selecting only favorable results.
  • Address attrition, missingness, spillovers, anticipation, treatment variation, and concurrent policies.
  • Separate design assumptions from tests and avoid claiming that assumptions were proven.

When these designs are not enough

Use randomized assignment when feasible. Instrumental variables need a defensible instrument and a local estimand. Synthetic controls require credible donor units and longer panels. Staggered adoption, fuzzy thresholds, multiple cutoffs, few clusters, and latent outcomes need specialized extensions. Statistical adjustment cannot repair an unknown assignment process or a comparison group with no credible counterfactual role.

Export results