Missing data affect more than sample size. They can change which participants contribute to an analysis, weaken precision, and bias estimates when the analysis and missingness assumptions do not fit the study. This workflow demonstrates a transparent multiple-imputation process for a longitudinal psychology study containing continuous, binary, and ordinal variables.
The example follows five connected decisions:
The report is a standalone worked example. The sourced file
R/missing_data_module.R is also designed for reuse in
Regression Outcomes, Growth Modeling, Mediation and Conditional Process,
Psychometrics, and other Dissertation Stats Helper products.
| Component | Standalone use | Reusable use |
|---|---|---|
| Variable-role settings | Documents this study | Replaced by product-specific roles |
| Method assignment | Chooses PMM, logistic, ordinal logistic, and passive methods | Called before another product’s focal model |
| Predictor matrix | Includes outcomes, predictors, auxiliaries, and all waves | Extended with the focal product’s interactions or nonlinear terms |
| Diagnostics | Reviews chains, distributions, and implausible values | Retained as a required pre-analysis checkpoint |
| Pooling | Demonstrates mixed and logistic models | Replaced by the other product’s substantive model |
| Sensitivity analysis | Applies a delta adjustment to a later outcome | Adapted to the outcome and plausible departure in that study |
Increase the number of imputations when the fraction of missing information is substantial or when stable tail-area estimates are important. Twenty imputations are adequate for this teaching example; real studies may require more.
The complete synthetic reference is used only to confirm that the
simulation behaved as intended. It is never supplied to
mice or any fitted model.
| variable | missing_n | missing_percent |
|---|---|---|
| outcome_w4 | 147 | 28.3 |
| engagement_w4 | 144 | 27.7 |
| employed_w4 | 137 | 26.3 |
| sleep_w4 | 133 | 25.6 |
| outcome_w3 | 105 | 20.2 |
| engagement_w3 | 103 | 19.8 |
| sleep_w3 | 98 | 18.8 |
| employed_w3 | 97 | 18.7 |
| outcome_w2 | 61 | 11.7 |
| sleep_w2 | 55 | 10.6 |
| engagement_w2 | 55 | 10.6 |
| employed_w2 | 52 | 10.0 |
| baseline_anxiety | 36 | 6.9 |
| motivation | 36 | 6.9 |
| outcome_w1 | 29 | 5.6 |
| engagement_w1 | 26 | 5.0 |
| employed_w1 | 15 | 2.9 |
| sleep_w1 | 11 | 2.1 |
| participant_id | 0 | 0.0 |
| treatment_num | 0 | 0.0 |
| program_group | 0 | 0.0 |
| age | 0 | 0.0 |
| first_generation | 0 | 0.0 |
| dropout_wave | 0 | 0.0 |
| missing_n | participants |
|---|---|
| 0 | 191 |
| 1 | 143 |
| 2 | 47 |
| 3 | 10 |
| 4 | 23 |
| 5 | 16 |
| 6 | 6 |
| 8 | 25 |
| 9 | 17 |
| 10 | 2 |
| 12 | 30 |
| 13 | 8 |
| 14 | 2 |
| program_group | missing_percent | participants |
|---|---|---|
| Practice-focused | 16.9 | 259 |
| Research-intensive | 11.7 | 261 |
| wave | missing_percent |
|---|---|
| 1 | 3.9 |
| 2 | 10.7 |
| 3 | 19.4 |
| 4 | 27.0 |
| engagement_w1 | outcome_w1 | baseline_anxiety | motivation | engagement_w2 | outcome_w2 | engagement_w3 | outcome_w3 | engagement_w4 | outcome_w4 | V11 | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| X233 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 |
| X15 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 |
| X12 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 |
| X26 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 2 |
| X12.1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 |
| X1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 3 |
| X11 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 |
| X1.1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 1 | 2 |
| X1.2 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 2 |
| X28 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 4 |
| X10 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 |
| X3 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 3 |
| X2 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 2 |
| X1.3 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 2 |
| X2.1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 5 |
| X10.1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 |
| X3.1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 3 |
| X1.4 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 1 | 1 | 2 |
| X32 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 6 |
| X22 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| X4 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 2 |
| X1.5 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 3 |
| X1.6 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 2 |
| X1.7 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 1 | 1 | 1 | 2 |
| X1.8 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 4 |
| X18 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| X1.9 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 2 |
| X2.2 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 3 |
| X1.10 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 3 |
| X6 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 5 |
| X4.1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 7 |
| X1.11 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 4 |
| X1.12 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 8 |
| X13 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| X3.2 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 2 |
| X1.13 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 3 |
| X1.14 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 2 |
| X3.3 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 5 |
| X2.3 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 7 |
| X1.15 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 2 |
| X2.4 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 2 |
| X7 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| X1.16 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 2 |
| X3.4 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 3 |
| X1.17 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 2 |
| X1.18 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 2 |
| X5 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 5 |
| X1.19 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 2 |
| X1.20 | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 3 |
| X2.5 | 0 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 2 |
| X1.21 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 8 |
| X1.22 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 2 |
| X1.23 | 0 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 3 |
| X1.24 | 0 | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 1 | 5 |
| X | 26 | 29 | 36 | 36 | 55 | 61 | 103 | 105 | 144 | 147 | 742 |
Monotone missingness means that once a participant drops out, all later scheduled measurements are absent. Intermittent missingness creates gaps followed by later observed values. Both occur in this example, so a method designed only for monotone dropout would not describe the full dataset.
In this synthetic study, missingness is associated with observed program group, first-generation status, earlier distress, engagement, and wave. Those variables therefore belong in the imputation model. The simulation also contains a small unobserved contribution, which motivates the later delta sensitivity analysis.
| variable | method |
|---|---|
| baseline_anxiety | pmm |
| motivation | polr |
| outcome_w1 | pmm |
| outcome_w2 | pmm |
| outcome_w3 | pmm |
| outcome_w4 | pmm |
| sleep_w1 | pmm |
| sleep_w2 | pmm |
| sleep_w3 | pmm |
| sleep_w4 | pmm |
| engagement_w1 | polr |
| engagement_w2 | polr |
| engagement_w3 | polr |
| engagement_w4 | polr |
| employed_w1 | logreg |
| employed_w2 | logreg |
| employed_w3 | logreg |
| employed_w4 | logreg |
| trt_anxiety | ~I(treatment_num * baseline_anxiety) |
| target | predictor | included |
|---|---|---|
| baseline_anxiety | age | TRUE |
| baseline_anxiety | first_generation | TRUE |
| baseline_anxiety | outcome_w1 | TRUE |
| baseline_anxiety | program_group | TRUE |
| baseline_anxiety | treatment_num | TRUE |
| outcome_w4 | age | TRUE |
| outcome_w4 | first_generation | TRUE |
| outcome_w4 | outcome_w1 | TRUE |
| outcome_w4 | program_group | TRUE |
| outcome_w4 | treatment_num | TRUE |
Predictive mean matching is used for continuous variables to retain plausible observed values. Logistic regression is used for binary variables, proportional-odds regression for ordered variables, and passive imputation keeps the treatment-by-anxiety product synchronized with its components.
The imputation model includes the outcomes, all substantive
predictors, the interaction components, repeated measures from every
wave, design variables, and useful auxiliary variables.
dropout_wave documents the observed pattern but is excluded
because it deterministically identifies blocks of missing later-wave
values. Excluding the outcome from an imputation model merely because it
is called an “outcome” can attenuate associations.
The imputation model is wide: every wave has its own column, allowing outcomes and auxiliaries across waves to predict one another. Each completed dataset is then reshaped to long form and analyzed with a participant random intercept. This preserves the fixed four-wave structure without treating repeated observations as independent during imputation.
For many occasions, unequal time spacing, participants nested in
sites, random slopes that drive missingness, or three-level data, use a
multilevel imputation method that explicitly identifies the cluster and
distinguishes within- from between-cluster predictors. Suitable tools
can include mice two-level methods, mitml, or
jomo. Match the imputation structure to the most complex
substantive model rather than adding a cluster identifier as an ordinary
numeric predictor.
| note |
|---|
| No logged events |
mice creates 20 completed datasets rather than one
filled-in dataset. Estimates are fit in every completed dataset and
combined with Rubin’s rules so that standard errors reflect both
within-imputation and between-imputation uncertainty.
Convergence is a property of the algorithm, not proof that the imputation model is correct. Extend the iterations if chains drift, remain separated, or change systematically near the end.
Imputed values need not reproduce the marginal observed distribution exactly. Large unexplained differences, spikes, impossible categories, or implausibly narrow variability should prompt review of coding, methods, predictors, and transformations.
| variable | rule | violations |
|---|---|---|
| baseline_anxiety | 0 to 30 | 0 |
| outcome_w1 | 0 to 100 | 0 |
| outcome_w2 | 0 to 100 | 0 |
| outcome_w3 | 0 to 100 | 0 |
| outcome_w4 | 0 to 100 | 0 |
| sleep_w1 | 0 to 14 | 0 |
| sleep_w2 | 0 to 14 | 0 |
| sleep_w3 | 0 to 14 | 0 |
| sleep_w4 | 0 to 14 | 0 |
| employed_w1 | 0 or 1 | 0 |
| employed_w2 | 0 or 1 | 0 |
| employed_w3 | 0 or 1 | 0 |
| employed_w4 | 0 or 1 | 0 |
| motivation | 1, 2, 3, 4, 5 | 0 |
| engagement_w1 | 1, 2, 3, 4, 5 | 0 |
| engagement_w2 | 1, 2, 3, 4, 5 | 0 |
| engagement_w3 | 1, 2, 3, 4, 5 | 0 |
| engagement_w4 | 1, 2, 3, 4, 5 | 0 |
Plausibility rules must come from the instrument, protocol, and codebook. They should catch impossible values without trimming merely unusual but credible observations.
| term | estimate | std.error | statistic | df | p.value | 2.5 % | 97.5 % | conf.low | conf.high |
|---|---|---|---|---|---|---|---|---|---|
| (Intercept) | 53.071 | 2.011 | 26.393 | 440.392 | 0.000 | 49.119 | 57.023 | 49.119 | 57.023 |
| treatment_num | -2.042 | 1.788 | -1.142 | 776.878 | 0.254 | -5.553 | 1.468 | -5.553 | 1.468 |
| factor(wave)2 | -2.167 | 0.471 | -4.602 | 1190.777 | 0.000 | -3.090 | -1.243 | -3.090 | -1.243 |
| factor(wave)3 | -4.243 | 0.515 | -8.235 | 318.841 | 0.000 | -5.256 | -3.229 | -5.256 | -3.229 |
| factor(wave)4 | -5.073 | 0.542 | -9.359 | 215.136 | 0.000 | -6.142 | -4.005 | -6.142 | -4.005 |
| baseline_anxiety | 1.261 | 0.085 | 14.833 | 714.741 | 0.000 | 1.094 | 1.428 | 1.094 | 1.428 |
| motivation_numeric | -0.224 | 0.309 | -0.725 | 490.259 | 0.469 | -0.832 | 0.384 | -0.832 | 0.384 |
| sleep | -1.173 | 0.153 | -7.643 | 150.437 | 0.000 | -1.476 | -0.870 | -1.476 | -0.870 |
| engagement_numeric | -1.361 | 0.154 | -8.825 | 258.271 | 0.000 | -1.665 | -1.057 | -1.665 | -1.057 |
| treatment_num:factor(wave)2 | -2.083 | 0.659 | -3.161 | 1345.351 | 0.002 | -3.375 | -0.790 | -3.375 | -0.790 |
| treatment_num:factor(wave)3 | -3.639 | 0.709 | -5.137 | 427.324 | 0.000 | -5.032 | -2.247 | -5.032 | -2.247 |
| treatment_num:factor(wave)4 | -6.370 | 0.742 | -8.580 | 248.638 | 0.000 | -7.832 | -4.907 | -7.832 | -4.907 |
| treatment_num:baseline_anxiety | 0.000 | 0.118 | 0.001 | 654.406 | 0.999 | -0.231 | 0.232 | -0.231 | 0.232 |
The outcome model is fit separately in all 20 completed datasets. The pooled estimates reflect uncertainty about the missing values. The treatment-by-wave terms address whether change differs between conditions, while the participant random intercept accounts for repeated observations.
| term | estimate | std.error | statistic | df | p.value | 2.5 % | 97.5 % | conf.low | conf.high |
|---|---|---|---|---|---|---|---|---|---|
| (Intercept) | 1.839 | 0.698 | 0.873 | 174.558 | 0.384 | 0.464 | 7.292 | 0.464 | 7.292 |
| treatment_num | 1.203 | 0.208 | 0.887 | 176.361 | 0.376 | 0.798 | 1.814 | 0.798 | 1.814 |
| baseline_anxiety | 0.922 | 0.030 | -2.679 | 162.418 | 0.008 | 0.869 | 0.979 | 0.869 | 0.979 |
| as.numeric(motivation) | 1.046 | 0.105 | 0.431 | 168.131 | 0.667 | 0.850 | 1.288 | 0.850 | 1.288 |
| outcome_w1 | 1.008 | 0.013 | 0.579 | 173.780 | 0.563 | 0.982 | 1.034 | 0.982 | 1.034 |
Exponentiated coefficients are pooled odds ratios. A binary outcome does not justify filling missing values with the most common category; the imputation model should preserve uncertainty and associations with other variables.
| term | mi_estimate | mi_conf_low | mi_conf_high | fmi | cc_estimate | cc_conf_low | cc_conf_high |
|---|---|---|---|---|---|---|---|
| (Intercept) | 53.071 | 49.119 | 57.023 | 0.182 | 55.401 | 49.847 | 60.954 |
| treatment_num | -2.042 | -5.553 | 1.468 | 0.121 | -1.652 | -6.466 | 3.161 |
| factor(wave)2 | -2.167 | -3.090 | -1.243 | 0.079 | -2.304 | -3.726 | -0.882 |
| factor(wave)3 | -4.243 | -5.256 | -3.229 | 0.223 | -4.132 | -5.563 | -2.702 |
| factor(wave)4 | -5.073 | -6.142 | -4.005 | 0.282 | -5.049 | -6.496 | -3.601 |
| baseline_anxiety | 1.261 | 1.094 | 1.428 | 0.129 | 1.173 | 0.913 | 1.432 |
| motivation_numeric | -0.224 | -0.832 | 0.384 | 0.170 | -0.547 | -1.342 | 0.249 |
| sleep | -1.173 | -1.476 | -0.870 | 0.344 | -1.343 | -1.727 | -0.958 |
| engagement_numeric | -1.361 | -1.665 | -1.057 | 0.253 | -1.417 | -1.831 | -1.003 |
| treatment_num:factor(wave)2 | -2.083 | -3.375 | -0.790 | 0.067 | -1.149 | -3.096 | 0.798 |
| treatment_num:factor(wave)3 | -3.639 | -5.032 | -2.247 | 0.186 | -3.179 | -5.133 | -1.226 |
| treatment_num:factor(wave)4 | -6.370 | -7.832 | -4.907 | 0.259 | -5.059 | -7.015 | -3.104 |
| treatment_num:baseline_anxiety | 0.000 | -0.231 | 0.232 | 0.138 | -0.053 | -0.390 | 0.284 |
The complete-case model uses only participants with every required value at all four waves. Differences from the imputed analysis may reflect changed sample composition, lower precision, chance, or the imputation assumptions. Agreement does not prove either analysis is unbiased, and disagreement should be investigated rather than resolved by choosing the preferred estimate.
| delta | term | estimate | std_error | conf_low | conf_high | fraction_missing_information |
|---|---|---|---|---|---|---|
| 0 | treatment_num | -8.659 | 0.727 | -10.084 | -7.235 | 0.321 |
| -2 | treatment_num | -8.617 | 0.729 | -10.046 | -7.188 | 0.317 |
| -5 | treatment_num | -8.554 | 0.748 | -10.020 | -7.088 | 0.297 |
The delta analysis asks how the treatment estimate changes if unobserved wave-four distress is systematically lower than its MAR-based imputation by 2 or 5 points. The direction and magnitude must be chosen from subject-matter knowledge, prior studies, or a range that stakeholders consider consequential. Sensitivity analysis does not identify the true missing values; it shows how dependent the conclusion is on an unverifiable assumption.
The synthetic dataset included 520 participants measured across 4 planned waves. Across the designated analysis variables, 14.3% of cells were missing. outcome_w4 had the highest variable-level missingness (28.3%). Missingness increased across waves and differed modestly by observed participant characteristics, making a blanket MCAR explanation difficult to defend. These patterns support including design variables, earlier outcomes, later outcomes, and auxiliary measures in the imputation model, while leaving MAR as an explicit assumption. Twenty imputations were created using predictive mean matching for continuous variables, logistic regression for binary variables, proportional-odds regression for ordinal variables, and passive updating for the treatment-by-anxiety product. Chain behavior, observed-versus-imputed distributions, logged events, and range rules were reviewed before fitting the substantive models. The pooled treatment-by-wave coefficient at the final wave was -6.37, 95% CI [-7.83, -4.91], p < .001. This estimate describes the synthetic model and should not be generalized to real doctoral-student populations. The complete-case and multiply imputed estimates were compared as a diagnostic rather than as a contest between methods. A delta-adjustment sensitivity analysis then examined whether the focal treatment estimate changed under increasingly pessimistic assumptions about unobserved wave-four distress. Conclusions should report both the primary MAR-based analysis and any sensitivity values that would alter the substantive interpretation.
| file |
|---|
| complete_case_vs_imputation.csv |
| delta_sensitivity.csv |
| imputation_methods.csv |
| imputation_predictor_matrix.csv |
| imputation_range_audit.csv |
| mids_object.rds |
| missingness_by_case.csv |
| missingness_by_group.csv |
| missingness_by_variable.csv |
| missingness_by_wave.csv |
| pooled_logistic_model.csv |
| pooled_mixed_model.csv |
| synthetic_complete_reference.csv |
| synthetic_longitudinal_missing.csv |