Purpose

A dissertation in practice asks a different question from a traditional study: did the changes we tested lead to improvement in our setting, and what did we learn from each cycle? The evidence is usually a weekly measure tracked over time, a small group of students or staff measured before and after, and feedback collected during each Plan–Do–Study–Act (PDSA) cycle. This workflow analyzes all three the way improvement scientists do: run charts and control charts to separate real change from week-to-week noise, careful small-sample pre/post comparisons, and a cycle-by-cycle summary that joins the numbers to what participants said.

The included data are entirely synthetic. They describe a fictional ninth-grade team testing three PDSA cycles to raise on-time arrival and reading growth. Run the complete example before adapting it. Improvement data show what happened in one setting; they support learning and local decisions, not general causal claims.

Improvement data answer different questions

Question Evidence in this workflow What it can and cannot show
Did the process change over time? Run chart with a baseline median and signal rules Detects non-random change; cannot say why it happened
Is the change beyond ordinary variation? p-chart and individuals (I-MR) chart Separates common-cause from special-cause variation
Did the focal group improve? Paired pre/post with effect sizes and a bootstrap interval Describes change in these students; no comparison group
What did each cycle teach us? Coded feedback by cycle joined to the data signals Supports Study and Act decisions; themes need interpretation

Improvement projects often need an institutional determination of whether they are quality improvement or human-subjects research. Settle that with your IRB before collecting data, and de-identify student records.

User settings

Simulate and import the example

Rows: 30
Columns: 6
$ week             <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16…
$ phase            <fct> Baseline, Baseline, Baseline, Baseline, Baseline, Bas…
$ enrolled         <dbl> 124, 115, 123, 124, 123, 116, 120, 116, 115, 123, 125…
$ on_time          <dbl> 93, 84, 91, 92, 100, 91, 88, 91, 84, 91, 95, 97, 84, …
$ exit_ticket_mean <dbl> 6.35, 5.83, 6.55, 5.76, 6.19, 5.83, 5.66, 6.55, 5.98,…
$ rate             <dbl> 0.7500000, 0.7304348, 0.7398374, 0.7419355, 0.8130081…
Rows: 26
Columns: 5
$ student_id     <chr> "S01", "S02", "S03", "S04", "S05", "S06", "S07", "S08",…
$ wcpm_pre       <dbl> 120, 100, 80, 88, 107, 77, 117, 131, 115, 110, 110, 65,…
$ wcpm_post      <dbl> NA, 101, 95, 94, 110, 85, 124, 141, 123, 119, 105, 90, …
$ belonging_pre  <dbl> 3, 2, 1, 2, 2, 2, 4, 2, 4, 1, 3, 3, 3, 3, 3, 2, 1, 4, 3…
$ belonging_post <dbl> 3, 3, 2, 2, 2, 3, 4, 3, 4, 1, 3, 4, 4, 3, 3, 1, 1, 4, 3…
Rows: 113
Columns: 5
$ excerpt_id <chr> "E001", "E002", "E003", "E004", "E005", "E006", "E007", "E0…
$ cycle      <fct> PDSA 1, PDSA 1, PDSA 1, PDSA 1, PDSA 1, PDSA 1, PDSA 1, PDS…
$ source     <chr> "Student", "Student", "Student", "Teacher", "Student", "Stu…
$ theme      <chr> "Clear morning routine", "Time pressure in class", "Clear m…
$ valence    <chr> "Support", "Barrier", "Support", "Support", "Barrier", "Bar…

Data audit

Check Value
Weeks of data 30
Weeks by phase Baseline: 10; PDSA 1: 6; PDSA 2: 7; PDSA 3: 7
Focal students 26
Students with both reading scores 24
Coded feedback excerpts 113

A run chart needs enough baseline points to describe ordinary variation; ten or more is a common guide, and fewer should be named as a limitation. Students without both scores are excluded from the paired analysis and reported, not silently dropped.

Part 1: Weekly measures over time

Run chart

Run-chart rules applied to on-time arrival rate; centre line = Baseline median (74.1%)
Rule Result Signal
Shift (6+ points on one side of the baseline median) Weeks 16–30 Yes
Trend (5+ points rising or falling) Weeks 15, 16, 17, 18, 19 Yes
Crossings of the median (too few suggests clustering) 10 crossings; fewer than 10 would be a signal No
Longest run 15 points Yes

The median comes from the baseline weeks and is extended forward, so later points are judged against the process as it was before any change. A signal says the pattern is unlikely to be random variation around that baseline; it does not identify which change caused it. Points exactly on the median are skipped when counting shifts, and repeated values are skipped when counting trends.

p-chart for the weekly proportion

Special-cause signals against Baseline limits
Signal Weeks
Points beyond the 3-sigma limits 24, 25, 28, 29
8+ consecutive points on one side of the centre line 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30
Pooled rate by phase with 95% Wilson confidence intervals
Phase Weeks On time Enrolled Rate 95% CI lower 95% CI upper
Baseline 10 905 1199 75.5% 73.0% 77.8%
PDSA 1 6 567 732 77.5% 74.3% 80.3%
PDSA 2 7 693 830 83.5% 80.8% 85.9%
PDSA 3 7 734 835 87.9% 85.5% 89.9%

A p-chart fits a proportion with a varying number of students each week. Keep the baseline limits until a signal has been sustained and the team has decided that the process has changed; then calculate new limits for the new phase and say so. Recalculating limits after every change hides the very signals you are looking for.

Individuals (I-MR) chart for a continuous weekly measure

Quantity Value
Centre line 6.10
Lower limit 4.69
Upper limit 7.51
Average moving range 0.53
Weeks beyond the limits 23, 24, 25, 26, 29, 30
Weeks in an 8+ point run 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30

The individuals chart estimates ordinary variation from week-to-week differences rather than from the overall standard deviation, so a gradual improvement does not inflate its own limits. Check the moving-range chart first: a very large moving range points to an unusual week worth investigating before the limits are trusted.

Part 2: Focal students before and after

Reading fluency (words correct per minute): 24 students with both scores (2 excluded for a missing score)
Measure n M SD Mdn
Pre 24 101.5 22.9 110.0
Post 24 112.0 21.5 115.5
Change 24 10.6 7.8 10.5
Analysis Estimate 95% CI lower 95% CI upper Statistic df p
Paired t test (mean change) 10.58 7.3 13.87 6.66 23 < .001
Bootstrap percentile interval (mean change) 10.58 7.5 13.58
Wilcoxon signed-rank (pseudo-median change) 10.50 7.5 14.00 292.00 < .001
Effect Estimate 95% CI lower 95% CI upper
d_z (mean change / SD of change) 1.31 0.78 1.85
d_av (mean change / root mean of pre and post variances) 0.46 0.33 0.60
Students gaining ≥ 10 WCPM 0.54 0.35 0.72

With a small group, lead with the size of the change and its interval, then the share of students who reached a meaningful gain; the p-value is the least informative number here. The bootstrap interval and the Wilcoxon test check that the conclusion does not depend on the change scores being normally distributed.

Students chosen because they started low tend to score closer to the average the next time even without any change, so part of their gain is regression to the mean. Without a comparison group it cannot be separated from the effect of the change ideas; name it as a limitation and, when possible, compare with prior cohorts or the school’s typical fall-to-spring growth.

An ordinal survey item

Belonging item (1–5): direction of change
Direction n Percent
Higher 8 30.8%
Lower 1 3.8%
Same 17 65.4%
Statistic Value
Median before 2.0
Median after 3.0
Wilcoxon signed-rank p (ties excluded) .023
Rank-biserial r .78

A single Likert item is ordinal, so report medians and the share of students whose response moved up, stayed, or moved down rather than a mean. Many students giving the same answer twice (ties) is common and reduces the information the signed-rank test can use.

Part 3: What each cycle taught the team

Coded feedback excerpts by theme and cycle
Theme Valence PDSA 1 PDSA 2 PDSA 3
Advisory check-ins Support 2 9 10
Bus and transport barriers Barrier 13 5 3
Clear morning routine Support 7 10 6
Family text reminders Support 2 10 7
Peer reading partners Support 2 2 9
Time pressure in class Barrier 8 5 3

PDSA cycle summary: what was tested, what the data and feedback showed, and what the team decided
Cycle Weeks Change tested (Plan/Do) Prediction On-time rate Weeks meeting a run-chart rule Weeks with a control-chart signal Exit ticket (mean) Most frequent feedback (Study) Barrier excerpts Decision (Act)
PDSA 1 11–16 Door greeting and a five-minute warm-up routine On-time arrival rises above 80% 77.5% 2 of 6 1 of 6 6.57 Bus and transport barriers (13); Time pressure in class (8) 62% Adapt: add a response to transport barriers
PDSA 2 17–23 Advisory check-ins plus family text reminders after two late arrivals Sustained shift toward the 85% aim 83.5% 7 of 7 7 of 7 6.92 Clear morning routine (10); Family text reminders (10) 24% Adopt for the ninth-grade team
PDSA 3 24–30 Add peer reading partners to the warm-up, keeping cycle 2 changes Arrival gains hold; reading practice improves 87.9% 7 of 7 7 of 7 7.65 Advisory check-ins (10); Peer reading partners (9) 16% Adopt and plan a spread test

When a signal begins in the last week or two of a cycle, as it can here, look at when each change was fully in place before crediting one cycle; changes often take time to reach their effect. The decision column is recorded by the team, not computed. Keep the prediction that was written in the Plan step next to the result: comparing prediction with outcome is where cycle learning happens, and it is often more convincing to a committee than any single statistic.

Dynamic results template

Weekly outcome. During 10 baseline weeks, the on-time arrival rate had a median of 74.1% (pooled 75.5%). A shift of 15 consecutive weeks above the baseline median began in week 16, during PDSA 1. Against baseline control limits, special-cause signals appeared in weeks 16–30. In PDSA 3, the pooled rate was 87.9% (95% CI 85.5% to 89.9%), compared with an aim of 85%.

Cycle learning. In PDSA 1 (door greeting and a five-minute warm-up routine), 2 of 6 weeks met a run-chart rule, so evidence of change was limited or came late in the cycle; feedback centered on bus and transport barriers, and the team decided to adapt. PDSA 2 (advisory check-ins plus family text reminders after two late arrivals) coincided with the sustained shift (7 of 7 weeks met a run-chart rule; on-time rate 83.5%); the team decided to adopt for the ninth-grade team. In PDSA 3, the rate was 87.9% and the most frequent feedback concerned advisory check-ins.

Focal students. Among 24 focal students with both scores (2 excluded), reading fluency (words correct per minute) rose from M = 101.5 to M = 112.0, a mean gain of 10.6 (95% CI [7.3, 13.9]; bootstrap [7.5, 13.6]), dav = 0.46. 13 of 24 students (54%) gained at least 10 WCPM. Lower-scoring students gained more (r = -.34), consistent in part with regression to the mean, and without a comparison group these gains cannot be attributed to the change ideas alone.

Edit this text before using it: describe your own change ideas and measures, keep the team’s reasoning for each decision, and state plainly what the design can and cannot show.

Reporting checklist

This list follows the logic of SQUIRE 2.0 and common dissertation-in-practice structures.

  • Describe the problem of practice, the local context, and the aim (what, by how much, by when).
  • Show the theory of improvement, such as a driver diagram, and how each change idea connects to it.
  • Define outcome, process, and balancing measures, how they were collected, and how often.
  • Report the baseline period and how the centre line and limits were set.
  • For each PDSA cycle, report the change, the prediction, what happened, what was learned, and the decision.
  • Present run and control charts with the signal rules you used, and name any limits you recalculated and why.
  • Report small-sample pre/post results with effect sizes, intervals, and the share reaching a meaningful gain.
  • Explain how qualitative feedback was coded and how it informed the Study and Act steps.
  • Address limits: no comparison group, history and maturation, regression to the mean, measurement changes, attrition, and the difference between local improvement and general causal claims.
  • State the IRB or quality-improvement determination and how student data were de-identified.

Exports

file
feedback_theme_counts.csv
focal_students_change.csv
package_versions.csv
pdsa_cycle_summary.csv
phase_rates.csv
pre_post_effect_sizes.csv
pre_post_tests.csv
run_chart_rules.csv
run_settings.csv
weekly_measures_with_signals.csv

References

  • Anhøj, J., & Olesen, A. V. (2014). Run charts revisited: A simulation study of run chart rules for detection of non-random variation in health care processes. PLOS ONE, 9(11), e113825.
  • Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America’s schools can get better at getting better. Harvard Education Press.
  • Langley, G. J., Moen, R. D., Nolan, K. M., Nolan, T. W., Norman, C. L., & Provost, L. P. (2009). The improvement guide (2nd ed.). Jossey-Bass.
  • Ogrinc, G., Davies, L., Goodman, D., Batalden, P., Davidoff, F., & Stevens, D. (2016). SQUIRE 2.0: Revised publication guidelines from a detailed consensus process. BMJ Quality & Safety, 25(12), 986–992.
  • Perla, R. J., Provost, L. P., & Murray, S. K. (2011). The run chart: A simple analytical tool for learning from variation in healthcare processes. BMJ Quality & Safety, 20(1), 46–51.
  • Perry, J. A., Zambo, D., & Crow, R. (2020). The improvement science dissertation in practice. Myers Education Press.
  • Provost, L. P., & Murray, S. K. (2011). The health care data guide: Learning from data for improvement. Jossey-Bass.
Dissertation Stats Helper · Reproducible statistical guidance for doctoral research · Learn from each cycle; claim only what the design supports.