Purpose

Most dissertations in psychology and education rest on a small set of analyses: correlations, comparisons of two or more groups, tests of association between categorical variables, and regression models that ask whether a set of predictors adds explanation beyond another. This workflow runs each of them the defensible way, with effect sizes and confidence intervals, assumption checks, a rank-based alternative, APA-style tables, and results text that updates with the data.

The included data are entirely synthetic. Two example studies share one analysis plan: a psychology well-being intervention and an education teacher survey. Switch between them in the settings. Run the complete example before adapting it, and choose each test from the research question and design, not from whichever result is most favorable.

Choose the analysis from the question

Research question Analysis in this workflow Rank-based alternative Effect size reported
Are two continuous variables related? Pearson correlation Spearman correlation r with 95% CI
Do two independent groups differ? Welch t test (Student t as a check) Mann–Whitney U Hedges’ g; rank-biserial r
Did scores change within the same people? Paired t test Wilcoxon signed-rank dz and dav; rank-biserial r
Do three or more groups differ? Welch and classic one-way ANOVA, post hoc comparisons Kruskal–Wallis ω²; ε²
Do two factors combine or interact? Factorial ANOVA (Type III), simple effects — Partial η² and ω²
Are two categorical variables associated? Chi-square test of independence Fisher’s exact test when expected counts are small Cramér’s V; odds ratio for 2 × 2
Does a block of predictors add explanation? Hierarchical regression — ΔR², sr², standardized β

Each analysis uses the cases available for its own variables, and the hierarchical regression uses one common complete-case sample so that every step is compared on the same people. Missing data deserve a planned approach; see the reporting checklist.

User settings

Simulate and import the example

Rows: 240
Columns: 15
$ participant_id       <chr> "P001", "P002", "P003", "P004", "P005", "P006", "…
$ age                  <dbl> 20, 23, 27, 23, 23, 36, 21, 21, 35, 23, 32, 20, 1…
$ student_status       <fct> Undergraduate, Graduate, Graduate, Undergraduate,…
$ condition            <fct> Mindfulness, Waitlist, Mindfulness, Mindfulness, …
$ delivery             <fct> Online, Online, Online, In person, Online, Online…
$ completed_program    <fct> Did not complete, Completed, Completed, Completed…
$ referred_for_support <fct> No, No, Yes, No, No, No, No, Yes, No, No, No, No,…
$ stress               <dbl> 12, 14, 32, 21, 13, 24, 10, 8, 15, 27, 24, 8, 12,…
$ sleep_quality        <dbl> 6.9, 5.4, 3.8, 5.9, NA, 6.0, 5.4, 4.9, 6.2, 6.9, …
$ social_support       <dbl> 4.26, 3.75, 3.22, 3.11, 3.47, 2.79, 3.43, 4.71, 3…
$ self_compassion      <dbl> 3.23, 3.18, 3.32, 3.29, 2.32, 3.94, NA, 3.00, 3.8…
$ anxiety              <dbl> 2, 8, 17, 10, 13, 17, 6, 0, 11, 14, 10, 4, 5, 11,…
$ wellbeing_pre        <dbl> 52, 53, 39, 52, 71, 45, 62, 67, 57, 50, 37, 59, 6…
$ wellbeing_post       <dbl> 64, 58, 41, 60, 82, 44, 74, 67, 57, 58, 34, 69, 6…
$ wellbeing_change     <dbl> 12, 5, 2, 8, 11, -1, 12, 0, 0, 8, -3, 10, 2, 4, -…

The change score (post minus pre) is created here because the one-way and factorial examples analyze change. When groups were formed before the pretest, an ANCOVA on the posttest with the pretest as a covariate is often more precise; the choice belongs in the analysis plan.

Data audit

Records and missingness

Stage n
Imported records 240
Variable Label Missing (n) Missing (%)
participant_id Participant 0 0.0
age Age 0 0.0
student_status Student status 0 0.0
condition Condition 0 0.0
delivery Delivery 0 0.0
completed_program Program completion 0 0.0
referred_for_support Referred for support 0 0.0
stress Perceived stress 0 0.0
sleep_quality Sleep quality 9 3.8
social_support Social support 0 0.0
self_compassion Self-compassion 6 2.5
anxiety Anxiety 0 0.0
wellbeing_pre Well-being (pre) 0 0.0
wellbeing_post Well-being (post) 0 0.0
wellbeing_change Well-being change 0 0.0

Descriptive statistics

Variable n M SD Mdn Min Max Skewness
Age 240 23.91 5.26 22.00 18.00 41 0.88
Perceived stress 240 15.35 5.08 14.00 6.00 37 1.03
Sleep quality 231 6.18 1.37 6.20 2.70 10 0.11
Social support 240 3.64 0.67 3.60 1.84 5 0.10
Self-compassion 234 2.85 0.62 2.78 1.12 5 0.36
Anxiety 240 9.53 4.22 9.50 0.00 21 0.11
Well-being (pre) 240 51.53 9.28 51.50 23.00 79 0.07
Well-being (post) 240 57.21 12.24 58.00 27.00 95 0.03
Well-being change 240 5.68 7.43 6.00 -17.00 27 -0.05
Variable Level n Percent
Student status Undergraduate 152 63.3
Student status Graduate 88 36.7
Condition Waitlist 71 29.6
Condition Mindfulness 99 41.2
Condition Gratitude 70 29.2
Delivery Online 138 57.5
Delivery In person 102 42.5
Program completion Completed 193 80.4
Program completion Did not complete 47 19.6
Referred for support No 226 94.2
Referred for support Yes 14 5.8

Skewness beyond about ±1 is a prompt to look at the distribution, not an automatic reason to switch tests. With moderate samples, the t test and ANOVA are fairly robust to non-normality; the rank-based alternatives below are reported as sensitivity checks.

Correlations

Means, standard deviations, and Pearson correlations with 95% confidence intervals
Variable M SD 1 2 3 4
1. Perceived stress 15.35 5.08 —
2. Sleep quality 6.18 1.37 -.36
[-.47, -.25]		 —
3. Social support 3.64 0.67 -.38
[-.49, -.27]		 .13
[.00, .26]		 —
4. Self-compassion 2.85 0.62 -.08
[-.21, .04]		 -.01
[-.14, .12]		 .33
[.21, .44]		 —
5. Anxiety 9.53 4.22 .63
[.55, .70]		 -.36
[-.47, -.24]		 -.43
[-.53, -.32]		 -.25
[-.37, -.13]
Variable 1 Variable 2 n r 95% CI lower 95% CI upper p Spearman ρ Spearman p p (adjusted)
Perceived stress Sleep quality 231 -0.363 -0.470 -0.246 < .001 -0.353 < .001 < .001
Perceived stress Social support 240 -0.385 -0.488 -0.272 < .001 -0.443 < .001 < .001
Perceived stress Self-compassion 234 -0.085 -0.211 0.044 .196 -0.106 .104 .392
Perceived stress Anxiety 240 0.633 0.550 0.703 < .001 0.630 < .001 < .001
Sleep quality Social support 231 0.132 0.003 0.256 .046 0.118 .073 .137
Sleep quality Self-compassion 225 -0.007 -0.138 0.124 .917 -0.029 .667 .917
Sleep quality Anxiety 231 -0.361 -0.468 -0.243 < .001 -0.353 < .001 < .001
Social support Self-compassion 234 0.327 0.208 0.437 < .001 0.329 < .001 < .001
Social support Anxiety 240 -0.428 -0.526 -0.319 < .001 -0.425 < .001 < .001
Self-compassion Anxiety 234 -0.250 -0.366 -0.126 < .001 -0.257 < .001 < .001

The APA table shows r with a 95% confidence interval rather than significance stars. Holm-adjusted p-values are provided for the full set of correlations. Spearman’s ρ is a check on outliers and non-linear monotonic patterns; when the two disagree markedly, look at the scatterplot before interpreting either.

Two independent groups

Perceived stress by Student status
Group n M SD Mdn IQR
Undergraduate 152 14.16 4.80 13 5
Graduate 88 17.40 4.92 17 7
Test Statistic df p Estimate 95% CI lower 95% CI upper Effect size Effect-size type
Welch t test -4.948 178.22 < .001 -3.233 -4.523 -1.944 -0.665 Hedges’ g
Student t test (equal variances) -4.980 238.00 < .001 -3.233 -4.512 -1.954 -0.665 Hedges’ g
Mann–Whitney U 3879.000 < .001 -3.000 -4.000 -2.000 -0.420 Rank-biserial r

Differences are Undergraduate minus Graduate. The Welch test is the default because it does not assume equal variances and loses little when variances are equal; the Student test is shown only for comparison. For the Mann–Whitney test, the estimate is the Hodges–Lehmann location shift, and the rank-biserial correlation summarizes how often a score in one group exceeds a score in the other.

Paired comparison

Measure n M SD
Well-being (pre) 240 51.53 9.28
Well-being (post) 240 57.21 12.24
Well-being change 240 5.68 7.43
Test Statistic df p Estimate 95% CI lower 95% CI upper
Paired t test 11.846 239 < .001 5.679 4.735 6.624
Wilcoxon signed-rank 23191.000 < .001 5.500 4.500 6.500
Effect Estimate 95% CI lower 95% CI upper
d_z (mean change / SD of change) 0.76 0.62 0.91
d_av (mean change / root mean of pre and post variances) 0.52 0.43 0.61
Rank-biserial r 0.73 0.66 0.79

dz divides the mean change by the standard deviation of the change scores and answers “how consistent was the change?”. dav divides the mean change by √((SD²pre + SD²post) / 2), so it is closer to a between-group d and easier to compare across designs; Lakens (2013) describes a variant that averages the two SDs directly, which gives nearly the same value. Both are shown with a small-sample correction. Report which one you use. The Wilcoxon estimate is the pseudo-median of the paired differences.

Three or more groups: one-way ANOVA

Well-being change by Condition
Group n M SD
Waitlist 71 1.27 5.86
Mindfulness 99 9.60 6.54
Gratitude 70 4.61 7.24
Test Statistic df1 df2 p
Welch ANOVA 38.315 2 148.555 < .001
Classic ANOVA 34.605 2 237.000 < .001
Brown–Forsythe (Levene, median) variance check 1.563 2 237.000 .212
Kruskal–Wallis 56.055 2 < .001
Effect Estimate 95% CI lower 95% CI upper
omega squared (ANOVA) 0.219 0.131 0.304
epsilon squared (Kruskal–Wallis) 0.235 0.147 0.329
Tukey-adjusted pairwise comparisons (equal-variance model)
Contrast Estimate SE df 95% CI lower 95% CI upper t p
Waitlist - Mindfulness -8.328 1.020 237 -10.735 -5.922 -8.161 < .001
Waitlist - Gratitude -3.347 1.105 237 -5.953 -0.740 -3.028 .008
Mindfulness - Gratitude 4.982 1.025 237 2.565 7.398 4.862 < .001
Welch pairwise t tests, Holm-adjusted p-values
Waitlist Mindfulness
Mindfulness < .001
Gratitude .003 < .001
Pairwise Wilcoxon rank-sum tests, Holm-adjusted p-values
Waitlist Mindfulness
Mindfulness < .001
Gratitude .004 < .001

Welch’s ANOVA is the primary omnibus test because group variances often differ. Follow it with the Welch pairwise tests; the Tukey comparisons assume equal variances and are shown because many committees expect them. ω² is less biased than η² for one-way designs. Report post hoc comparisons only for the questions you planned, and say which correction you used.

Two factors: factorial ANOVA

Cell sizes, means, and standard deviations
Condition Delivery n M SD
Waitlist Online 42 1.05 5.83
Waitlist In person 29 1.59 5.97
Mindfulness Online 50 7.66 5.97
Mindfulness In person 49 11.57 6.56
Gratitude Online 46 2.74 7.43
Gratitude In person 24 8.21 5.36
Type III ANOVA with sum-to-zero contrasts
Term Sum of squares df Statistic p Partial η² η² lower η² upper Partial ω²
Condition 2816.476 2 35.247 < .001 0.232 0.142 0.318 0.222
Delivery 606.953 1 15.192 < .001 0.061 0.015 0.129 0.056
Condition × Delivery 214.209 2 2.681 .071 0.022 0.000 0.068 0.014
Simple effects: Delivery within each level of Condition (Holm-adjusted p in the last column)
Contrast Condition Estimate SE df 95% CI lower 95% CI upper t p p (adjusted)
Online - In person Waitlist -0.539 1.526 234 -3.545 2.468 -0.353 .724 .724
Online - In person Mindfulness -3.911 1.271 234 -6.415 -1.408 -3.078 .002 .005
Online - In person Gratitude -5.469 1.592 234 -8.605 -2.333 -3.436 < .001 .002

Type III tests with sum-to-zero contrasts keep each main effect interpretable as an average over the other factor when cell sizes are unequal. When the interaction is meaningful, interpret the simple effects and the plot before the main effects. A non-significant interaction is not evidence that effects are identical; look at its confidence interval and size.

Categorical association: chi-square and Fisher’s exact test

Counts and row percentages: Condition × Program completion
Completed Did not complete Completed (row %) Did not complete (row %)
Waitlist 63 8 88.7 11.3
Mindfulness 80 19 80.8 19.2
Gratitude 50 20 71.4 28.6
Expected counts under independence
Completed Did not complete
Waitlist 57.1 13.9
Mindfulness 79.6 19.4
Gratitude 56.3 13.7
Standardized residuals (beyond about ±2 marks cells that depart most from independence)
Completed Did not complete
Waitlist 2.10 -2.10
Mindfulness 0.13 -0.13
Gratitude -2.25 2.25

A table with small expected counts. The second table uses a rare outcome to show when the decision rule switches to Fisher’s exact test.

Counts and row percentages: Condition × Referred for support
No Yes No (row %) Yes (row %)
Waitlist 66 5 93.0 7.0
Mindfulness 93 6 93.9 6.1
Gratitude 67 3 95.7 4.3
Expected counts under independence
No Yes
Waitlist 66.9 4.1
Mindfulness 93.2 5.8
Gratitude 65.9 4.1
Standardized residuals (beyond about ±2 marks cells that depart most from independence)
No Yes
Waitlist -0.52 0.52
Mindfulness -0.13 0.13
Gratitude 0.66 -0.66
Table n χ² df χ² p Fisher p Expected counts < 5 Reported test Cramér’s V V lower V upper Odds ratio (2 × 2)
Condition × Program completion 240 6.718 2 .035 .032 0 Chi-square 0.167 0 0.284
Condition × Referred for support 240 0.503 2 .777 .830 2 Fisher’s exact 0.046 0 0.144

The decision rule used here is conservative: when any expected count is below 5, report Fisher’s exact test (exact for 2 × 2 tables, Monte Carlo with a fixed seed for larger tables). Cramér’s V is reported without bias correction so that it matches most textbooks; for a 2 × 2 table, the odds ratio from Fisher’s test is often the more useful effect size. The Pearson test is run without the Yates continuity correction.

Hierarchical regression

Stage n Excluded
Records with the outcome and all predictors 225 15
Model comparison across blocks (the same complete-case sample at every step)
Step Predictors added R² Adjusted R² ΔR² F change df1 df2 p (F change)
Block 1: background Age, Student status 0.067 0.059 0.067 7.986 2 222 < .001
Block 2: stress and sleep Perceived stress, Sleep quality 0.427 0.416 0.360 68.997 2 220 < .001
Block 3: psychosocial resources Social support, Self-compassion 0.492 0.478 0.065 13.928 2 218 < .001
Final-step coefficients: unstandardized b with 95% CI, standardized β, and squared semipartial correlation
Term Estimate SE Statistic p 95% CI lower 95% CI upper β sr²
Age 0.098 0.067 1.463 .145 -0.034 0.229 0.123 0.005
Student status: Graduate -0.459 0.750 -0.612 .541 -1.936 1.019 -0.052 0.001
Perceived stress 0.408 0.048 8.592 < .001 0.315 0.502 0.491 0.172
Sleep quality -0.468 0.163 -2.874 .004 -0.790 -0.147 -0.153 0.019
Social support -0.921 0.348 -2.647 .009 -1.607 -0.235 -0.147 0.016
Self-compassion -1.215 0.351 -3.461 < .001 -1.907 -0.523 -0.177 0.028
Variance inflation (squared GVIF^(1/(2·df)) for factors, comparable to VIF)
Term VIF
Age 3.01
Student status 3.14
Perceived stress 1.40
Sleep quality 1.22
Social support 1.32
Self-compassion 1.13

ΔR² is the proportion of outcome variance explained by a block beyond the blocks entered before it, and its F test compares the nested models. The order of blocks is a research decision that must be fixed in advance; it changes ΔR², not the final model. sr² is the variance uniquely explained by one predictor in the final model. Standardized β is computed as b × SDx / SDy, which matches the β that SPSS reports; for a factor, the b itself (a group difference in outcome units) is usually the clearer effect to report.

Regression diagnostics

Check Value
Cases with Cook’s distance above 4/n 13
Largest Cook’s distance 0.058
Cases with |standardized residual| > 3 0

Flagged cases are prompts to check data entry and substantive plausibility, not automatic deletions. If you remove or change anything, report the analysis with and without those cases.

Dynamic results template

Correlations. Pearson correlations were computed among 5 variables using all available pairs. The strongest association was between perceived stress and anxiety, r(238) = .63, 95% CI [.55, .70], p < .001 (Holm-adjusted p < .001). Spearman correlations led to the same substantive conclusions unless noted.

Two groups. Perceived stress was compared between student status groups with Welch’s t test. Undergraduate participants (M = 14.16, SD = 4.80, n = 152) and Graduate participants (M = 17.40, SD = 4.92, n = 88) differed: Graduate participants scored 3.23 points higher, 95% CI [1.94, 4.52], t(178.2) = 4.95, p < .001, Hedges’ g = 0.66 [0.40, 0.93]. A Mann–Whitney test gave a consistent result, p < .001, rank-biserial r = .42.

Paired change. Across 240 records, well-being moved from M = 51.53 (SD = 9.28) before to M = 57.21 (SD = 12.24) after. The mean change was 5.68, 95% CI [4.73, 6.62], t(239) = 11.85, p < .001, dav = 0.52 [0.43, 0.61].

Three or more groups. A Welch one-way ANOVA examined well-being change across condition groups, F(2, 148.6) = 38.32, p < .001, ω² = .22 [.13, .30]. Means were highest for Mindfulness (M = 9.60) and lowest for Waitlist (M = 1.27); pairwise comparisons used Welch t tests with Holm correction. The Kruskal–Wallis test agreed, H(2) = 56.06, p < .001.

Two factors. A 3 × 2 between-subjects ANOVA with Type III sums of squares examined well-being change. The main effect of condition was F(2, 234) = 35.25, p < .001, partial η² = .23. The condition × delivery interaction was F(2, 234) = 2.68, p = .071, partial η² = .02; simple effects of delivery at each level of condition are reported in the table above.

Categorical association. Condition × Program completion: χ²(2, N = 240) = 6.72, p = .035, Cramér’s V = .17 [.00, .28]. For Condition × Referred for support, 2 expected counts were below 5, so Fisher’s exact test was reported, p = .830.

Hierarchical regression. Anxiety was regressed on 3 blocks of predictors using the same 225 complete cases at each step. Block 1: background explained 6.7% of the variance. Each later block added explanation (see the model-comparison table); the final block (psychosocial resources) added ΔR² = .065, F(2, 218) = 13.93, p < .001. The final model explained 49.2% of the variance (adjusted R² = .48).

Edit this text before using it: keep only the analyses in your plan, replace generic phrasing with your constructs, and make sure every claim matches the design (for example, correlations and cross-sectional regressions do not establish causation).

Reporting checklist

  • State each research question and the analysis chosen for it before looking at results.
  • Report the analysis sample for every test and how missing data were handled.
  • Give descriptive statistics (M, SD, n, or counts and percentages) alongside every inferential test.
  • Report effect sizes with confidence intervals, not only p-values.
  • Name the variant: Welch or Student t, Welch or classic ANOVA, dz or dav, Pearson or Fisher.
  • Say which multiple-comparison correction you used and for which family of tests.
  • For factorial designs, state the sums-of-squares type and how you probed any interaction.
  • For hierarchical regression, justify the block order and report R², ΔR², the F change test, and final coefficients.
  • Describe assumption checks as supported, questionable, or violated, and report sensitivity analyses.

Exports

file
categorical_associations.csv
core_analyses_data_psychology.csv
correlations.csv
descriptive_statistics.csv
factorial_anova.csv
factorial_simple_effects.csv
hierarchical_regression_coefficients.csv
hierarchical_regression_steps.csv
one_way_anova.csv
package_versions.csv
run_settings.csv
t_tests_and_rank_tests.csv

References

  • American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.).
  • Delacre, M., Lakens, D., & Leys, C. (2017). Why psychologists should by default use Welch’s t-test instead of Student’s t-test. International Review of Social Psychology, 30(1), 92–101.
  • Field, A. (2024). Discovering statistics using IBM SPSS Statistics (6th ed.). Sage.
  • Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863.
  • Maxwell, S. E., Delaney, H. D., & Kelley, K. (2018). Designing experiments and analyzing data (3rd ed.). Routledge.
Dissertation Stats Helper · Reproducible statistical guidance for doctoral research · Choose each test from the question and the design.