Purpose

This workflow creates a documented, analysis-ready dataset while preserving the raw source. It separates automated checks from decisions that require researcher judgment.

The workflow covers:

  1. Importing data
  2. Reviewing structure and missingness
  3. Identifying duplicate IDs
  4. Checking valid ranges and potential outliers
  5. Recoding categorical variables
  6. Parsing dates
  7. Exporting cleaned data and audit records

The included dataset is synthetic. Replace it with your own CSV only after the example runs successfully.

User settings

Edit this section before using the workflow with another dataset. Keep variable names lowercase with underscores after checking the imported-name table.

Importing data

Read the raw file

The import step applies declared missing-value codes and standardizes column names. raw_data remains unchanged after this section.

Imported and standardized column names
original_name cleaned_name
participant_id participant_id
Age age
Gender Raw gender_raw
Program Stage Raw program_stage_raw
Stress Score stress_score
Support Score support_score
Outcome Score outcome_score
Completion Date completion_date

Import summary

measure value
Imported rows 81
Imported variables 8
Unique nonmissing IDs 79
Missing IDs 0

Variable structure

variable class distinct_values
source_row integer 81
participant_id character 79
age character 29
gender_raw character 7
program_stage_raw character 7
stress_score character 31
support_score character 24
outcome_score character 34
completion_date character 65

Missingness review

By variable

Missingness is summarized before recoding or record removal. This reveals missing values present in the source as well as values converted from the declared missing codes.

variable missing_n missing_percent
age 1 1.2
stress_score 1 1.2
support_score 1 1.2
completion_date 0 0.0
gender_raw 0 0.0
outcome_score 0 0.0
participant_id 0 0.0
program_stage_raw 0 0.0
source_row 0 0.0

By record

source_row participant_id missing_n missing_percent
4 P004 1 12.5
7 P007 1 12.5
16 P016 1 12.5
1 P001 0 0.0
2 P002 0 0.0
3 P003 0 0.0
5 P005 0 0.0
6 P006 0 0.0
8 P008 0 0.0
9 P009 0 0.0
10 P010 0 0.0
11 P011 0 0.0
12 P012 0 0.0
13 P012 0 0.0
14 P014 0 0.0

Duplicate ID review

Duplicate identifiers can indicate repeated records, repeated measurements, or data-entry errors. Do not remove them without considering the study design.



| source_row|participant_id |age |gender_raw       |program_stage_raw |stress_score |support_score |outcome_score |completion_date | duplicate_count|
|----------:|:--------------|:---|:----------------|:-----------------|:------------|:-------------|:-------------|:---------------|---------------:|
|         12|P012           |36  |Another Identity |writing           |25           |27            |85            |2026-01-21      |               3|
|         13|P012           |30  |M                |Data Collection   |16           |27            |67            |2026-05-11      |               3|
|         81|P012           |36  |Another Identity |writing           |25           |27            |85            |2026-01-21      |               3|

Prepare a working copy

All subsequent transformations are made to clean_data. Character values are trimmed and repeated internal spaces are reduced.



| source_row|participant_id |variable      |original_value |issue                                           |
|----------:|:--------------|:-------------|:--------------|:-----------------------------------------------|
|         32|P032           |outcome_score |not recorded   |Nonmissing value could not be parsed as numeric |

Range and outlier checks

Valid-range violations

A valid-range violation is impossible or inadmissible under the study’s coding rules. The rules come from the user settings and should match the codebook.



| source_row|participant_id |variable      | value|expected_range |
|----------:|:--------------|:-------------|-----:|:--------------|
|         17|P017           |age           |   140|18 to 100      |
|         20|P020           |stress_score  |    45|0 to 40        |
|         31|P031           |outcome_score |   110|0 to 100       |

IQR-based potential outliers

The 1.5 × IQR rule identifies observations that warrant review. It does not prove that a value is erroneous. Potential outliers are retained and flagged.



| source_row|participant_id |variable | value| lower_fence| upper_fence|
|----------:|:--------------|:--------|-----:|-----------:|-----------:|
|         59|P059           |age      |    53|          19|          51|

Distribution plots

Variable recoding

Gender

The recode uses explicit source values. Unrecognized nonmissing values remain missing in the recoded variable and appear in the review table.

No records were flagged.

Doctoral program stage



| source_row|participant_id |variable      |original_value |
|----------:|:--------------|:-------------|:--------------|
|         25|P025           |program_stage |other          |

Combined recode review



| source_row|participant_id |variable      |original_value |
|----------:|:--------------|:-------------|:--------------|
|         25|P025           |program_stage |other          |

Date review



| source_row|participant_id |original_value |issue                                  |
|----------:|:--------------|:--------------|:--------------------------------------|
|         26|P026           |2026-13-01     |Date could not be parsed as YYYY-MM-DD |

Record-level flags and duplicate decision

Analysis-ready export

Final missingness

variable missing_n missing_percent
age 2 2.5
outcome_score 2 2.5
stress_score 2 2.5
completion_date 1 1.3
program_stage 1 1.3
support_score 1 1.3
date_review_flag 0 0.0
gender 0 0.0
numeric_parse_flag 0 0.0
outlier_flag 0 0.0
participant_id 0 0.0
range_flag 0 0.0
recode_review_flag 0 0.0
source_row 0 0.0

Audit summary

measure value
Imported rows 81
Final rows 79
Duplicate rows removed 2
Range violations found 3
Potential outlier cells flagged 1
Unmatched categorical values 1
Unparseable dates 1
Numeric values that could not be parsed 1
Missing cells in final export 9

Cleaning decisions

issue configured_action
Declared missing-value codes Convert to missing during import
Duplicate IDs keep_first
Valid-range violations set_missing
IQR-based potential outliers Flag for researcher review; retain value
Unmatched categorical values Set recoded value to missing and flag
Unparseable dates Set parsed date to missing and flag
Unparseable numeric values Set parsed value to missing and flag

Export files

Exported analysis-ready data and audit files to: C:\Users\nedim.yel\OneDrive - University of Massachusetts Boston\Desktop\AI\Dissertation Stats Helper\Premium Products\Data Cleaning and Audit_v_2026_09_22\Output

Preview

source_row participant_id age gender program_stage stress_score support_score outcome_score completion_date outlier_flag range_flag recode_review_flag date_review_flag numeric_parse_flag
1 P001 22 Man Writing 16 37 65 2026-02-06 FALSE FALSE FALSE FALSE FALSE
2 P002 39 Nonbinary Defense 25 22 65 2026-02-02 FALSE FALSE FALSE FALSE FALSE
3 P003 43 Woman Data collection 20 22 68 2026-02-26 FALSE FALSE FALSE FALSE FALSE
4 P004 NA Woman Defense 39 34 68 2026-03-27 FALSE FALSE FALSE FALSE FALSE
5 P005 38 Woman Analysis 28 29 78 2026-05-29 FALSE FALSE FALSE FALSE FALSE
6 P006 33 Prefer not to answer Defense 17 28 79 2026-03-05 FALSE FALSE FALSE FALSE FALSE
7 P007 34 Man Proposal NA 23 74 2026-04-26 FALSE FALSE FALSE FALSE FALSE
8 P008 48 Man Writing 30 33 78 2026-04-16 FALSE FALSE FALSE FALSE FALSE
9 P009 35 Woman Candidacy 27 32 66 2026-02-05 FALSE FALSE FALSE FALSE FALSE
10 P010 32 Nonbinary Data collection 10 36 66 2026-01-11 FALSE FALSE FALSE FALSE FALSE

Methods write-up template

The data were imported and audited using a reproducible R workflow. User-defined missing-value codes were converted to missing values, variable names and character spacing were standardized, and duplicate participant identifiers were reviewed. Numeric values were compared with prespecified valid ranges, and values outside those ranges were set to missing. Potential outliers were identified using the 1.5 × interquartile-range rule and retained with record-level flags. Categorical variables were recoded using explicit mappings, and unmatched values were flagged for review. The final analysis-ready dataset contained 79 records and preserved source-row identifiers so that all changes could be traced to the imported file.

Interpretation guidance

  • A missing value is not automatically a data error. Describe its pattern and likely mechanism before choosing deletion or imputation.
  • An outlier is not automatically invalid. Confirm the source, measurement process, and influence on the planned model.
  • A duplicate ID may represent an error or a legitimate repeated observation. Match the decision to the research design.
  • Keep the exported issue tables with the project so committee members and collaborators can reconstruct the cleaning decisions.

References

  • Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley.
  • van Buuren, S. (2018). Flexible Imputation of Missing Data (2nd ed.). CRC Press.
  • Wickham, H., et al. (2019). Welcome to the tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686
Dissertation Stats Helper · Reproducible statistical guidance for doctoral research · Cleaning rules should follow the study codebook and preserve a traceable record of every decision.