Research Design and Statistical Consulting
George M. Diekhoff, Ph.D.

Statistics First. Data Collection Second!

Do not wait to run your statistics until you’ve collected your data.                                                                                                 
What?! How can you run the statistics before you’ve collected your data? Of course you have to have numbers in order to do any number crunching. What I meant is that it is terribly unwise to assume that you’ll be able to figure out how to run your statistics after the data have all been collected.                                                                                  

At least a quarter of the projects I work on require inordinate amounts of data reconstruction, repair, and reformatting before any statistical analysis can begin, simply because the researchers jumped straight to data collection without thinking through how the data would be analyzed. Admittedly, thinking about collecting data is a lot more fun than running the statistics, but there’s not much point in collecting data that can’t be evaluated statistically.

Here’s what I always recommend. Once you’ve gotten your research questions nailed down and you’ve thought through how you want to collect data, create a mock data file. In other words, make up some numbers that you think mimic the results you’ll get when you collect the real data according to your plan. Then see if you can run the statistics you need on your mock data. You may discover that you’ve not formatted the data file correctly and need to fix that. You may discover that the analyses you want to run require one or more additional variables. You might find that you need to code responses in a more statistics-friendly fashion. You might find out that you aren’t asking the right questions at all.                                                                                                                                                                                         
It’s easy to make the necessary changes at that point because you haven’t already started collecting any actual data.It’s a good idea to work with a statistical consultant in the planning stages of research to make sure that you’re collecting data in a way that enables the best statistical analyses possible to address your research questions.

s