This is an Eval Central archive copy, find the original at evalacademy.com.
Let’s be clear: sums and frequencies are not the desired product of qualitative questions. In qualitative approaches, we want to describe, to present details and nuances and interesting outliers. But as evaluators, we need to do more than just report what is—we need to comment on what it means. In familiar evaluation terms, moving from the “what” to “so what?”
Qualitative purists may hiss at the idea of quantifying qualitative data. But as evaluators, our job is to apply evaluative thinking to our qualitative findings. Not all findings are as material as others—in other words, the one respondent who thought their nutrition class provided just the right amount of detail is likely overshadowed by the eleven who described feeling overwhelmed at the volume of information. Evaluators would be remiss not to introduce an element of quantification to their qualitative data.
Caveat: I do not intend to suggest that a higher number of respondents reporting a similar answer is always more important. Outliers and small groups matter, and understanding those outliers is a major part of why qualitative approaches are used.
But we do need to be able to describe the proportion of respondents who report similar answers.
The key to quantifying qualitative findings is consistency. Editing reports where descriptions of qualitative data included words like “a lot,” “the majority,” “many” and “most” left me wondering why those particular words were chosen. How is “a lot” different from “many?” Are “the majority” and “most” roughly the same number of respondents? And if I was asking those questions, I know our stakeholders would be asking them, too.
To give my staff concrete guidance, I found this framework… online… somewhere… maybe in 2013? (If this is your framework, or you know who created it, please let me know! I’ve been using these definitions in evaluation and reporting workshops for a few years, and have seen it used in Government of Canada documents, but without attribution.)
|
Few
|
Less than 10% of participants
|
|
Several
|
Less than 20%
|
|
Some
|
More than 20%
|
|
Many
|
Nearly 50%
|
|
A majority
|
More than 50%, but fewer than 75%
|
|
Most
|
More than 75%
|
|
Vast majority
|
Nearly all participants, with some still having different views
|
|
Unanimous, or almost all
|
All participants, or the vast majority gave similar answers and the rest did not comment
|
These definitions may work for you. Or you might take issue with some of the ranges and want to create your own. As I said before, consistency is key! Try using this framework in your next report, and include it in your methods appendix.
Sign up for our newsletter
We’ll let you know about our new content, and curate the best new evaluation resources from around the web!
We respect your privacy.
Thank you!