What are Evaluation Questions? Cartoon Glossary

This is an Eval Central archive copy, find the original at freshspectrum.com.

This is a series of posts providing quick of overviews of important topics in research and evaluation. Each post in this series will include at least 3 cartoons from my archives and at least 3 links to recommended resources. I only give quotes here and recommend that you follow the links below each quote for more detailed information.

freshspectrum cartoon by Chris Lysy. Person saying, "Does anyone else feel like we could do better?"

Key Evaluation Questions (KEQs) are the high-level questions that an evaluation is designed to answer – not specific questions that are asked in an interview or a questionnaire. Having an agreed set of Key Evaluation Questions (KEQs) makes it easier to decide what data to collect, how to analyze it, and how to report it.

KEQs usually need to be developed and agreed on at the beginning of evaluation planning – however sometimes KEQs are already prescribed by an evaluation system or a previously developed evaluation framework.

Try not to have too many Key Evaluation Questions – a maximum of 5-7 main questions will be sufficient. It might also be useful to have some more specific questions under the KEQs.

Better Evaluation – Specify the Key Evaluation Questions

freshspectrum cartoon by Chris Lysy. Person saying, "Before we develop the survey let's start with the bigger question this evaluation is designed to answer."

Other person, "I'll double check my notes, but I don't think we have one of those."

Before we decide what types of data we need (qualitative or quantitative) we need to know the bigger question of the project: What specific problem does the project or program address?

Evaluation questions, similar to research questions in academic research projects, guide the methods and tools used to collect data to understand the problem under investigation. Evaluation questions may seem intuitive, and thus be quickly developed to get to the more detailed program planning. But, without well-developed, relevant, and accurate evaluation questions, developed with stakeholders connected to the problem, projects can move around a problem without addressing the most appropriate issues.

American University – Asking Program Evaluation Questions

freshspectrum cartoon by Chris Lysy. Person saying, "We wanted an evaluation, you're just asking basic who, what, where, when, why, and how questions."
Other person responds, "That's right, before we can evaluate we need to understand what's really going on."

I keep six honest serving-men
(They taught me all I knew);
Their names are What and Why and When
And How and Where and Who.
— Rudyard Kipling (1865–1936), The Elephant’s Child

For professionals as diverse as journalists, police detectives, lawyers, and evaluators, Kipling’s five Ws and one H is the formula for full understanding and a complete report. These are descriptive, factual, and open-ended questions. None can be answered “yes” or “no.” You have to find out what happened. When first entering a program situation (for example, on a site visit), it can be helpful to begin with some basic facts to get the lay of the land. Keep it simple: Who’s proposing to do what? Where? When? How? Why?

Evaluation Flashcards developed by Michael Quinn Patton for the Otto Bremer Trust