Evaluation is for everyone

This is an Eval Central archive copy, find the original at camman-evaluation.com.

What a time to be alive. Back when we could still gather in groups to sing together, that was a lyric from a song I loved in the drop-in choir group I sang with every week. It is time now, and what a time to be alive.

Yesterday was Juneteenth. Tomorrow is National Indigenous Peoples Day. It’s the height of Pride season. It’s the summer solstice. It’s the first day of Cancer season. There’s a full moon in Capricorn on Thursday, which is also the Strawberry Moon, which is also the last supermoon of the year. Somewhere in there it’s also going to be my 35th birthday. And we are still living within crisis within crisis within crisis within crisis within crisis. What a time to be alive.

I mark the passing of time with reflection. I love to skip back through my calendar apps and my photo albums and remind myself where I was, what I was doing, what I was thinking about and trying to get done a month ago, a year ago, a decade ago. I realized yesterday that I have just passed the ten-year mark of being an evaluator.

Ten years ago last month, I began leading my first evaluation project. Still a grad student at the time and would be for another couple of years, but that was the real threshold for me. That was the moment I stepped into the role of evaluator and took on that responsibility. The work I started doing then had all the portents and shades of what I would spend the next ten years plus trying to realize in my practice. I wanted to do work that was meaningful, helpful, accessible, and exciting for people pouring their hearts into transforming their corner of the world.

It’s not yet quite the three year anniversary of when I felt truly inspired in how to do that. Three years ago, in September 2018, when I took a course called Transformative Evaluation Landscape from Kim Van Der Woerd, Sofia Vitalis, and Elders Lillian Howard and Roberta Price, and with a cohort of peers experienced for the first time a complete immersion into a space where we were asking each other and ourselves, why are we evaluating, who are we evaluating for, and according to whose values and worldviews?

I’ve told that story many times, but I don’t think I’ve shared yet the phrase that got stuck in my head and my heart by the end of that course, that I’ve been carrying ever since. I’ve realized that this phase is the thesis statement of my practice journey, my theory of change, my core principle. Evaluation is for everyone.

It took me over a year to even unpack what that sentence means to me. Evaluation is for everyone. I didn’t know what it meant in my head so much as I felt it with the certainty of my whole body. Evaluation is for everyone. That is the beating heart of my practice.

Evaluation is for everyone

means that evaluation is supposed to serve everyone. Not just the people with the money. Not just the people with the power to set policy. The power dynamics in evaluation are intense and so much of the decision-making revolves around appeasing and appealing to those with a very specific kind of power. Rarely have I seen the energy expended on engaging and satisfying funders matched with what will engage and satisfy the people directly impacted by the program and the evaluation. This is a systemic issue, not the fault of individual organizations and people trying to keep their work alive and funded, but as evaluators we play into it and collectively seem to have resigned ourselves to it being an inevitability. I remember getting into a Twitter dust-up with someone who argued that the people with the money ultimately should get to decide what they do with that money, and evaluation therefore is there to serve them first and foremost. Others out there have already done the work of unpacking a statement like that, including Edgar Villenueva in Decolonizing Wealth and the entire Equitable Evaluation framework, so I’ll mostly leave it at that, but I will also add that being able to pay for evaluation and then having evaluation influence funding is an embarrassingly obvious “success to the successful” system trap that we should be a lot more concerned about as an equity issue in evaluation. Evaluation cannot serve equity when it is extractive, taking data from some to generate recommendations and reports for others, and when it uncritically reinforces and upholds the inequitable distributions of power already present.

Evaluation is for everyone

means that everyone can and does do evaluation. Everyone is an evaluator. Evaluation is a fundamental human behaviour of learning from experience and making judgement calls based on what you were trying to do, what happened, and what you want to happen next. When I do evaluation capacity-building work, I always come from the standpoint of, “You already know how to think and work this way, you do it all the time; I’m going to offer some language and framing so that we can talk about how we’re doing it together, do it more intentionally, and feel more confident and less lost”. There is nothing about evaluation that requires a technical degree or a particular credential. I became a professional evaluator when I started doing paid evaluation work and I brought to that a host of skills and life experience I had acquired up to that point, only a subset of which came from formal evaluative training (and only a subset of that training was ultimately useful to me as a practitioner). To be a professional evaluator is to step into the responsibility of a particular role in a project (which is not always the role we think it is), but it is not to lay exclusive claim to what evaluation is as an essential human endeavour. I have never once entered a project as an evaluation consultant and found that no evaluation was happening. People are already in there, working with the data they have, using the critical and creative evaluative thinking that comes naturally to them, generating, refining, implementing, and adapting their theories of change. The quality and utility of my work increased vastly when I stopped doing evaluation like a social science researcher, mediating and creating knowledge for people, and started doing it like a participatory facilitator, hosting the spaces and opportunities for people to create the knowledge they need to do their work better and recognizing their inherent capacity and capability to do that.

Evaluation is for everyone

means that evaluation needs everyone to be part of it. It’s for all of us to do together. We can evaluate as individuals and reach individual conclusions and choices, but when the change requires collective work, our evaluation must also be collective. The things we are struggling with are huge, bigger than we can address on our own or with individual programs and organizations. For evaluation to be more than an institutionally-mandated surveillance-and-compliance exercise and to not get stuck in a space of organizational navel-gazing and toxic intellectualization, we have to notice and challenge our individualistic approaches to evaluation. This is difficult, in no small part because our funding models and the dominant cultural frameworks we’re all being forced to work within reinforce individualism at every step. But every evaluation we do is a chance to see the paradigm we’re in and look for opportunities to play with it, subvert it, push back on it, and imagine something different. What would it change about our work if we entered it with the fundamental belief that evaluation is a collective, communal act? That it’s not enough to circulate reports and products after the fact, but that the process, the evaluative act itself, belongs to and depends on a community of people invested in a change and a state of being in the world? That this is not a niche form of evaluation appropriate to special circumstances but reflective of a worldview and a belief that we deserve to live in systems that are arranged to respect our agency and our interdependence.

Evaluation is for everyone. Put it on a t-shirt.

Happy solstice, friends.