This is an Eval Central archive copy, find the original at danawanzer.com
On May 25, 2020, George Floyd was killed by police officers.
This news is not new. George’s cries of “I can’t breathe” echo the same cries
by Eric Garner, who was also killed by police officers on July 17, 2014. Black,
indigenous, and people of color (BIPOC) are all disproportionately more likely to be stopped,
arrested, incarcerated, and killed by police than White people: indigenous
people are 3.5 times more likely and Black people are 3 times more likely to
be killed by police than White people.
The problem is not just with police or prisons. Segregation still exists, with both racially homogeneous schools and neighborhoods. Black and Latinx families are less likely to live in owner-occupied housing compared with White families. Voter suppression disproportionately affects BIPOC. The employment rate is around 15% lower for Black people than for White people. The end of affirmative action policies across states and colleges have led to significant declines in the number of BIPOC at large state schools. Black and Brown students are much less likely to graduate college with a degree compared to White and Asian students. People of color have poorer health outcomes (e.g., diabetes, heart disease, worse life expectancy) and are more likely to die from Covid-19. The list goes on.
I bring up this information to show the racial disparities
present in a variety of areas: health, education, voting, policing and incarceration,
housing, and more. These areas are many of those that we evaluators study.
And those programs and policies disproportionately
serve BIPOC. When a white person evaluates a program serving BIPOC, then
there is even greater need for a culturally responsive evaluation approach.
Culturally Responsive Evaluation
In 2011 the American Evaluation Association issued a statement on cultural competence in
evaluation. In it, they describe that culture affects all parts of an
evaluation, and “evaluations cannot be culture free.” It is part of our ethical
duty as evaluators to attend to culture in our evaluations and to make valid
inferences. However, to do so requires a “shared understanding within and
across cultural contexts.”
Although an evaluator can develop such a shared
understanding through deliberate practice and training, a number of
studies in cross-cultural evaluation suggest that culturally competent
evaluation is best done by evaluators who share the same cultural identity as
the evaluands. Therefore, the field of evaluation should focus on training
and promoting BIPOC evaluators so that our field can promote culturally
Indeed, culturally responsive evaluation should be the default
evaluation approach. If culture is everywhere, and no evaluation can be culture-free,
then culturally responsive evaluation is good evaluation. If we truly
care about the validity of our evaluation findings, then we need to employ
culturally responsive evaluation practices in all our evaluations.
White Culture in Evaluation
Yet this can be difficult for White evaluators like myself.
We naïvely believe that we have no culture, but we come to that conclusion
because the White culture is the dominant culture in society. White culture has
become the one in power, the one that becomes the “default” and the “norm.”
What are the characteristics
of a White dominant culture? Perfectionism, a sense of urgency,
defensiveness, quantity over quality, worship of the written word, paternalism,
either/or thinking, power hoarding, fear of open conflict, individualism,
objectivity, and a right to comfort for those in power. One need only peruse
EvalTalk to see all these characteristics come full bloom once the White
dominant culture is threatened (as evident in the early June 2020 conversations).
When White culture is the dominant culture, then the
evaluations designed by White evaluators become the default evaluation. If you
want to see the White dominant frame in evaluation, look at our predominantly
White evaluation theory tree. Look out who we call the “fathers” of evaluation.
Look at who is being cited (and,
therefore, read). Look at how we call non-White evaluation culturally
responsive evaluation (instead of just evaluation).
When AEA put together the Race and Class Dialogues in 2017,
there was incredibly low attendance across the sessions. The 2018
Member Survey revealed that of the 1,484 respondents who answered the
question 582 (39%) were not aware of them and 629 (42%) did not participate. Of
the 273 (%) that did respond, 15.8% found them “not at all useful,” 29% said
they were “somewhat useful,” and 55% said they were “useful.” If evaluators
were not oblivious, then they decided not to participate. And for those that
did participate, nearly half decided they were not very useful, despite our
statement on cultural competence. However, further reading of the 2018 Member
Survey shows that 18% of members were not aware of the statement on cultural
competence, 24% had not read it, but at least most who did take the time to
read it found it useful (68%). White evaluators need to become familiar with
these resources and incorporate them into their regular evaluation practices.
We live in a society in
which the dominant frame is that of the White, male, Christian, heterosexual,
well-educated, affluent, and able-bodied person. Each of us may identify with
none, one, some, or all of these dominant frames, and each of these dominant frames have
a form of oppression associated with it: racism, sexism, religious oppression,
heterosexism, classism, ageism, and ableism.
We should first recognize our identities for what they are.
For example, I have the privileges and systematically receive the benefits of
being a White, heterosexual, well-educated, middle-class, and able-bodied
person. I may be systematically disadvantaged by my gender, and perhaps my age,
but my privileges are numerous. And because these privileges are part of the
dominant culture, they are often taken for granted and considered “normal.”
It is the responsibility of the privileged to use that power
and privilege. Too often, we have placed the burden of change on those
experiencing the oppression. There are many things we can do to improve our
evaluation practices as White evaluators:
1. Learn about Your White Privilege
Take time to learn about the privileges you hold. This blog
post focuses on white privilege, and so here are some resources to begin
understanding your white privilege. The first step we can take as White
evaluators is to understand our White dominant culture.
Here are a list of books I have read or are highly
recommended by my evaluator colleagues:
Fragility: Why It’s So Hard for White People to Talk About Racism by Robin
- How to Be An
Antiracist by Dr. Ibram X. Kendi
You Want to Talk About Race by Ijeoma Oluo
Are All the Black Kids Sitting Together in the Cafeteria? And Other
Conversations About Race by Beverly Daniel Tatum
and White Supremacy: Combat Racism, Change the World, and Become a Good
Ancestor by Layla F. Saad
Whiteness: The Need to Talk About Race and How to Do It by Shelly Tochluk
Grandmother’s Hands: Racialized Trauma and the Pathway to Mending Our Hearts
and Bodies by Resmaa Menakem
If you’re interested in podcasts, articles, films and TV
series, or organizations to follow, I highly recommend this
I also recommend building racial equity habits. Eddie Moore
Jr. and Debby Irving created a 21-day racial equity habit
building challenge. They suggest for 21 days that you do at least one
action to further your understanding of power, privilege, supremacy,
oppression, and equity. They provide suggestions in the categories of things to
read, listen, watch, notice, connect, engage, act, reflect, and stay inspired. Although
they named it a 21-day challenge, it does not have to be 21 days nor do the
days have to be continuous. Just like we can never become culturally competent
(rather we can only do culturally competent actions), our work here is never
2. Learn about and incorporate Culturally Responsive Evaluation Practices
The AEA statement on cultural competence points to a need to
have a shared understanding. Although evaluators who share the culture of the
program and its participants are best positioned to do culturally responsive
evaluation, White evaluators can still deliberately practice and receive
training to better do culturally responsive evaluation in contexts they do not
share the culture.
There is a great deal of humility needed to do this type of
work. A humility that the program knows what is best for them. A humility to
set aside the White dominant culture and all the assumptions and norms you have
internalized from birth. It is hard work, and you will mess up. Think of
yourself as a beginner in this work and be open to learning.
3. Critically reflect on your
As you are incorporating culturally responsive evaluation
practices, you should be attending to the defaults, assumptions, and norms you
have carried into your evaluation practice. You will have many thoughts and
feelings going through this process. Create a space with other White
evaluators to discuss these thoughts and feelings and help each other through
this process. Do not rely on your BIPOC colleagues to help you through
this. They have told us for years, and they tell us now they are tired. Listen
to them and do your own work.
Some ideas of questions and comments you may want to ponder
regarding your evaluation practices:
- Think critically about for whom in your
evaluation work. For whom does the program work? For whom does the program unintentionally
(or intentionally!) harm?
- When you think of evaluation of a program, what
comes to mind? When you think of evaluation of the sameprogram
serving BIPOC, what comes to mind? Are they the same, or different? What do
you think that says about you and your evaluation practice?
- If you work with other evaluators, who are you hiring?
Do they primarily look like you? Does your evaluation team share the lived
experiences of the people in the programs you are working with?
- Who do you include in the evaluation process? Madison
concluded in her review of 20 years of New Directions for Evaluation volumes
that “underrepresented groups continue to be presented as subjects of
evaluation rather than as invested stakeholders.” The stakeholders most often
to be included in the evaluation consist primarily of the program leaders who,
like evaluators, are less likely to
- What choices do you routinely make in your
evaluation practices? Some major choices include the design and methods you
use. For each of those choices, examine how race, class, gender, sexual
orientation, and more affect the decisions you make. Refrain from thinking that
they do not make a difference. For example, the creators
of modern statistics did so to promote a eugenics agenda and data is not value-neutral,
equitable, or unbiased.
4. Critically Reflect on the Field of Evaluation
Recognize that racism is everywhere, including at the AEA conference,
on EvalTalk, and more. BIPOC evaluators have called out evaluation spaces for
being racist and unsafe spaces for them for years. Start listening to them,
believing them, and supporting them.
The AEA conference in 2019 made one step forward when it
Respite & Healing Space” for BIPOC attendees to rest, heal, and connect
with other BIPOC attendees. White allies and accomplices were invited to
contribute financially as their way to support the space. Listen and act when
other opportunities are presented for you to show your allyship for our BIPOC
If you are a teacher in evaluation, think critically
about who you accept as students and what you teach. Recruit and accept BIPOC
students into your program. Encourage your institution to hire and create
safe work environments for BIPOC. Check your syllabus: who do you assign as
readings? What topics are covered? How are race, class, gender, and other
topics infused into your curriculum? A week on culturally responsive evaluation
is a starting point, but it is not enough.
If you are a researcher of evaluation, think critically
about who you cite. Our BIPOC colleagues are not cited at the same rates as
White evaluators. Check your reference list that you have a balance of
representation by race, gender, nationality, and more.
Listening, stepping back, and critically reflecting is hard work. But what is harder and just as necessary is for us to speak up, stand up, and step up in alliance of our BIPOC colleagues. Advocate for change in your institution, in our association, and in our field in general. Advocate for change with the organizations you partner with, citing our guiding principles and statement of cultural competence when you write an evaluation proposal. Speak up when you see prejudiced behaviors or racist systems. It is time. Step up.
Do you have comments, questions, feedback, or suggestions? Add a comment below or use the contact form to email me on my website.
 I focus on the United States in this blog post because that is the nation I know best, but this is likely true for other nations as well. I mention because Khalil Bitar rightfully pointed out that these conversations are largely USA-centered, mostly reactive, and there is limited participation by the often unheard voices.