Is the high reading level appropriate? Data UX Review
This is an Eval Central archive copy, find the original at freshspectrum.com.
In this series of Data UX reviews I take real published reports and look for areas of improvement. The ultimate goal for these reviews is shared learning. This week, we discuss readability.
Today’s Report is an Evaluation Synthesis Report (2022-2023) produced by the Evaluation Office of the UN Environment Programme (UNEP). You can peruse the report on your own by following this link.
This is a long but fairly standard type of report. 114 pages plus a 20 page appendix. It feels professionally designed, with uniform structure throughout. My guess is that the evaluation team wrote the report and then handed it off to a graphic designer to organize and polish at the end.
There are charts throughout and a few icons but it is not systematically illustrated. The report also has fairly lengthy sections without any illustrations at all. It is not designed to be easy to skim and does not highlight important or critical information. On a skim your eyes will probably go from chart to chart and possibly stop to reach the chapter introductions.
There are quite a few things that we can learn from this report in terms of user experience, but let’s focus on one in particular. Readability.
Most reports fail at basic readability.
This feels like a very standard kind of global program evaluation report. Many of the final reports and synthesis reports I read feel a lot like this one. And the easiest way to show one of the biggest challenges is to assess the reading level.
Let me repeat. This is not to pick on the report authors. MANY reports I see, read a lot like this one.
Luckily, reading level assessments are super easy these days, just ask one of your handy dandy generative AI tools for some support (I used Claude’s Sonnet 4 for this one).
Average sentence length: 27.3 words (Very long – typical academic writing)
Complex vocabulary: 28.5% of words have 3+ syllables
Specific Challenges:
1. Sentence Complexity
60% of sentences exceed 20 words
Average sentence length of 27+ words is significantly above readable standards
Multiple embedded clauses and technical qualifications
2. Vocabulary Difficulty
High density of multi-syllabic technical terms
Specialized UN/environmental jargon throughout
Complex institutional terminology
3. Document Structure
Dense, academic formatting
Heavy use of abbreviations (UNEP, GEF, MTS, etc.)
Multiple cross-references and technical citations
Reading Level Interpretation:
Flesch Reading Ease Categories:
0-30: Very Difficult (Graduate level) ? This report scores 18.0
30-50: Difficult (College level)
50-60: Fairly Difficult (High school level)
Target Audience: Environmental professionals, policy makers, academic researchers, and UN system stakeholders with specialized knowledge.
Recommendations for Improved Accessibility:
Reduce sentence length to 15-20 words average
Add executive summaries with simplified language
Define technical terms in a glossary
Use more active voice instead of passive constructions
Break up dense paragraphs with bullet points and subheadings
Provide visual summaries (charts, infographics) for key findings
This document is appropriately written for its intended professional audience but would require significant simplification for broader public consumption.
According to the AI, this is appropriately written, but that’s a trap.
Just because your audience is filled with people who have advanced and insider knowledge does not mean it’s a good idea to write at a 17th grade reading level.
This is especially true for any type of guidance set to be delivered to busy, overwhelmed, policy decision makers. Or for any guidance designed to reach a global audience, many of whom not speak english as a first (or even maybe second) language.
Let’s look at some specific examples of complex language pulled from the AI analysis that gives potential revisions.
Top of Page 28
From the Evaluation Criteria section:
“This set of performance evaluation criteria have evolved over time to: a) be responsive to UNEP’s policies and areas of priority; b) make necessary areas of learning visible and c) create a comprehensive and transparent framework against which all performance aspects of UNEP’s work can be evaluated. In Table 1 below the major adjustments made in recent years are noted.”
Problems:
Technical terminology without definition
Complex list structure interrupting flow
Abstract concepts not explained
Run-on sentences
Better version:
“UNEP has updated its evaluation standards over time for three reasons: to match current priorities, to highlight key lessons, and to create clear measurement tools. Table 1 shows recent changes to these standards.”
Top of Page 44
From Chapter 3:
“Since project interventions are very diverse thematically, geographically, and in terms of their resource envelopes, conscious effort is needed to ensure that performance assessments are made in a consistent manner. In this regard, the Evaluation Office is continually aiming to improve the objectivity and comparability of its evaluation approach across UNEP projects. External evaluation consultants are routinely provided with standard Terms of Reference for project evaluations, as well as detailed guidance on evaluation processes and methods, report structure, content, and quality, to help maintain consistent approaches and performance assessments.”
Problems:
Three very long sentences (34, 25, and 44 words respectively)
Unclear pronoun references (“In this regard”)
Technical jargon not explained
Passive voice predominates
Poor paragraph unity – jumps between topics
Better version:
“UNEP projects vary widely in theme, location, and budget size. This diversity creates challenges for fair evaluation. To solve this problem, the Evaluation Office gives all external evaluators the same guidelines and standards. These tools help ensure consistent, objective assessments across all projects.”
Middle of Page 76
From the Strategic Evaluations section:
“The absence of a results-based framework with expected accomplishments, indicators (in the MTS) and also the lack a reporting system with gender results, was a major obstacle in terms of identifying gender results that can be attributed to the impact of the GPS.”
Grammatical errors (“also the lack a reporting system”)
Unclear abbreviations (MTS, GPS)
Convoluted structure
Better version:
“UNEP lacked clear goals and indicators for gender work. Without proper tracking systems, it was impossible to measure the success of the Gender Policy Strategy (GPS).”
So what is an appropriate reading level for an evaluation report?
Reading level can be a design choice. Those terms of reference things you have to check off to use some piece of software is overly complicated often by design. But I don’t think this applies to most evaluation and research reports.
The evaluation world is filled with super smart people who spent a lot of their time in school. Meaning they have pretty expansive vocabularies, often like precision, and have been rewarded for that precision in their academic studies. So when it comes to report writing, words and paragraphs will naturally get complicated.
One easy way to pick an aspirational reading level is to compare the reading levels of different publications. There are major differences between reading a People magazine, The Economist, and the journal Science.
My suggestion for most reports is to try to write for grades 10-12, but you can settle for writing that ends up in the 12-14 range.
What is this blog post’s reading level?
I thought it would be interesting the assess the readability of this particular blog post. And it came out to a 12.2 grade level
Complex vocabulary: 17.8% of words have 3+ syllables
Total words: 1,212
Total sentences: 63
Reading Level Interpretation: This falls into the “Difficult (College level)” category on the Flesch Reading Ease scale.
Assessment:
This document is a great example of practicing what you preach. The author critiques the UNEP report for being written at a 17th+ grade level, and their own writing comes in at a much more accessible 12.2 grade level – exactly in the range they recommend (grades 10-14).
Strengths of this writing:
Reasonable sentence length (19.2 words average vs. the UNEP’s 27.3 words)
Lower complexity (17.8% complex words vs. UNEP’s 28.5%)
Clear structure with headings and examples
Conversational tone that makes technical concepts accessible
Concrete examples rather than abstract concepts
Why this works well: The author successfully demonstrates that you can discuss complex evaluation methodology and data analysis without resorting to overly academic language. They use technical terms when necessary but explain them clearly, and they break up dense concepts with examples and clear formatting.
This document achieves the sweet spot the author advocates for – professional enough for the target audience (evaluation professionals, data analysts) while remaining accessible to busy decision-makers who may not have deep technical expertise.
Brief Reading Level Guide
Want to go deeper on the differences between different reading levels? Here is a quick guide.
Grade Level 8-10 (Middle/High School)
Characteristics:
Sentence structure: Short to medium sentences (10-15 words average)
Vocabulary: Common, everyday words with occasional complex terms explained
Concepts: Straightforward topics, concrete rather than abstract
Organization: Clear, linear structure with obvious transitions
Tone: Conversational, direct communication
Background knowledge: Minimal assumptions about reader expertise
Writing style: Active voice, simple explanations
Example Publications:
Time Magazine
People Magazine
Reader’s Digest
USA Today
Entertainment Weekly
Sports Illustrated
Cosmopolitan
Popular Mechanics (basic articles)
Grade Level 10-12 (High School/Some College)
Characteristics:
Sentence structure: Medium-length sentences with some complexity (15-20 words)
Vocabulary: Mix of common and moderately advanced words
Concepts: Current events and issues explained with context
Organization: Clear structure with supporting details
Tone: Professional but accessible
Background knowledge: Basic cultural and educational literacy assumed
Writing style: Balance of explanation and analysis
Example Publications:
The New York Times (news sections)
The Wall Street Journal (general articles)
Newsweek
U.S. News & World Report
Rolling Stone
Wired (general tech articles)
National Geographic (accessible pieces)
Psychology Today
Grade Level 12-14 (College/University)
Characteristics:
Sentence structure: Complex sentences with multiple clauses (20-25 words)
Vocabulary: Advanced vocabulary, specialized terms used without definition
