This is an Eval Central archive copy, find the original at freshspectrum.com.
Are you a little hesitant about all this AI stuff? Me too. But these new AI tools can actually be pretty dang useful and easy to use.
For the most part I’m a new tech skeptic. Before I add another tool to my personal toolkit, I want it to be beyond that early adoption phase. So when everyone started getting excited about AI writing tools like ChatGPT and AI design tools like DALL-E I didn’t immediately hop on the bandwagon.
But after a bit of play, it’s hard not to be impressed by ChatGPT. Not because of its future potential but because of its immediate practical usefulness. This post is going to be about one of those practical uses.
How can we use AI tools to make long “academic” reports and guides a little more readable?
“This is too easy to understand” -Said no one ever.
Before we get into the challenge, let’s start with an underlying assumption. There is no value in making anything harder to read.
Or, in other words, just because many in your audience have advanced post graduate degrees does not mean you should be writing for a post graduate reading level.
The Challenge – Starting with a Guide
In order to find a report or guide for this challenge I just picked a US government agency at random. I then scrolled down the publications list and picked the first large report-like thing I could find. This guide from the Department of Education’s National Center on Education Statistics met that criteria.
My challenge is to pick a section of the report and then use free AI tools to:
- assess the readability
- and rewrite the section to make it shorter and more readable.
Picking a Section – Why Education Indicators Matter
So the following text comes from page 2 (part of the guide’s introduction). This is a space where you want the guide to be easily readable and somewhat free from Jargon. I don’t think that’s the case here. This section can definitely be shorter and more readable.
Why Education Indicators Matter
Education indicators are critical to multiple areas of agency activities. They allow agencies to monitor student achievement and goals, identify gaps in student performance for potentially vulnerable populations, and evaluate the effectiveness of new policies and practices. Indicators inform important analyses, such as benchmarking, comparing groups, and assessing the benefits of funding allocations. They can offer greater transparency to stakeholders by providing easily accessible information and offering a means of clearer communication.
Education agencies use indicators for varied strategic purposes. For example, districts trying to fill course vacancies need workforce and staff data about teachers, such as certifications or professional development, to help ensure effective placement. By providing shared metrics across the education system, indicators also help increase stakeholder understanding and lay the foundation for tools such as crosswalks to state report card data.
Because a single assessment, such as a final exam score, rarely yields a complete picture of a student’s learning, indicators that use multiple measures and multiple types of assessment help teachers and schools realize a more complete picture of an individual student’s needs. This picture helps educators better allocate limited intensive intervention resources. Indicators also often are used to allocate resources and help direct (or redirect) funds for students and staff.
At the classroom level, indicators can help teachers personalize instruction for groups and individuals. By collecting a range of data, analyzing them, and giving them to classroom teachers, agencies support their educators as they address specific student needs and make data-informed decisions to improve learning.
Many advancements in education have impacted indicators, including changes to data systems (such as improved longitudinal data systems), federal data collections and accountability systems, legislation, mandatory public reporting, privacy protections and data security. In recent years, some agencies have considered adjusting their approaches to standard practices like attendance and grading, which will influence the use of indicators. Agencies also may consider making temporary changes in response to major events (such as those related to the coronavirus disease [COVID-19] pandemic) permanent. Formalizing these types of changes to indicators related to grading standards, graduation requirements, or disciplinary methods will affect data comparability. New indicators also have emerged as key sources of information for decision-making. For example, learning modality data that track whether students are attending school in person, virtually, or in a hybrid model have become increasingly important as agencies work to understand student learning conditions and outcomes.
As the design and use of education indicators shift with agencies’ changing needs, advances in technology, and outside influences, many data experts have noted the need for a wider range of measures, including those that would address issues like social-emotional learning, engagement, expectations of teachers, rigor of assignments, alignment with grade level, and educator practices.
Assessing Readability with the Hemingway App
So my plan is to use AI tools to both assess the readability and rewrite the text.
The tool I’m going to use to assess readability is the Hemingway App. It’s simple to use (just copy paste) and doesn’t even require you register for an account. If you like it, you can also download an inexpensive desktop app version that can be used without an internet connection.
I ran the original text through for an initial assessment.
According to the Hemingway app the readability for this section is Post-Graduate. There are 457 words and 15 of the 19 sentences are very hard to read.
Rewriting with ChatGPT
The next step is to use ChatGPT to shorten the section and then increase the readability.
At the time this blog post was written ChatGPT is available as a Free Research Preview. It is also currently on its January 9 release. At the moment you need to register for an account but the tool is otherwise free to use.
I asked chat GPT to “Rewrite the following in 100 words” and then I pasted in the text I shared above. Here is what the tool gave me (the only thing I changed was to add a few paragraph breaks).
Education indicators are vital for monitoring student achievement and identifying gaps in performance for vulnerable populations. They also evaluate the effectiveness of new policies and practices and inform important analyses such as benchmarking, comparison of groups and assessing the benefits of funding allocations. They offer greater transparency to stakeholders by providing easily accessible information and clearer communication.
Indicators are used for strategic purposes such as filling course vacancies and allocating resources. They can also help teachers personalize instruction and make data-informed decisions to improve learning. Many advancements in education have impacted indicators such as changes to data systems, federal data collections, legislation, and privacy protections.
In recent years, some agencies have considered adjusting their approaches to standard practices like attendance and grading which will influence the use of indicators. Some new indicators have emerged as key sources of information for decision-making such as learning modality data. As the design and use of education indicators shift with agencies’ changing needs, advances in technology and outside influences, the need for a wider range of measures has become increasingly important.
Now that I have new text, let’s run it through the Hemingway App again.
Okay, so the text is shorter, it moved from 457 words to 177. But still not an easy read. Hemingway labeled it as post-graduate readability with 9 of 9 sentences marked as very hard to read.
Iteratively Changing the ChatGPT Prompts
So now I’m going to iterate using only the two AI tools back and forth. I’ll rewrite with ChatGPT and assess with Hemingway.
Iteration 2 – Rewrite the following in 100 words for a high school reading level
ChatGPT reply
Education indicators are important tools that help schools and other education agencies keep track of student progress and success. They allow agencies to monitor student achievement and goals, identify areas where students may be struggling, and evaluate the effectiveness of new policies and practices.
Indicators also help agencies make important decisions about how to allocate resources and support teachers. They can provide more transparency and clear communication to stakeholders. Indicators are used for various strategic purposes, such as filling course vacancies or allocating resources. They can also help teachers personalize instruction and make data-informed decisions to improve learning.
New advancements in education have impacted indicators, including changes to data systems, legislation, and privacy protections. Some agencies have considered adjusting their approach to standard practices like attendance and grading, which will influence the use of indicators. Indicators are also emerging as key sources of information for decision-making, such as learning modality data that track how students are attending school.
As the design and use of education indicators continue to evolve, experts have noted the need for a wider range of measures.
Hemingway App assessment
Readability went from Post-Graduate to Grade 15. There are still a lot of very hard to read sentences according to Hemingway.
Iteration 3 – Rewrite the following in 100 words for a middle school reading level.
ChatGPT reply
Education indicators are tools that schools use to track student progress and success. They help schools know how well students are doing, identify areas where students may need extra help, and evaluate new policies and practices.
Indicators also help schools make decisions about how to use resources to support students and teachers. They can provide more information to people who are interested in the school and make communication clearer.
Schools use indicators for different reasons, like filling open spots in classes or allocating resources. They can also help teachers personalize instruction and make decisions based on data.
Advances in education have affected indicators, like changes to data systems, laws, and privacy rules. Some schools have considered changing how they track things like attendance and grading, which will affect how they use indicators. Indicators can also be used to track new things, like how students are attending school.
Experts say that more indicators are needed to track different aspects of student and teacher performance.
Hemingway App assessment
By asking for a middle school reading level ChatGPT wrote something that the Hemingway App labels as a Grade 12 reading level. This version is down to 163 words with fewer very hard to read sentences.
Iteration 4 – Rewrite this for 4th graders.
ChatGPT reply
Indicators are like signs that schools use to check how well students are doing. They help schools see where students might need extra help and how to make new rules that work well. Indicators also help schools decide how to use resources to help students and teachers. They give more information to people who want to know about the school and make it easier to talk about what’s happening.
Schools use indicators for different reasons, like making sure there are enough teachers for all the classes or deciding where to put resources. They can also help teachers teach in a way that works best for each student and make decisions based on information.
Sometimes, new ways of doing things in school can change the way indicators are used. For example, some schools are thinking about changing how they track attendance or grades, which will affect how they use indicators. Indicators can also be used to track new things, like how students are going to school. Some people say we need more indicators to check different things about students and teachers.
Hemingway App assessment
By asking ChatGPT for 4th grade, we made it to grade 10. Now there are only 2 of 10 sentences that are very hard to read.
3 Final thoughts on this approach.
1. Tools like the Hemingway App give us a way to assess our writing “externally.”
One of the challenges we face making better reports is that the reports you are trying to improve were written by human beings. And for many of us, it’s hard to tell our fellow human beings that their work is way too long, too boring, or too jargon filled. So if that’s you, you can just let the tool tell them their stuff is hard to read.
2. ChatGPT gives you a good first revision pass (or second, or third).
It can be really tough for a person to take something that’s 450 words and bring it down to 180 words. But ChatGPT isn’t going to grumble about it. Using the tool conversationally to help you wordsmith and simplify can really take a lot of the effort out of revisions.
3. From Human to AI to Human again.
I think the biggest opportunity exists in folding AI into your overall writing process. It’s not a replacement. The stuff that ChatGPT wrote (and Hemingway assessed) could definitely be improved with a little human intervention from a good editor. But the overall amount of work involved gets cut down significantly.
What are your thoughts?
Is this something you would try? Why or why not? Let me know in the comments.