EdCircuit Staff 
By Jenny Grant Rankin, Ph.D.
I f your students scored 11% to 48% correct on a test, would you give them each an A+? Would you consider them proficient and trust them to tutor their peers on the test’s content? I’m guessing you wouldn’t be pleased with 11-48%. Most educators don’t find that performance to be sufficient.
Yet evidence suggests the inferences educators make when they view data are only 11% to 48% accurate. Educators use data to inform their decisions, and those decisions ultimately impact students. Thus most educators don’t find poor data use to be sufficient.
So we educators strive to improve our data use. The most common interventions we apply are:
- more (and improved) training for educators
- added staff to support educators (e.g., data coaches)
- stronger leadership in data use
Who is responsible for each of those added interventions? Educators are. Educators are the ones who fund those efforts, arrange those efforts, find time for those efforts, and participate in those efforts. It’s no wonder educators are exhausted.
Professional development (PD), support staff, and leadership to improve data use are beneficial and recommended. However, they still leave educators with no more than 48% accuracy when using data to inform decisions. Educators need to remember the fact that they are highly qualified, well-educated, and intelligent and consider whether there is another factor impacting their data use – a factor that is not their fault. Because there is.
If educators struggle with understanding the implications of something as simple as a basic bar chart, there is something wrong with the bar chart. For too long, the data systems and tools educators use have been throwing data at educators as if it is simple, and leaving it to the educators to figure out the often-deceptive data themselves. Responsible data system vendors are embracing the concept of offering over-the-counter data (OTCD), which involves following research-based standards to report education data in a way that is easy to understand and use, just as over-the-counter products embed usage guidance for easy use. Educators can advocate for these supports within their data systems:
OTCD Component 1: Label
Data reports should be properly labeled with titles that clearly indicate their content. Reports should also feature footers or other annotations that succinctly tell educators what they need to know to correctly analyze that particular report’s data. In a study of 211 educators of varied backgrounds and roles, educators’ data analyses were rendered 307% more accurate when such a footer was present on a data report. Accuracy rose to 336% when respondents specifically indicated having used the footer.
OTCD Component 2: Supplemental Documentation
Report-specific supplemental documentation should accompany every data report in a data system (such as through a within-report link, or handed to educators in an in-service). These include:
- a single-page reference sheet that helps the educator easily understand the report and analyze its data
- a 2-3-page reference guide that helps the educator easily use the report and its data
In the 211-educator study, educators’ data analyses were rendered 205% more accurate when such a reference sheet was present and 273% more accurate when a guide was present. Accuracy rose to 300% when respondents specifically indicated having used the sheet, and rose to 436% when respondents specifically indicated having used the guide. Data system providers can use free templates to create their own supplemental documentation.
OTCD Component 3: Help System
Just as 12 million people per month use WebMD, so do educators crave a help system to guide them when they use data while alone. A well-crafted help system within the data system should provide tech lessons (i.e., how to use the product) and also data analysis lessons. A user-friendly help system with short, targeted lessons has been found to cause users to need 40% less training time and to successfully complete 50% more tasks.
OTCD Component 4: Package/Display
A bar chart should not simply look easy to read. In fact, if it looks easy to read but renders misunderstanding, then it is a bad bar chart. Each data report’s displays should be selected and designed to ensure its particular dataset’s implications are understood. There are many ways visual display can be manipulated to encourage understanding.
OTCD Component 5: Content
Educators need a comprehensive offering of data reports that help answer varied questions. Those reports must also place needed, current data at educators’ fingertips so they don’t have to hunt in various places for straightforward answers.
How This All Happens
Regardless of which interventions are applied in school districts to improve data use, educators have to advocate for their data system providers to follow research-based best practices when reporting data to educators. OTCD Standards summarize over 300 studies and other expert sources to indicate how best data systems can provide the OTCD components described above. With all the responsibilities educators have to juggle, how wonderful that others can help.
The opinions expressed here are solely those of Jenny Grant Rankin.
Dr. Jenny Grant Rankin is a former award-winning teacher, school site administrator, school district administrator, and chief education & research officer (CERO) at Illuminate Education. Her Ph.D. in Education features a specialization in School Improvement Leadership, and her research is widely published. Dr. Rankin’s honors include Teacher of the Year, the #EduWin Award, EdTech’s Must-Read Higher Education Technology Blogs List, finalist for EdTech Digest’s Trendsetter Award, and having the U.S. flag flown over the United States Capitol in recognition of Dr. Rankin’s (then Mrs. Pasqualino’s) dedication to her students. Dr. Rankin regularly presents her research at conferences such as for the U.S. Department of Education’s Institute of Education Sciences (IES) National Center for Education Statistics (NCES) STATS-DC Conference. Her website (www.overthecounterdata.com) contains free resources and updates related to making data easy for educators to use. Follow her on Twitter: @OTCData
Chaz Austin