Making Digital Education Tools Pay Off

In 2017, Diana Bidulescu of Houston ISD described how a large district tried to turn a flood of classroom data into usable information. The article appears below as published. After it, a new section looks at what research since then says about data literacy for teachers and how schools can build it.

Moving from Data Rich, Information Poor to Big Data, Big Gains

Over the past few years, K-12 classrooms have been inundated with digital tools that make learning fun, and capture various data.

Teachers rejoiced at the variety of fun and engaging free digital tools that they could use with their students, the wonderful interaction the tools brought and the great impact on classroom instruction that they had. They were able to see student responses and intervene promptly to correct misconceptions, to give students homework and track the progress through the tasks, they were even able to tag their test questions to learning standards to determine what understanding of the concept that the student had. Schools and district administrators rejoiced at the teacher enthusiasm and student excitement with the digital transition and looked forward to seeing improved learning outcomes.

Fast forward one year, and classrooms burst at the seams with a wealth of information about students. The teachers and administrators start realizing that some tools, while great at engagement, provide little else to help student learning, while others have information that they would like included with their regular, learning standards-based batch. So much data, so little time to put it all together, and such limited means to add it up and compare it! Some say it’s a square peg in a round hole, some call it apples and oranges, some say it’s too much, while others say it’s not enough. It seems we’ve struck gold, but don’t know how to dig it out! Welcome to the data rich and information poor era!

This is when we start realizing that we have access to an enormous and varied pool of information, but have little means to make it work together to have an accurate and aligned image of the student progress and truly personalize learning.

The K-12 Big Data Dilemma

We can call this the K-12 Big Data Dilemma. As we well know, the Big Data is present in all areas of our life, caused by our adoption of digital tools. To make sense of the Big Data, companies have algorithms that mine the data and make connections. They then produce a report of the findings, with correlations and possible recommendations. In K-12 we call that a really good teacher. However, with the amount and variety of data we now have, and adding to that the experience the teachers have aggregating so many data points, we soon see that we, too, need an algorithm, as well as a plan. The great awakening hence begins with a big question: how do we help our teachers use this Big Data? How can a teacher use a wonderful free or paid digital tool, but get relevant data that can be added to the mix to create a full and comprehensive view of the student’s performance?

How do we move from “Data Rich, Information Poor” to “Big Data, Big Gains”?

Step One: Digital Access and Interoperable Systems

In a future not far away, we will all be able to replicate Alt School, a place where technology and good teaching come together to provide students with personalized education based on the data. But to get there tomorrow, we have to build realistic and systemic supports now. The first step is to ensure student digital access, and that all the online learning systems, which collect vast amounts of data as students progress through the game, test, or learning activity are able to import/export data in standard formats that allow for the data to be aggregated accurately, and simplify the cross-walking work that districts and schools currently have to do to get relevancy and accuracy.

In Houston ISD (HISD), we started the digital transition four years ago, and it included 1-1 programs, the implementation of an LMS, a digital content and curriculum initiative that made all high school books available exclusively online, and the circle was completed with a transition from paper and pencil to eAssessments (online testing) on the state and formative assessment platforms. The HISD eAssessment program complements the integration of digital curriculum, which advances the instructional goals of personalized learning, data ownership and improved mastery; this is facilitated by a formative assessment platform, OnTrack, powered by School City, that satisfies our authoring and reporting needs, aside from being nimble with online assessments as it is with paper. All systems and digital tools purchased by HISD are conformant with IMS Global interoperability standards, and facilitate a cohesive data flow.

Step Two: Building Digital Assessment Literacy

The second step in moving the district to big gains from big data is to build Digital Assessment Literacy skills in teachers and staff. With the new ESSA allowing districts more freedom in the assessment selection and implementation of educator-controlled, local performance assessments, the requirement is also that the assessment system must be valid and reliable, aligned with the state academic standards, provide coherent and timely information about the student’s mastery level, and be provided in multiple languages for English Learners.

Because so much of the student academic data comes from teacher-student interaction in class, and this information is “live,” it is important to learn how to capture it in a meaningful format, which enables for it to be aggregated. It was therefore important to train teachers on how to build digital assessments that are reliable and standards- aligned. In HISD the schools are becoming accustomed to teaching and learning on mobile devices, and online formative assessments are increasing at a vertiginous rate. To support the digital transformation, and prepare for next gen assessment, Digital Assessment Literacy is focused both on the utilization of assessments and data, and on the assessment authoring aspect. Data relevance, heat maps, and Data Driven Instruction (DDI) are elements that campus and district staff are continuously training on and putting in practice when creating assessments, as well as when making data driven learning decisions. To support ease of test authoring for teachers we purchased the new formative assessment platform, School City. School City enables the test authors at all levels to align the test items to learning standards, and embed accessibility features, which mimic the Texas state standardized assessment accessibility features to support all student learning needs. It also provides a reporting feature that is easy to read and helps teachers support students’ learning with personalized learning resources. Maybe the most important and valued feature is the student facing data, which is released at the end of the test where students see what questions they missed and how they are connected to their learning, before they even see the grade.

Three Elements for a Successful Transition to Data

Big or small, campus or district, for anyone trying to move from Data Rich to Big Gains, here are three elements that must be present for successful transition to Data.

Digital Assessment Literacy – Maybe one of the most needed, assessment literacy helps teachers understand how to construct their tests for relevance, validity and maximum reach without taxing students with long or frequent assessments. Guiding questions can be: what test to use when, how do we code test items and is one question relevant to determine student mastery?

Data Literacy – Without knowledge of data, teachers are unable to guide their students efficiently through their learning, Moreover, they would not understand how to build activities, assessments and projects that yield the much-needed “live” academic information. Guiding questions can be: what data is needed in my subject, should reading level matter in Math, how much weight should I give my test, is grade only a number, do I have apples and oranges and what is next?

Logistics and flexibility – The greatest challenge in achieving big gains from big data is training teachers and administrators how to efficiently read, interpret and implement the data. Like all good PD, all trainings have to be applicable, in a low stakes environment and with the focus being on mastering the Digital Assessment Literacy and Data Literacy processes. Guiding questions can be: is PD in PJs helping teacher plan data-driven lessons, what tools do teachers use to gather data from student assessment, how to read a heatmap and what are the next steps?

As we build upon Big Data, Big Gains, it is important to remember that our teachers are the ones who must understand and find the tools and processes easy to use that help our students be successful. The secret to personalization is in helping all stakeholders make sense of various data, which paint a complete picture of the student. The only way to do so without the great algorithms, is to build capacity in our teachers so they create relevant data yielding lessons and assessments. And just like the Gold Rush, sharing out loud nuggets of data inspires others to equip themselves and go digging for it!

1. ADT Magazine – App Economy Report: ‘Big Data Will Lead the Way’
2. AASA – Using Data to Improve Schools
3. Smart Data Collective – Challenges of Big Data in Education

Data literacy for teachers in 2026: what the research says about training that works

Data literacy for teachers is the ability to find the right information about students, read it correctly, and use it to decide what to teach next. As Diana Bidulescu explains above, a district can own plenty of digital tools and still be short of information a teacher can act on. Bidulescu’s answer in Houston ISD had three parts: interoperable systems, digital assessment literacy and data literacy, and training that is practical and low in stakes. A recent review of the research gives a useful way to test that answer against what is known about teacher training.

What a 2025 review of the training research found

In March 2025, Fabián Sandoval-Ríos, Carla Gajardo-Poblete and Juan Antonio López-Núñez published a systematic review in Frontiers in Education of training programs meant to build teachers’ data literacy. The review covered 16 studies published between 2014 and 2023, involving 636 pre-service teachers, 1,712 in-service teachers and 95 schools, in the United States (8 studies), Germany (5) and the Netherlands (3).

The programs with the strongest effects shared four features: they taught a structured way of making decisions from data, used real or realistic data, had teachers work together, and followed up over a long period. Confidence was the most consistently positive outcome, results on knowledge were mixed, and few significant changes appeared in attitudes toward data. The authors also noted that many studies put data analysis ahead of instructional application, and that some studies reported knowledge gains without evidence that they led to better decisions in the classroom. Only two of the 16 studies assessed instructional decisions directly.

The review has limits that a school leader should keep in view. The studies used 16 different measuring instruments, only about half reported effect sizes, few followed teachers over time, and most scored 3 or 4 out of 5 on a quality appraisal tool. The evidence base is small, and only about a quarter of the studies had a control group. The findings point in a consistent direction, but they do not prove that any one program will work in a given district.

How the review lines up with Bidulescu’s three elements

The first element Bidulescu names, digital assessment literacy, concerns how teachers build and choose tests. Bidulescu’s guiding questions, such as which test to use when and whether one question is enough to judge mastery, ask teachers to understand the instrument before they read its results. That matters because a report is only as sound as the items behind it. A teacher who knows an interim test has two questions on a standard will treat a low score on that standard as a prompt to look further, not as a verdict.

The second element, data literacy itself, is where the review adds most. Its finding that training should be organized around a decision, rather than around statistics, fits Bidulescu’s guiding questions about what data a subject needs and what to do next. The third element, logistics and flexibility, matches the review’s emphasis on authentic data and follow-up: training works better when teachers practice on their own students’ results and return to the work over months.

What school and district leaders can decide

Leaders can start with one recurring decision rather than a broad data initiative. A common example is what a grade-level team does within a week of an interim assessment: which students are regrouped, which skill is retaught, and how the team will know whether the reteaching worked. Building training around that cycle gives teachers real data to practice on and a clear purpose for reading it.

Time should be set aside in the schedule for teams to meet, because the review found collaboration among the features of stronger programs. Follow-up matters as much as the first session, so a plan that includes check-ins across a semester is better than a single workshop. Leaders should also decide in advance what they will count as success. Teacher confidence is easy to measure and was the most consistently positive outcome in the review, but the more useful question is whether teaching decisions changed.

Finally, tools matter less than the habits around them. Bidulescu describes the platforms Houston ISD chose, and a district with different systems can still apply the same ideas: data that can be combined across tools, tests that match the standards, and reports a teacher can read quickly. The article describes the district’s approach and does not report student results, so it is best read as a description of a plan rather than as proof that the plan worked.

Where this connects to other edCircuit guides

Readers working on how assessments are designed and used across a school will find more in the edCircuit assessment guide. Leaders planning the training itself can turn to the professional development guide.

More on Professional Development