How fraud happens under data-driven accountability

Getty Images/PointImages

By John Thompson

I enjoyed most of the submittals in the Fordham Institute’s Wonkathon, but my favorite was Max Eden’s, “Reformer, Heal Thyself. You’ve Ruined High School.” I don’t agree that all of high school has been ruined, but technocratic reformers did wreck my Oklahoma City high school, and they did so by ignoring the predictable ways that their data-driven accountability systems would produce huge amounts of unintended harm.

Eden wrote:

The notion that sitting a bureaucrat trained by the Broad Academy in a chair could fundamentally change the life trajectories of thousands of deeply disadvantaged students within just a couple of years is, to put it mildly, willful wishful thinking. On the other hand, the systems, expectations, and professional incentives provide means and motive to commit fraud. In the rare event that reporters ferret out the fraud, technocratic wonks provide alibis rather than accountability.

Our high school first received a Broad Academy–trained superintendent in 2007. Ninety percent of our students were low income, but his only education experience was as a technology director of a school district in which only a quarter of pupils were low income. He told my students that he planned to install television cameras in classrooms so he could sit in his office and supervise instruction. One thing he would monitor was whether each class was “on the same page” in teaching the same aligned and paced lessons.

On the eve of No Child Left Behind, our school was successfully reducing absenteeism, increasing graduation rates, and raising test scores. I must emphasize that we had no funding for those extra efforts. We voluntarily did the additional work which produced the gains.

When faced with NCLB’s impossible targets, however, the system’s priorities were switched to make accountability statistics look better than they were. In other words, we fabricated data. And had reformers listened to teachers and students, they would have known that such fraud would be inevitable.

In the first class period of the day, absenteeism usually exceeded 25 to 30 percent—if a teacher took roll when class started. But we’d take attendance at the end of the ninety-minute period. Students could hang out with friends, drop in for the last five minutes, and still earn credit for being in class. We did the same thing in the period after lunch. It didn’t take students long to catch on and walk the halls for eighty-five minutes before reporting to class. But our attendance rates went up.

Another easy way to produce miraculous gains in attendance was to enter absences in the computer using the notations “P” or a “C,” indicating that a principal or a counselor had had a conversation with the truant student—even when they hadn’t. This would drop the “U,” for “unexcused.” Throughout my twenty-year career, even before NCLB, this was the primary way dozens of absences were dropped from the computer every few weeks. But the law’s data-driven accountability made the practice much more common.

No Child Left Behind also led to the rise of “credit recovery” as a means to erasing absences disappear—and, worse, jacking up graduation rates. My students called it, “exercising your right-click finger.”

The learning culture of our high-challenge school had long been undermined by dozens of class-cutters strolling the halls during periods. After attending class for the full duration became optional, we often faced hundreds of these students, who contributed to an epidemic of violence in the corners of the building. But our annual attendance rates jumped from less than 80 percent to more than 90.

This pattern was repeated at another one of our district’s high schools, whose rates rose to unbelievable levels. After a federal investigation, its principal was forced to resign, but nothing happened to his colleagues who borrowed his methods. The next year, all of our neighborhood high schools claimed identical miracles. (The already-high-performing magnets showed no change.)

These fraudulent behaviors worsened significantly when schools fell hopelessly behind NCLB’s schedule for raising test scores. In 2005, a friend regaled me with the trick his principal had brought from Texas for legally excluding low test scores from official calculations. Even if students were pre-enrolled to attend the school, parents would be required to come to the building to fill out new paperwork. If parents did not visit the building, their children would be dropped from the rolls until they completed the process. The kids might only miss a day or so of class, but they would not be counted as "enrolled for the full year," and their test scores could be excluded. So our school adopted this strategy and sent such students home on the first day of school, thus enrolling them. This comprised one third of our kids, and the school was thrown into chaos.

Yet the biggest catastrophe began in 2006, when staff allotment became a data-driven process where numbers alone determined which teachers taught which students, and how many of them. There was no way to heed human judgments and override the formula when necessary. This caused major problems.

Many of these resource allotments were determined by students’ eligibility for free or reduced priced lunch. Due to the extreme proliferation of school choice, our run-of-the-mill inner-city school had lost our working-class kids and students from situational poverty. But the metric doesn’t distinguish between these low-income students and those from extreme generational poverty, who are much more likely to endure multiple traumas. So the system failed to estimate the resources we needed. I, for example, was assigned to teach 247 students. And most of the other teachers were assigned at least 210.

The resulting mayhem then spun completely out of control after the district took a data-driven approach to reducing suspensions. Fordham has done much to illuminate the harm done by this approach to school discipline, so I’ll just stress another byproduct of the hurried, unfunded process. Principals were held accountable for meeting NCLB’s student performance targets and reducing suspension rates, but they were not empowered to create safe and orderly climates. In my school, for example, rather than writing a disciplinary referral that would count against the our metrics, an assistant principal often instructed one of the policeman assigned to our building to write a $240 ticket for disturbing the peace. Our high school, which was down to 300 or so students, eventually had 185 arrests during a school year that lasted 173 days.

I could go on with examples of unintended consequences that foreshadowed the scandals cited by Max Eden and others. But my point is different. Had reformers spent time in our schools, we could have accurately predicted the result of data-driven accountability. And as we anticipated, the damage grew much worse after the Obama administration doubled down on punitive test-based policies.

By that time, however, I was at the end of my career, and the full brunt of reform was disproportionately endured by young and inexperienced teachers. To see how that turned out, check out recent teacher uprisings in West Virginia, Oklahoma, Arizona, Kentucky, and Colorado, often led by educators from districts micromanaged by Broad graduates, and read some of their reasons for striking.

John Thompson is an award-winning historian, award-winning teacher, a blogger, and the author of A Teacher’s Tale: Learning, Loving and Listening to Our Kids.

The views expressed herein represent the opinions of the author and not necessarily the Thomas B. Fordham Institute.