The problem with graduation rate statistics

Ohio’s largest online school, the Electronic Classroom of Tomorrow (ECOT), has recently caught flack for its low graduation rate. A New York Times article, for example, averred, “Publicly funded online schools like ECOT have become the new dropout factories.” It is true that a mere 39 percent of the ECOT’s class of 2014 graduated in four years, meaning that thousands of pupils failed to reach the high school finish line on time. Meanwhile, a recent GradNation report called out the low graduation rates of some alternative, charter, and virtual schools (for a deeper dive into the charter school rates, see Susan Aud Pendergrass’s excellent piece on Flypaper). For some, these statistics are proof positive of educational failure.

We are in no way defending “dropout factories” of any stripe. It’s well known that Ohio’s virtual schools (like those in almost every other state) have struggled mightily to demonstrate an impact on student growth, and we’ve made no secret of our own misgivings about ECOT and many of its peers. But when it comes to graduation rates, how much of the blame belongs to the schools themselves? Is it possible that the way these numbers are calculated yields results that are, one might say, falsely deflated?  

Like other states—and as required under federal regulations—Ohio uses something called the adjusted cohort graduation rate (ACGR). This method looks at a school’s ninth-grade cohort and tracks how many of those students graduate within four years (or five, if using a five-year statistic). Sounds straightforward, right? But because pupils transfer into and out of schools between the start of ninth grade and the end of twelfth grade, the cohort composition can change over time. Hence, to calculate the baseline number of students for which each Ohio school or district is held accountable, incoming transfers are added into the cohort while outgoing transfers are subtracted—provided they’ve enrolled in another school (i.e., not dropped out).

If there were no transfers, it would be reasonable to hold high schools fully accountable for the graduation of pupils in their ninth-grade cohorts. But when a school’s pupils are highly mobile or transitory, accountability for their graduation gets more complicated. These cohort adjustments—while not unreasonable at first blush—could unfairly cast blame on a school when its late-transferring (in-bound) students fail to graduate on time. (The reverse is true as well—we could mis-assign credit for graduation even though the school had little to do with it.)

Consider a hypothetical student who spends her first three years of high school attending one of the Columbus City Schools before spending her fourth enrolled in an online school. Under the ACGR method, the school of record at the end of her senior year would be held accountable; thus, if she doesn’t graduate on time, the online school would lose credit. But what if, at the time of transfer, she was far behind academically—credit deficient and not on track for passing Ohio’s graduation exams? Should her last school of attendance shoulder the entire blame when just one-quarter of her high school career was spent in it? Of course not, but that’s the current accounting practice.

This scenario is quite likely in schools that serve lots of mobile students, including those at greatest risk of dropping out. And if, as it claims, ECOT (and schools like it) enrolls many transfer students who are far behind academically, then its low graduation rate as calculated via ACGR might not be an accurate gauge. How many students did it educate throughout their entire high school careers? How much of its final “adjusted cohort” transferred in much later? We don’t know, and that’s what makes accountability based on graduation rates a topic of legitimate contention.

Consideration ought to be given to how schools should be held accountable for those who arrive (or depart) during high school. (This would be good for all public high schools in the state, not just ECOT and its ilk.) One approach is to apportion responsibility according to the duration of a student’s enrollment at a particular high school. In the example above, the Columbus high school would be held accountable for 75 percent of the non-graduation of the student in question, while the online school would be held 25 percent accountable. Relative to the ACGR method, the district’s high school would face a penalty for not ensuring that the student remained on a sure graduation track from ninth grade through eleventh grade, while the last school of attendance would gain partial reprieve because it only educated the student for one-quarter of her high school career. Another alternative would assign full responsibility to the school where a student is enrolled for the greatest amount of time from ninth grade through twelfth grade.[1] Compared to current practice, either option would more fairly apportion to schools the credit or blame for students’ graduation or lack thereof.

A more refined approach to graduation accountability might also better align school incentives with student interests. Current practice tempts schools to nudge low-achieving students toward an alternative school in order to avoid accountability for non-graduation and artificially inflate the “sending” school’s own graduation rate. (Such alternatives could include dropout-recovery charter schools—sometimes sponsored by a district—or online schools.) We don’t know how often this occurs.[2] But if we assign schools partial (if not full) responsibility for non-graduation, they won’t be able to duck accountability by counseling out struggling pupils who might actually be better served in their current schools.

High school graduation rates will continue to serve as workhorse accountability measures under the revised federal education law (ESSA). The law requires a graduation rate “indicator” on school report cards, and states must intervene in schools with low graduation rates. To be sure, federal laws and regulations might eventually forbid a different approach to calculating graduation rates. But Ohio policy makers should at least press federal officials for the flexibility to implement an alternative graduation rate measure. A proper accounting method would better ensure that schools are fairly assigned responsibility for students’ graduation—and align incentives in a way that could help more students earn their diplomas.




[1] ESSA includes a provision that requires, in certain circumstances, a school district (or charter school) to assign accountability for a student’s non-graduation to either the school in which she spent the greatest proportion of time from grade 9–12 or to the school in which she was most recently enrolled.

[2] An example of a school withdrawing students to enroll in district-sponsored “digital academy” can be found in the auditor of state’s “Interim Report on Student Attendance Data and Accountability System,” (October 4, 2012), page 21.

 

 
 
Aaron Churchill
Aaron Churchill is the Ohio Research Director of the Thomas B. Fordham Institute.