Greater transparency can mitigate the gaming of accountability measures: An example from Georgia

Getty Images/Arndt_Vladimir

Jeremy Noonan

Through their Strategic Waiver School System (SWSS) initiative, Georgia's Department of Education (GaDoE) is leading the way in empowering local districts to own responsibility for educational reform. The terms of the five-year SWSS contract are that the system receives a waiver from the GaDoE for a host of state regulations, and in exchange for this flexibility, the system submits to accountability for increased student performance. 

This strategy is laudable for its intentions, namely to facilitate an attitude change among local leaders from a compliance mindset to an achievement mindset. By removing regulatory strictures, it aims to empower those leaders to innovate to raise student achievement in a way that is appropriate for local contexts. The strategy shows how Georgia takes seriously the principle of local control.

With less “top-down” accountability for compliance with regulations, “local efforts [to ensure that districts make proper educational decisions] are now more important than ever,” according to Louis Erste, the associate state superintendent who oversees SWSS contract implementation. But is such “bottom-up” local accountability strong enough to fill the void?

For these local efforts to be effective, stakeholders need a high degree of transparency from local systems. Yet this may be hard to get. For Georgia’s experiment in local control to succeed, the GADoE should make it easier for stakeholders to get more information concerning both the use and consequences of the waivers.

SWSS schools should have to disclose publicly which waived regulation they are actually not following in a given school year. The waived regulations may include some rules that seem pretty vital to student welfare, such as teacher qualifications and class size limits. These do not necessarily entail that schools will hire teachers without college degrees and increase class sizes to sixty; it only means that they could do so. But local stakeholders are left in the dark concerning which deviations from the regulations actually occur. If my child has a math teacher who is not certified and who has a bachelor’s degree in history, or if his gifted class balloons from fifteen to thirty students, I would want to know so that I can keep a closer eye on how the class is going.

Moreover, without public disclosure of waived rules, adverse effects on student achievement might be concealed from stakeholders. This could lead to situations in which a school’s low performance should trigger corrective accountability actions from the GaDoE but fails to do so. To understand how, let’s look at some achievement criteria in a sample contract.

Student achievement is measured by Georgia's rating system, the College and Career Readiness Performance Index (CCRPI), which is a score out of one hundred that is determined by a complex formula, including a base of eighteen “achievement indicators” for high schools. Per the contract, systems typically have at least a couple of paths for meeting CCRPI goals: one, increase their CCRPI score each year by 3 percent of the gap between their baseline, set in the first year of the contract term, and the top score of one hundred; or two, exceed a predicted CCRPI score that is based on the school's racial and economic demographics (a feat known as "Beating the Odds").

The flexibility afforded by SWSS can be misused by local leaders in a way that enables schools to reach their CCRPI targets while actually harming student achievement in certain areas. In other words, achievement may appear to rise adequately overall, even though the achievement of those affected by the waived rules may decline because those effects don’t show up in the CCRPI.

There are a number of ways this could happen, but I will focus on two. Four- and five-year graduation rates comprise 15 percent of the CCRPI score. With graduation rate fraud becoming a national pandemic, stakeholders should be concerned about schools inflating their graduation rates by taking such shortcuts as illegitimate online credit recovery (OCR) courses, which has been a well documented problem in Georgia. With less checks from the state for employing legitimate means, schools leaders face greater temptation to employ such unsavory methods to increase rates.

My own district in Douglas County, Georgia, has recently enjoyed graduation rates that are 8–10 percent higher than the state rate—in spite of much lower student achievement—in part because of its own misuse of OCR courses. It is impossible to say what the rate would be without these compromises, relative to state norms, but our schools’ CCRPI scores seem to have risen at least 1–1.5 points out of one hundred from this inflation.

Another way schools can boost CCRPI is by enrolling large numbers of unqualified students in Advanced Placement courses, while inflating their grades to ensure that they all receive credit. One of the eighteen CCRPI high school “achievement” indicators is the percent of graduates earning school credit for an AP, International Baccalaureate, or dual-enrollment course (the inclusion of which was a major point of contention between Georgia’s governor and superintendent). A school that decides to enroll all its seniors in a single AP course, regardless of students' readiness—a move that recently triggered an audit at one Georgia school—could earn as much as a two-point CCRPI boost.

To put these CCRPI increases in context, just a single-point increase is not trivial: With a baseline score of seventy, a school only has to raise its score 0.9 points in a year to satisfy the terms of the contract. And whether a school is classified as “beating the odds” could well be decided within these small margins. In 2017, for example, all five of the high schools in my district “beat the odds,” with three doing so by fewer than 1.9 points.

These high schools have managed to meet their targets using, in part, the means described above. In addition to inflated graduation rates, AP enrollment has increased manifold, with AP exam scores plummeting, in spite of a nearly 100 percent AP course passing rate. (Whether students pass the AP exam has no bearing on whether they earn credit for the course.) One high school that only beat the odds by about 0.2 points got 1.2 of the 1.9 AP-course credit points, even though just 3 percent of its AP students actually passed an exam!

The waiver agreement incentivizes reckless enrollment growth by eliminating limits on class sizes attached to additional state funding for AP courses. One high school in our district, for instance, had thirty-five students enrolled in a single AP Statistics course (twenty-one is the max size under the regulations). Shockingly, all thirty-five scored a one out of five on the AP exam (for which a three is a passing score). But this horrid result did not show up in the CCRPI, which was instead increased by the mere participation of students who earned course credit.

Parents do not know about the 100 percent failure rates on AP exams because systems do not have to publish such results. If a given AP course has a larger class size than previously required, or if a math course is taught by a non-certified instructor, those parents should be notified, and the exam scores made public, in comparison to prior years’ scores, so that the public can assess the direct effects of the flexibility afforded by SWSS. By requiring schools to disclose academic data for students directly affected by waiver decisions, the GaDoE would mitigate the likelihood of such abuses.

Strong local accountability is essential for Georgia’s bold experiment in local control to succeed in truly improving student achievement. The GaDoE would do well to empower it by requiring greater transparency from its strategic waiver school districts, including data on how their decisions affect student achievement.

Jeremy Noonan is a father of four school-age children and a certified science instructor in Georgia, who has taught for ten years in public school, private school, and home school cooperatives. He also runs Citizens for Excellence in Public Schools, a local education advocacy group.

The views expressed herein represent the opinions of the author and not necessarily the Thomas B. Fordham Institute.