Standards, Testing & Accountability

Previous research has found that oversubscribed urban charter schools produce large academic gains for their students. But are these results related to test score inflation (defined by one assessment expert as increases in scores that do not signal a commensurate increase in proficiency in the domain of interest)? In other words, do these schools merely figure out how to prepare their students to do well on the high-stakes exam, or are they contributing to real learning writ large?

To explore this question, a recent study examines state testing data from 2006 to 2011 at nine Boston middle school charters with lottery-based admissions. By exploiting the random nature of the lottery system, prior studies have found that these schools produce substantial learning gains on the Massachusetts Comprehensive Assessment System (MCAS).

To carry out the analysis, author Sarah Cohodes breaks down the learning gains by the various components of the state assessment—akin to how one might disaggregate overall gains by student subgroup. A math assessment might contain several different testing domains (e.g., geometry versus statistics), with some topics being tested more frequently than others. Cohodes’s hypothesis is as follows: If the gains are attributable to score inflation, we might expect to see stronger results on...

This new research report from Educational Testing Services is a solid contribution to the evidence base—rather than the opinion base—about the so-called “opt-out” movement. Author Randy E. Bennett finds that parents’ refusal to let their children sit for standardized tests is “a complicated, politically charged issue made more so by its social class and racial/ethnic associations. It is also an issue that appears to be as much about test use as about tests themselves.”

Opt-out true believers will likely dismiss out of hand anything coming from a testing outfit. but they ought to take a long look. The report does a good job synthesizing data from both the national and state departments of education, published surveys, and other sources to put between two covers exactly what is known—and can be sensibly divined—about who is opting out and why. “Parents who opt their children out appear to represent a distinct subpopulation,” the report notes. In New York, for example, “opt-outs were more likely to be white and not to have achieved proficiency on the previous year’s state examinations.” Test refusers are also less likely to be poor or to attend a school district serving large numbers of low-income families or ELL students. None...

A sixth grader in Mountain Brook, Alabama, can be considered one of the luckiest in the country, enrolled in a district where he and his classmates read and do math three grade levels above the average American student. But a child of similar age in Birmingham, just five miles north on Route 280, would be in considerably worse shape; there, kids perform 1.8 grade levels below average. So how could a ten-minute drive transport students to a different educational galaxy? Well, look at some numbers compiled by a team of Stanford researchers: Mountain Brook is 98 percent white, with a median household income of $170,000. Birmingham is 96 percent black, with a median household income of $30,000. Sometimes the figures speak for themselves.

John Bel Edwards, the recently elected Democratic governor of Louisiana, has had an eventful few months. After being inaugurated in January, he’s wrangled with state lawmakers over their leadership selection process and hustled to patch a huge crater in the budget. But his education agenda, largely aimed at curbing the growth of the state’s charter sector and cutting funding for voucher students, has run aground over the last few weeks. After the state’s newly...

Dave Yost

I am a conflicted man.

Professionally, I lead Ohio’s auditing staff, a team of financial experts whose job it is to verify that tax dollars are being properly spent and to root out any misuse or theft of public money. That includes charter school spending.

Yet personally, I’m a strong proponent of the charter school movement. I believe in the lifetime benefits of school choice and affording all parents the ability to choose the school that will best serve their children.

My friends sometimes question how I can be so tough on charters when I personally support them. The answer, I tell them, is simple: We don’t play favorites. We can’t. We shouldn’t. Doing so would erode the public’s trust in our office, which we must faithfully and ardently protect. To ignore the misdeeds of the few problem charters would stain the great work of many. Turning a blind eye to the problems in a charter school, or any school, would mean that we failed our children, which is never an option.

It’s a conflict that public officials often face when their official duties require them to make decisions running counter to their personal beliefs.

The mission of the auditor...

Editor's note: This post is the third in an ongoing discussion between Fordham's Michael Petrilli and the University of Arkansas's Jay Greene that seeks to answer this question: Are math and reading test results strong enough indicators of school quality that regulators can rely on them to determine which schools should be closed and which should be expanded—even if parental demand is inconsistent with test results? Prior entries can be found here and here.

It’s always nice to find areas of agreement, but I want to be sure that we really do agree as much as you suggest, Mike. I emphasized that it should take “a lot more than ‘bad’ test scores” to justify overriding parental preferences. You say that you agree. But at the end, you add that we may have no choice but to rely primarily on test scores to close schools and shutter programs—or else “succumb to ‘analysis paralysis’ and do nothing.”

This is a false dichotomy. If all we have are unreliable test scores, we don’t have to make decisions based on them or “do nothing.” Instead, we could rely on local actors who have more contextual knowledge about school or program quality. So if the charter board, local...

Editor's note: This post is the second in an ongoing discussion between Fordham's Mike Petrilli and the University of Arkansas's Jay Greene that seeks to answer this question: Are math and reading test results strong enough indicators of school quality that regulators can rely on them to determine which schools should be closed and which should be expanded—even if parental demand is inconsistent with test results? The first entry can be found here.

The prompt for this forum promised that we would explore “areas of agreement and disagreement.” I’m pleased, Jay (and not altogether surprised), to see that we share a lot of common ground. Let me start with that, then save what I see as our major dispute (what we can learn from reading and math scores) for another post.

I’m thrilled that you dismissed the extreme position of some libertarians, who argue that society should never override the choices of parents. You write:

I…do not mean to suggest that policy makers should never close a school or shutter a program in the face of parental demand. I’m just arguing that it should take a lot more than “bad” test scores to do that.

I agree entirely, and on...

The school choice tent is much bigger than it used to be. Politicians and policy wonks across the ideological spectrum have embraced the principle that parents should get to choose their children’s schools and local districts should not have a monopoly on school supply.

But within this big tent there are big arguments about the best way to promote school quality. Some want all schools to take the same tough tests and all low-performing schools (those that fail to show individual student growth over time) to be shut down (or, in a voucher system, to be kicked out of the program). Others want to let the market work to promote quality and resist policies that amount to second-guessing parents.

In the following debate, Jay Greene of the University of Arkansas's Department of Education Reform and Mike Petrilli of the Thomas B. Fordham Institute explore areas of agreement and disagreement around this issue of school choice and school quality. In particular, they address the question: Are math and reading test results strong enough indicators of school quality that regulators can rely on them to determine which schools should be closed and which should be expanded—even if parental demand is inconsistent with...

Last week, the Department of Education released the 2015 Nation’s Report Card for twelfth graders. As with the fourth- and eighth-grade scores provided last fall, there was little to celebrate. In the core subjects of math and reading, average scores held firm at the same unimpressive level they’ve been at since 2009. The scores of low-performers—whether defined as the proportion of students “below Basic” or those in the bottom decile—actually declined for the first time in at least a decade.

There was one glimmer of good news: High-end reading scores (whether defined as the top decile or the percentage of students at NAEP’s “Advanced” level) rose by a statistically significant margin—the first time that’s happened since 1998. Indeed, this qualified as only the second such upward bump ever for high-end twelfth graders. (Since 1990, there has never been a statistically significant jump at the high end in math or science for high school seniors.)

Moreover, this year’s high-end reading gains occurred despite all other scores (average and low-end reading and math, as well as high-end math) being down or flat across all core subjects in the fourth, eighth, and twelfth grades. That fact that is itself rather unusual. High-end fourth- and eighth-grade reading scores,...

Next week, in a series daily blog posts, Jay Greene and I will explore areas of agreement and disagreement around the issue of school choice and school quality. In particular, we will address the question: Are math and reading test results strong enough indicators of school quality that regulators can rely on them to determine which schools should be closed and which should be expanded—even if parental demand is inconsistent with test results?

The school choice tent is much bigger than it used to be. Politicians and policy wonks across the ideological spectrum have embraced the principle that parents should get to choose their children’s schools and local districts should not have a monopoly on school supply.

But within this big tent there are big arguments about the best way to promote school quality. Some want all schools to take the same tough tests, and for low-performing schools (those that fail to show individual student growth over time) to be shut down (or, in a voucher system, to be kicked out of the program). Others want to let the market work to promote quality and resist policies that amount to second-guessing parents.

Look for the first post, from Jay, on Monday....

A new, somewhat unsettling NBER working paper by Thomas Dee and colleagues examines the prevalence and implications of teachers tampering with student test scores on New York State Regents exams.

The analysts focus on exams taken between the 2003–04 and 2009–10 school years in New York City, which can be reliably linked to students. To qualify for a “local” diploma, the lowest degree available in New York, students entering high school before fall 2005 had to score at least a 55 on all five core Regents exams (English, Math, Science, U.S. History/Government, and Global History/Geography). In fall 2008, local diplomas were eliminated, and students were required to receive at least a 65 score on all five tests.

Up until 2012–13, Regents exams were graded by teachers from students’ own schools, and a policy was in place that required exams with scores just below the cutoff to be re-scored by the schools. The analysts document clear spikes around the cutoffs in an otherwise smooth test score distribution. In other words the scores immediately below the cutoffs appear less frequently than expected from a “well-behaved empirical distribution,” and the scores at or just above the cutoffs appear more frequently than expected, suggesting that scores just below...

Pages