Rain of Errors

Claim: Rolling back education reform will improve outcomes for students, especially poor students.

Reality: There is no evidence for this claim.

The most frustrating thing about Diane Ravitch’s new book, Reign of Error, isn’t the way she twists the evidence on school choice and testing, her condescending tone toward leaders trying to improve educational outcomes, or her clever but disingenuous rhetorical arguments. No, what’s most frustrating is that we education reformers have left ourselves exposed and vulnerable to her attacks by overselling, and underthinking, our own ideas.

Truth be told, there are parts of the school-reform agenda today that are easy pickings for our opponents. Chief among these is the move to create prescriptive, top-down, statewide teacher-evaluation systems based largely on classroom-level test-score gains. Akin to Obamacare, it’s an idea that seems appealing at first blush (let’s recognize our best teachers and fire our worst!) but quickly devolves into a Rube Goldberg nightmare, with state officials trying (for example) to figure out how to link gym teachers’ performance to reading scores.

Fixing schools, especially from afar, is difficult, treacherous work, yet those of us in the reform community have tried to turn it into a morality play between good and evil. “We know what works, we just need the political will to do it,” goes the common refrain. Balderdash. We know that some things work better than others, and we also know that powerful interest groups (especially the unions) are wedded to the status quo. But anyone with half a brain or more than five minutes of experience also knows that people are complicated, schools are even more complicated, and education is a people-and-schools business. There are no easy answers.

Into these waters wades Ravitch, the repentant reformer, the double agent. She knows the weaknesses in our arguments because she was once one of us. And she exploits them piece by piece.

Which is not to say that she’s fair minded or even handed. She’s neither. For instance, she turns the overwhelming evidence that school vouchers generally benefit a great many recipients (while harming none) into a statement that students failed to experience “dramatic” gains. Guess what? No interventions in education (or the rest of social policy) would meet that daunting standard.

But her book is not a complete disaster for reformers. Far from it, in fact, for Ravitch walks into a trap of her own devising. She acknowledges in the introduction that her last effort, The Death and Life of the Great American School System, failed to offer a positive plan for improving student outcomes. So she sets about to offer one in Reign of Error. In describing it, however, Ravitch commits the exact same errors for which she lambastes reformers. She oversells the evidence, she fails to consider likely unintended consequences, and she doesn’t think through implementation challenges. The skeptical, hard-nosed (if biased and data-slanting) Ravitch of the first half of her book turns into a pie-in-the-sky dreamer in the second half.

Consider her solutions:

1. Provide good prenatal care for every pregnant woman.

2. Make high-quality early-childhood education available to all children.

3. Make sure every school has a “full, balanced, and rich curriculum.”

4. Reduce class sizes.

5. Provide medical and social services to the poor.

6. Devise actionable strategies and specific goals to reduce racial segregation and poverty.

(She lists five other “solutions” that simply amount to rolling back reforms: Ban for-profit charters and charter chains; eliminate high-stakes standardized testing; don’t allow “non-educators” to be teachers, principals, or superintendents; don’t allow mayoral control of the schools; and don’t view education as a “consumer good.”)

So what would a hard-nosed, data-honest Ravitch say about these six ideas?

Claim: Reducing preterm births (via better prenatal care) would improve the life chances of half a million children in the United States every year.

Reality: The government already provides prenatal care to poor women through Medicaid and other programs. One reason the United States has an unusually high number of pre-term births is that it has an unusually high proportion of babies born to young, unwed, uneducated mothers who are less likely to take advantage of quality prenatal care. Solving that problem requires changing a culture that shrugs at fourteen- or sixteen- or eighteen-year-olds getting pregnant (often not for the first time). Ravitch says not a word about those complexities (or anything else about family-structure woes).

Claim: Early-childhood programs have abundant research to support them.

Reality: Most of the evidence for preschool comes from a few boutique programs that were unusually effective and expensive. They served a handful of exceptionally needy young children. High-quality evaluations of Head Start show few gains or gains that fade out after a few years. Evaluations of newer, large-scale programs (like those in New Jersey, Oklahoma, and Texas) suffer from selection-bias problems — we don’t know whether the children enrolled in them might be different in important ways from their peers who didn’t enroll. In other words, the research on preschool is a lot like the research on charter schools: We can find examples of high-quality programs that get great results and we can find plenty of the other kind, but we don’t yet know how to take the great ones to scale.

Claim: Since the advent of No Child Left Behind, many schools have cut back on every subject that was not tested.

Reality: NCLB led to some modest declines in the time allocated to history and science in elementary schools (surely not a good thing). But the well-rounded, content-rich schools that Ravitch desires (as do I) haven’t existed en masse for decades. Ravitch wrote a whole book (Left Back) explaining why this is so — and it had to do not with recent testing and accountability regimes but with the education profession’s commitment to progressivism and romanticism. She wrote another whole book (The Language Police) that vividly explains why so much that passes for history and literature in our schools is banal and not worth learning, and she wrote yet another book (What Do Our Seventeen-Year-Olds Know?) showing how little of it they were learning long before NCLB was even a gleam in George W. Bush’s eye, indeed long before he became governor of Texas.

Claim: The benefits of class-size reduction are so large that the cost is well worth it, in terms of higher achievement levels, higher graduation rates, and lower special-education referrals.

Reality: The evidence indicates that class sizes must be reduced dramatically—to fifteen students or fewer—in order to get an impact, and even then it matters only for the very youngest students in the very earliest grades. Yet class-size reduction is costly in more ways than just in dollars: Expanding the teacher work force makes it that much harder to maintain high standards for entry into the profession (another goal Ravitch asserts), meaning it could actually reduce achievement. (That was California’s experience in the 1990s.) In other words, there are trade-offs at work.

Claim: Wrap-around services, like after-school programs, will close the achievement gap.

Reality: There is no evidence for this claim. For instance, a Brookings Institution study of the Harlem Children’s Zone—one of the few reforms that Ravitch likes—found its students performing on par with peers from charter schools that did not provide wrap-around services.

Claim: We need a new push for school desegregation in order to narrow racial achievement gaps.

Reality: There’s some evidence indicating that integrated schools have a positive impact on the achievement of minority students, especially blacks. But does Ravitch forget her book The Troubled Crusade, which described the disastrous history of forced desegregation? There is no political support for refighting the busing wars of an earlier generation. The recent trend toward gentrification in some cities creates some new opportunities for integrated schools, but these will be limited. Yes, it would be nice if all schools were integrated; it would also be nice if all children had two parents at home. It’s not going to happen. Many low-income and minority students will continue to attend racially and socioeconomically isolated schools for the foreseeable future; the challenge is to make those schools as effective as possible.

Improving schools and helping disadvantaged children escape poverty are heroic challenges. They are complex undertakings rife with uncertainty and potential for missteps. Some proposed solutions would actually make things worse. If Ravitch’s bromides push education reformers toward greater realism, that would be healthy indeed. But who will push Ravitch and her new friends toward greater realism on the anti-poverty agenda? America’s kids are waiting.

This article originally appeared on the National Review Online.

I’ve been in Asia for other reasons (looking into the education of gifted students), but while on the ground in Tokyo, I learned of a fascinating policy dispute that, in the U.S., would be even more controversial.

Compulsory education in Japan runs through ninth grade, but nearly everyone goes to, and graduates from, high school (twelfth grade). Admission to individual high schools, whether public or private, is competitive, and the competition is intense to get into the best and highest-status of them. (At the one I visited the other day, 90 percent of successful entrants had attended juku—cram school—for multiple years to prep for the school’s demanding three-part entrance exam. Yet only 160 of 1000 applicants made it across the threshold.)

Though private schools play a smallish role at the elementary and junior high levels, they’re a big deal for Japanese high school students. A remarkable 30 percent of pupils nationwide attend them, and in the sprawling Tokyo prefecture, it’s as many as 60 percent.

During the postwar years, total enrollments were soaring, and the government determined that encouraging private schools was a bargain. They absorbed a goodly share of the added students at a low price for taxpayers, because parents and other private sources covered most of the cost of facilities and operation.

Along the way, each prefecture negotiated with its private schools a division of the total enrollment such that they were assured a certain level of attendance and the public sector didn’t have to plan for more than a predictable share of the whole.

As enrollments have tipped downward, however—part of Japan’s current demographic crash—some private schools have had more difficulty attracting enough pupils. And they have an influential lobby, particularly with the Shinzo Abe government that took power ten months ago.

Additional background: Since 2010, public high schools in Japan have been tuition-free, and private high schools have enjoyed public subsidies equivalent to the government per-pupil outlay for public schools. This was the work of the previous government and intended both to foster universal high school education and to maintain rough parity between the two sectors.

Enter the Abe government with a stunningly controversial proposal: to impose tuition charges for public high school attendance by children of wealthy families (annual income of about $92,000 upward) and to use the proceeds from that tuition charge to subsidize the attendance of low income children in private schools!

This would reportedly affect some 800,000 high school students and would presumably cause some to exit the public sector, while pumping up private school rolls and making it easier for poor families to exercise that option.

In U.S. terms, it blends the liberal idea of income redistribution with the conservative idea of helping poor kids choose private education.

It would also presumably require prefectures to revisit the public-private enrollment divisions that they’ve worked out over the years.

There’s all sorts of resistance to this plan—some critics declare that it violates the “International Covenant on Economic, Social and Cultural Rights.” (This pact may be as unfamiliar to you as to me, perhaps because the U.S. has not ratified it, but it passed the U.N. General Assembly almost half a century ago and Japan has agreed to honor it.) A simpler and more widespread criticism is that it violates the Japanese belief in free, universal education, at least through high school. Some political commentators say it is simply the new government taking revenge on a treasured accomplishment of its predecessors.

That seems cynical, but I can’t read the political entrails in Japan myself. I did learn this, however: When I asked why this change was not being forcefully resisted by teacher unions and other defenders of public education, my informants’ response was a chortle and the comment that the teacher unions are not nearly as influential with the present government as are the private schools.

Eat your hearts out, American voucher proponents. The more so as this policy shift makes its way through the legislative process.

This article was updated on October 31, 2013, for inclusion in the Education Gadfly Weekly.

After a week of insider chatter predicting that L.A. schools chief John Deasy would resign in February, the L.A. Unified Board of Education issued him a satisfactory evaluation—and his contract has been extended through 2016. It’s no secret that L.A.’s teacher union has no love of Deasy, due in part to his support for the parent trigger, his push for student-performance-based teacher evaluations, and his Breakfast in the Classroom program. Most recently, he has been criticized for his handling of the district’s $1 billion iPad rollout. For Dara’s analysis of what’s next for Deasy and LAUSD, check out this week’s Education Gadfly Show podcast. [Link to podcast]

On Tuesday, the Department of Justice filed a motion asking a Louisiana district court for more time to produce documents requested by the state’s attorneys—including the federal desegregation orders upon which the DOJ based its lawsuit against the Bayou State’s school-voucher program in the first place. Governor Jindal promptly responded, pointing to this as yet another example of the Obama administration’s incompetence. “Were these documents lost in the Obamacare website? Or did the Department of Justice just ignore the documents and file a lawsuit against the state without having all of the information available?” Whatever the case, Holder’s certainly made a hash of it. Here’s hoping he cancels the lawsuit entirely.

In the next school year, field testing of new Common Core assessments will be complete, and states will be faced with the weighty decision about which tests they will use to measure student learning going forward. Two state consortia, the Partnership for Assessment of Readiness for College and Careers (PARCC) and the SMARTER Balanced Assessment Consortium (SBAC), are currently developing the leading options for Common Core–aligned assessments. But states in which anti–Common Core sentiment runs deepest have begun to back away from the consortia (to date, four states—Alabama, Georgia, Oklahoma, and Utah—have officially withdrawn), leading to consternation among Common Core supporters and joy among detractors. This has left remaining states in a pickle: If additional states withdraw, will the cost of consortia-developed assessments skyrocket as the fixed costs are spread over fewer states? This new report from the Brookings Institution’s Brown Center sought out answers—and contains good news for those states choosing to stay the course. After calculating the projected costs for consortia-developed assessments, summarizing key differences between the two, and estimating the costs for non-consortia testing options such as ACT-developed assessments and vendor-developed, state-specific tests, author Matthew Chingos found that price fluctuations that could occur if more states withdraw would be relatively minor. For example, even when considering the possible departure of Florida, PARCC’s second-largest member, the price of PARCC’s tests would only increase by about 60 cents per student. If only the fifteen states currently field testing PARCC were to ultimately adopt the tests, test costs would increase by just $2.50 per student. For SBAC, Chingos estimates that the withdrawal of the six consortia states experiencing the most political contention would only increase the cost of its tests by $2.60 per student (though he notes that there is slightly more uncertainty about SBAC’s price estimates than about PARCC’s). Importantly, the report also stresses that as a nation, we do not spend very much on assessments; PARCC and SBAC are currently estimated to ring in at $29.50 and $22.50 per student, and the current state average is $27. This seems a drop in the bucket when compared to the $10,500 spent per student annually. When the success of the standards hinges on the quality of the tests, this seems a shortsighted place for states to penny pinch. 

SOURCE: Matthew M. Chingos, Standardized Testing and the Common Core Standards: You Get What You Pay For? (Washington, D.C.: Brookings Institution, October 2013).

The “fifty-state review” of educational policies has proliferated into a literary genre of its own. Extant are fifty-state reviews of academic standards, charter school laws, a whole plethora of ed-reform policies, teacher-union strength, and even bullying laws. Add to this growing body of literature the National Council on Teacher Quality’s (NCTQ) recent fifty-state review of teacher evaluation policies. For NCTQ analysts, it’s not merely the teacher-evaluation tool per se that is important—it is also about how schools use evaluations in staffing decisions. The following are the three key takeaways from this study: First, teacher-evaluation policies have moved speedily—propelled, in part, by federal policy—through state legislatures. For example, in 2009 just fifteen of the fifty-one states (including D.C.) mandated that teacher evaluations be based on objective measures of student achievement. Now, forty-one states require such measures. Second, the degree to which teacher-evaluation systems are rigid or flexible varies across states. More than half of states (twenty-nine) give local districts considerable latitude in designing and implementing a homegrown evaluation system. The rest of the states take a heavier-handed approach, either mandating a single statewide system for all districts or a statewide system where districts can seek an “opt-out” waiver from the state if they implement a comparable evaluation system. Third, among the states with a teacher-evaluation policy based on achievement measures, state policies vary widely in how evaluations are linked to staffing decisions. Just six states tie evaluations to teacher salaries, while twenty-three states tie unsatisfactory evaluations to dismissal. To conclude, NCTQ rightly acknowledges that teacher evaluation is daunting task, fraught with both logistical and political hazards. In order to “stay the course” on teacher-evaluation policy, NCTQ provides fifteen practical recommendations that should help fair-minded policymakers traverse the precarious field of teacher-evaluation policy.

SOURCE: Kathryn M. Doherty and Sandi Jacobs, State of the States 2013: Connect the Dots: Using evaluations of teacher effectiveness to inform policy and practice (Washington, D.C.: National Council on Teacher Quality, October 2013).

The latest study by IES attempts to document how American eighth graders compare to their peers around the globe. Using NAEP scores to predict performance on TIMSS, an international test that examines what students know about math and science, analysts included thirty-eight countries and nine other educational systems in their inquiry. And the results? Not terrible. Eighth-grade students in thirty-six states outperformed the international TIMSS average in math, and those in forty-seven states did so in science. In the interest of naming names, the states that performed below that average in math included Oklahoma, Tennessee, West Virginia, Alabama, and the District of Columbia, while four systems—South Korea, Singapore, Hong Kong, and Taiwan—bested every U.S. state in math. Massachusetts did well in math compared to other systems, but when matched against the top performers, its scores weren’t anything to write home about: For instance, 19 percent of its eighth graders scored at the “advanced” level—compared to roughly 50 percent in Taiwan, Korea, and Singapore. As for science, forty-seven states scored higher than the TIMSS average, while three states’ scores—Mississippi, Alabama, and D.C.—scored lower. Six out of ten of the top scorers in science were in the U.S., including a decent chunk of the northeast states: Massachusetts, Vermont, New Hampshire, and Maine. In general, the U.S. fared better in science than it did in math. That said, we ought to keep in mind a few caveats: (1) NAEP and TIMSS measure different things [link to podcast segment], even if the scores on them can be made statistically equivalent; (2) NAEP tests mostly public school students, while TIMSS includes private schools; and (3) the countries that were included in the study are both developed and developing countries, so the comparisons must be considered in that light. The bottom line: Our eighth graders are mostly average, but still lagging behind the world’s highest performers.

SOURCE: National Center for Education Statistics, U.S. States in a Global Context: Results From the 2011 NAEP-TIMSS Linking Study (Washington, D.C.: Institute of Education Sciences, October 2013).

New from a workgroup of the National Association of Charter School Authorizers (NACSA), this report maps an oft-overlooked space in the charter-accountability world: How charters that serve special populations, such as students who have dropped out, are held accountable for performance. Two key points emerge (which are really applicable to all charters): (1) Make the charter contract the central instrument of accountability and (2) be open to different yet detailed and rigorous approaches to evaluating academic success or failure. Interestingly, the report recommends not making significant changes to operational and financial indicators or methods of oversight for alternative schools. Approaches to the performance frameworks can vary from setting different cut scores to wholly different accountability measures specific to alternative schools. The report discusses proficiency, growth, and multiyear graduation rates, as well as providing credit for re-engaging students who have dropped out and improving attendance, mastery of material, and college/career readiness. Some of the more thought-provoking proposed measurements included job stability and time employed in a particular position, reconnecting with family members, personal growth, and volunteer work. For programs targeting formerly incarcerated students, recidivism rates could be examined; for programs that work with addiction, perhaps the time a student remains drug/alcohol free might be a measure. Additionally, the authors also include a synthesis of several studies of dropout prevention (including one that starts in first grade). In Ohio, we as an authorizer are considering re-writing our own accountability agreements with the schools we authorize. Several of the measures—especially the idea of weights given to different indicators—seem promising as a means to fairly gauge whether and to what degree schools are educating their students, regardless of the focus of the program. Those developing or tweaking performance frameworks, for alternative schools or otherwise, should give this report a read.

SOURCE: National Association of Charter School Authorizers, Anecdotes Aren’t Enough: An Evidence-Based Approach to Accountability for Alternative Charter Schools (Chicago: National Association of Charter School Authorizers, October 2013).

Dara attempts to understand why Brickman hates Halloween. In the meantime, they tackle Michigan’s legislative strategy for keeping the Common Core, John Deasy’s job status, and the cost of high-quality tests. The TIMSS-NAEP linking study isn’t all bad news for U.S. eighth graders, says Amber.

Many proponents of private school choice take for granted that schools won't participate if government asks too much of them, especially if it demands that they be publicly accountable for student achievement. Were such school refusals to be widespread, the programs themselves could not serve many kids. But is this assumption justified?

A new Fordham Institute study provides empirical answers. Do regulations and accountability requirements deter private schools from participating in choice programs? How important are such requirements compared to other factors, such as voucher amounts? Are certain types of regulations stronger deterrents than others? Do certain types schools shy away from regulation more than others?

These are just some of the questions that David Stuit, author of the Fordham study, will discuss with a panel featuring John Kirtley of Step Up for Students (Florida), Larry Keough of the Catholic Conference of Ohio, and Paul Miller of the National Association of Independent Schools.