Flypaper

This is the third article in a series that looks at a recent AEI paper by Colin Hitt, Michael Q. McShane, and Patrick J. Wolf, “Do Impacts on Test Scores Even Matter? Lessons from Long-Run Outcomes in School Choice Research.” The first and second essays are respectively titled “How to think about short-term test score changes and long-term student outcomes,” and  “When looking only at school choice programs, both short-term test scores and long-term outcomes are overwhelmingly positive.”

Yesterday, I argued that Hitt, McShane, and Wolf erred in including programs in their review of “school choice” studies that were only incidentally related to school choice or that have idiosyncratic designs that would lead one to expect a mismatch between test score gains and long-term impacts (early college high schools, selective enrollment high schools, and career and technical education initiatives). If you take those studies out of the sample, the findings for true school choice programs are overwhelmingly positive for both short- and long-term outcomes.

Today I’ll take up another problem with their study: They set an unreasonably high bar for a study to show a match between test score changes and attainment. Let me quote the authors...

 
 

In December, Education Next published an article by Rick Hess called “Three Cheers for Imposter Syndrome” that, in part, argues that experts in education would be wise to be more humble. “And, since authentic humility seems to be in pretty short supply these days,” he writes, “I'll happily accept impostor syndrome as a useful facsimile.”

Impostor syndrome is “the fear that you'll be found out at any moment as an impostor who doesn't belong in your job or can't do an important task.” Conventional wisdom would tell us that the “impostor” is someone whose pedigree, depth of knowledge, and/or confidence doesn’t mesh with those of his or her peers. That certainly seemed to be what Hess was feeling when he himself felt like one as he waited to take his Graduate Record Examination.

“Listening to the knowing chatter of the students around me, I was filled with self-doubt,” he recalls. “The room seemed full of budding experts. I wondered how they could know so much and how I would ever keep up.”

But If we take a step back and look specifically at the world of education policy and reform, those who tend to suffer from impostor syndrome...

 
 

This is the second article in a series that looks at a recent AEI paper by Colin Hitt, Michael Q. McShane, and Patrick J. Wolf, “Do Impacts on Test Scores Even Matter? Lessons from Long-Run Outcomes in School Choice Research.” The first essay is titled “How to think about short-term test score changes and long-term student outcomes.”

The AEI paper focuses on a specific question: Is there is a disconnect for school choice programs when it comes to their impact on student test scores versus their impact on student attainment outcomes, namely high school graduation, college entrance, and college graduation rates?

It claims to find such a disconnect. As the authors put it, “A school choice program’s impact on test scores is a weak predictor of its impacts on longer-term outcomes.” But read the fine print because this conclusion follows from two big decisions the authors made, both of which are highly debatable. Had they gone the other way, the results would show an overwhelmingly positive relationship between short-term test score changes and long-term outcomes.

What were those decisions?

  1. The authors included programs in the review that are only tangentially related to school choice and that drove
  2. ...
 
 

Last month, the American Enterprise Institute published a paper by Colin Hitt, Michael Q. McShane, and Patrick J. Wolf that reviewed every rigorous school-choice study with data on both student achievement and student attainment—high school graduation, college enrollment, and/or college graduation. They contend that the evidence points to a mismatch, specifically that “a school choice program’s impact on test scores is a weak predictor of its impacts on longer-term outcomes.”

This week, I plan to write a series of commentaries on the paper, which I believe is fundamentally flawed. I have several concerns, including:

  1. The authors included programs in the review that are only tangentially related to school choice and that drove the alleged mismatch, namely early-college high schools, selective-admission exam schools, and career and technical education initiatives.
  2. Their coding system—which they admit is “rigid”—sets an unfairly high standard because it requires both the direction and statistical significance of the short- and long-term findings to line up.
  3. In their conclusions, they extrapolated from their findings on school choice programs and inappropriately applied them to schools.

That’s a lot to unpack, so I’m going to do this over several posts. I hope you’ll join me for the ride.

First, though,...

 
 
Natalie Wexler

Education reform has come under attack lately, and not just from the usual suspects. Some of the critics are reformers themselves. Others have responded that reform is working just fine. But few on either side are acknowledging the three basic mistakes that have undermined the success of the reform movement from its inception.

The recent criticism has been spurred by reports that the soaring graduation rate in Washington, D.C.—a city long hailed as a national model for education reform—hid the fact that many students had been promoted from grade to grade when they shouldn’t have been. An investigation revealed that over a third of last year’s D.C. public school graduates weren’t entitled to receive diplomas—and, according to at least one teacher, some were unable to read and write.

D.C., it seems, is not an isolated case. Similar scandals have emerged elsewhere, and teachers around the country have reported pressure to pass students who flunked their classes or simply didn’t show up. All of this has led some education reformers to charge that the successes that have been celebrated by the movement for years are largely a mirage.

In response, defenders like Arne...

 
 
Sasha Pudelski

It may seem counterintuitive, but conservative organizations from the Heritage Foundation, to FreedomWorks, to the Club for Growth are pushing an education bill this year that would significantly enlarge the bureaucracy at the U.S. Department of Education. That’s right, the same organizations that have decried the “bloated education bureaucracy” and that give awards to members of Congress who are “fighting daily to shrink government and the federal bureaucracy” are urging Congress to significantly increase secretarial authority over K–12 and higher education.

Why are they doing this? To create a new federal education voucher program that would allow dollars to flow out of public schools and into private schools and businesses. As Congress draws to a close without a signature school voucher victory, these organizations are pounding the pavement to try and garner a big win while both chambers still remain under Republican control. The piggybank for this voucher bill is the Impact Aid program, which is the oldest K–12 federal education program and was created to support school districts impacted by a federal presence, such as military installations, Indian treaty or trust land, and other federal facilities.

The specific voucher legislation, the Military Education Savings Account Act,...

 
 

“Fortnite” has taken over my house and there’s a good chance it has taken over yours, too.

It is the newest and coolest video game, particularly popular with boys. And when I say I’ve never seen my kids love anything more than this game, I’m really not kidding. They’ll hop out of bed at 6 a.m. on a weekend to play. They’ll race to get their homework done so they can play. They’ll even try to feign being sick so they can stay home to play. (Mom doesn’t fall for that one.)

They will even spend hours watching videos of other people playing. Good use of time, right? They argue that it is.

And don’t get me wrong. This isn’t a rant because I hate the game. I don’t.

Unlike many video games that isolate boys into what can be a very unsocial and even antisocial space, “Fortnite” is interactive. In the spirit of the old school employees at Banana Republic, most kids wear headsets that allow them to communicate, hands free, with one another while they play.

I have to confess that, because kids rarely talk on the phone anymore (and talking on the phone was life when...

 
 

New York City’s high school choice program can be, depending on your perspective, refreshing or daunting. When it comes time to choose where they want to spend (hopefully) the next four years, every eighth grader in the nation’s largest school system gets a 600-page school directory enumerating over 700 high schools and program options, before submitting an application ranking up to twelve choices. In theory, this kind of education agora should create all manner of opportunities for students to ferret out just the right high school. But low-income, black, and Hispanic residents and those who do not speak English at home have proven more likely to choose and/or be assigned to high schools with lower graduation rates. Could a targeted intervention in the form of simplified information about school options bring order to chaos and enhance “match”? The short answer appears to be “yes” based on an experiment conducted by New York University economics and education policy professor Sean P. Corcoran and his co-authors, Jennifer Jennings of Princeton, Sarah Cohodes of Columbia, and Seton Hall’s Carolyn Sattin-Bajaj.

The four conducted a randomized field experiment in 165 high-poverty New York City middle schools during the 2015–16 school year, in which students...

 
 

An increasing number of headline-grabbing graduation scandals have renewed the public’s interest in how students earn a high school diploma. A recent paper from the Center for American Progress adds to the discussion by examining high school graduation requirements in all fifty states, as well as the District of Columbia and Puerto Rico, to determine whether they are aligned with college admissions requirements and a variety of quality indicators.

To complete their state-by-state comparisons, the authors reviewed the state-level high school coursework needed to earn a standard diploma, as well as the admissions requirements for each state’s public university system.[1] These requirements were divided into two categories: years of study within each subject area and the course type and sequence within each subject.

The authors used Carnegie units to measure the years of study required by both high school graduation and college admissions standards; one Carnegie unit is equivalent to 120 hours of class time. States were placed into one of three categories based on their coursework requirements for math, English, science, social studies, foreign language, fine arts, physical education/health, and electives.[2] In states where both the high school and public...

 
 
Alexander B. Kress

Sad, disappointed, but not yet despairing. These are my emotional reactions to the release today of the 2017 National Assessment of Educational Progress (NAEP) results of fourth and eighth grade reading and mathematics.

Perhaps I should add “relieved.” Given the hostility that replaced comity in the education world and the widespread assault on accountability in the last decade, I suppose it could have been worse.

Yet how are we to handle the simple fact that we made no progress whatsoever in the last eight years? Achievement results on all measures are absolutely flat. The scale score numbers are virtually the same as they were in 2009. Given the progress we had made before and the progress we still need to make, this stagnancy—being stuck in the mud—is inexcusable and unacceptable.

What happened to bring us to this sorry place? There will be as many hypotheses as there are hypothesizers. Here is my take.

Student achievement was flat in the decade that preceded the flowering of accountability. Then we had meaningful achievement growth in the decade in which accountability peaked. And now we see absolute flatness in the eight years during which accountability has been weakened.

The lesson was and...

 
 

Pages