Standards, Testing & Accountability

The donkey debate edition

Education in the first Democratic debate, whether Common Core is winning the war, Arne Duncan’s influence on President Obama’s education policy, and the effects of scaling-up pre-K. 

Amber's Research Minute

SOURCE: Dale C. Farran and Mark W. Lipsey, "Expectations of sustained effects from scaled up pre-K: Challenges from the Tennessee study," Brookings Institution (October 2015).

Transcript

Kevin:                   Good morning Mr. and Mrs. America. From border to border and coast to coast, and all the ships at sea, this is your host, Kevin Mahnken at the Thomas B Fordham of Institute. Here at the Education Gadfly Show and online at edexcellence.net. Now, please join me in welcoming my co-host, a first time guest of the Gadfly show so take it easy on her, Ellen Alpaugh. The Lincoln Chafee of Education reform. I intentionally picked Lincoln Chafee because I figured there was a possibility that maybe you haven't been keeping up with the news or-

Ellen:                     Oh I know, he's running for President.

Kevin:                   No, no, I'm going to put this to the test. I'm going to see. I've written up a multiple choice question about who Lincoln Chaffee might be. Of these four, Ellen, try and see who or what Lincoln Chaffee is. I can give you a hint. It's concerned with Rode Island, okay? So Lincoln Chafee is either A-

Ellen:                     I knew that.

Kevin:                   A small Woonsocket law firm that handles trusts and estates. B, a soft rock duo famous for their 1978 hit Kiss on my Lips.

Ellen:                     Ohh, that's tempting.

Kevin:                   Or C, the parent company of Lincoln Logs. Which of those 3?

Ellen:                     Is it C?

Kevin:                   No, no, it's not.

Ellen:                     It's none, there's none of the above.

Kevin:                   It's D, Lincoln Chafee is the former Senator, current-

Ellen:                     Of Rhode Island.

Kevin:                   Recently former governor of Rhode Island.

Ellen:                     Yeah.

Kevin:                   As you may have seen, he was present at the Democratic presidential primary debate, first one of the season. Now, we were kind of looking at that. What did you think of sort of some of the performances? Were you like thumbs up, thumbs down on any of the candidates?

Ellen:                     Yeah. I think Hillary did pretty well and yeah, I don't know.

Kevin:                   Yeah, she looks like she is probably the front runner. I mean that's sort of what everyone assumed at the out said. However after like 6 months of relentless criticism, I think maybe people were starting to get doubts. I think she really reasserted herself last night as the candidate to be. I was not crazy about the performances of everyone else. Martin O'Malley looked like he's basically running for vice president, trying to flag the Glass Stegall issue and see if he'll get a vice presidential nod. Bernie Sanders looks like he's running for president of the student government at Sara Lawrence college. It's just not like the line up of Titans that you see for the Republican. Now, actually our first question has to do with the debate last night. So I'm going to throw it over to Clara to ask our first question.

Clara:                     The first democratic presidential debate was last night. Was the lack of discussion around education K-12 concerning?

Kevin:                   All right, that's an interesting question. As you mention, there was really very little discussion of education, I mean none of K through 12 education.

Ellen:                     Yeah, nothing on K through 12 but there was some mention of college.

Kevin:                   Yeah, higher ed got some discussion. Sort of all the candidates had college affordability plans, how to reduce tuition costs and so forth at state universities. But what about, is this basically just a case of it being a pre-non sexy issue? They're not going to mention it and no harm, no foul, what do you think?

Ellen:                     Well I think it goes beyond it not being a non sexy issue. I think the candidates are really scared to make a stance on this. It's super divisive. Anything they're going to make a stance on whether it be standards, or school choice or anything, they're going to lose as many voters as they may gain by doing so. That's my thought and it also speaks to, it's also supported by the fact that Democratic candidates just recently declined an invitation from Campbell Brown to speak about this very topic at the education summit later this month.

Kevin:                   Yes. It sounds like, yeah you're quoting from our brilliant college Kate Stringers. He's castigating the Democratic candidates for not going to this 74 candidate form.

Ellen:                     That's correct, Kevin.

Kevin:                   And that does strike me as sort of a case of political cowardice on their part. Do you, I mean, my impression because I also happen to write a little piece about this, still though you weren't good enough to bring that up, it was that basically these issues that don't get brought up in political campaigns, they're ... Issues that don't end up being talked about, they don't end up ever being addressed either when a candidate comes into office. That is, political scientist have sort of looked at this and what they found is that candidates end up making promises and presidents end up keeping promises. Presidents prioritize delivering on things they have promised to their base, even if that's just in the primary. You know, and then they get through the primary and they're running in the general election. They're going to do fine but they always end up going back.

                                I mean President Obama campaigned on health care reform. He campaigned a little bit on climate change. I've seen action on both of those issues. Now something like Guantanamo Bay was like a want to have but it’s not like he really ran on it in 2008. So you see a lot less activity on that issue. Do you suppose that if we heard anything on K through 12 from these guys it would be anything ground breaking or ...

Ellen:                     Well, not so much. I think that everyone's track records on the democratic side show that they're pretty much in support of most of the reforms, the standards at least. And whatever name they are.

Kevin:                   Yeah, yeah, yeah I think you're right. Like Hillary last night had to dodge a lot of accusations that maybe she's running for Obama's third term. And it sounds to me like probably that's ... They're going to end up lining up behind high standards, probably going to have to square the circle with charter schools which democrats have had a hard time with for a long time. Maybe in fact it's kind of the dog that didn't bark. We'll see. I hope we'll get more talk from both Democrats and Republicans about this. Because it’s not as if the Republicans have been chatter boxes on education either. Clara, how about number 2?

Clara:                     Secretary of Education, Arne Duncan's tremulous education reform policies prove divisive amongst liberals. Did he manipulate the Obama administrations vision for eta form?

Kevin:                   Excellent question. Clara is getting down to the bottom of the mystery of just who has been pulling president Obama's puppet strings all these years.

Ellen:                     It's been me.

Kevin:                   It's been, Ellen Alpaugh from her perch in the corner at, pushing a mop and a broom around, in the office at the Thomas B Fordham of Institute is actually been dictating education policy. This is just interesting. This question comes from an article that was written by Jonathan Chat from New York magazine. I highly recommended it. It’s called Was Arne Duncan Secretly Obama's Boss All Along? And what he effectively says is that Arne Duncan resigned a few weeks ago. That's sort of old news but what you're starting to see now is the retrospectives from different political commentators, liberals and he's quoting a piece specifically from Charles Pierce of formally the Boston Phoenix, now of Grantland and an idol of mine journalistically. Pierce says that, I can't quote it exactly but basically the reform type policies, things that race to the top, prioritizing high standards, these were policies that President Obama basically implemented reluctantly. He uses the word at the behest of Arne Duncan. That he went along, that he went along to go along. Ellen, is that your impression of how the White House works, that the Secretary of Education who calls the shots?

Ellen:                     I wouldn't say so, no.

Kevin:                   I shouldn't think so.

Ellen:                     I hope not.

Kevin:                   Yeah, I mean, it seems to me that ... You're work on most in the comms side I mean, what has your impression been on people's views of Arne Duncan? I mean, I think he's sort of been made to take the fall here.

Ellen:                     Yeah, and it’s really easy to beat up on somebody who is the face of education reform for the United States, especially when a lot of people don't want anyone in the federal government to be talking about it at all.

Kevin:                   Yeah, quite so, quite so. Duncan in a lot of these cases, the fall guy like you say is an un-elected person. It's somebody that was sort of thrusts in a roll and made the face of a movement of a set of policies. Arne Duncan is convenient for that in that way he sort of reminds me of Eric Holder. Former attorney general who people speculated was like Obama's lightning rod. That these figures were being kept in place because they sort of allowed liberals and conservatives to both beat up on the straw man and Obama would escape unscathed.

                                I have to say that the argument to be seems to be a fallacious one. These liberals are saying, you know it’s too bad that Arne Duncan got to run education for the last seven years if only the president had known, had he ever looked at what was coming out of his department of education. Which is, you know, obviously absurd. It reminds me of like a Gallo prisoner in Russia in the 1950's saying like if only Comrade Stallings knew how bad things were here, he wouldn't stand for something like this. I mean, this is the president. The president makes the pick, he makes the selection for personnel. And there after, his policies are implemented. So I guess I'd like to see from now on for a little more accountability. If you have something against the president, well then NEA, whoever the critic from the left may be, direct it towards the guy who is probably making the call, make sense?

Ellen:                     Well said, Kevin.

Kevin:                   Aw, thank you. I'm going to throw it over to Clara for number 3.

Clara:                     Politico recently wrote that Common Core has recently won the war. Is it true? Is the great battle of education reform over?

Ellen:                     Well, I guess we can all go home now.

Kevin:                   Yeah. I was hoping so you know. Our human resources guy will be happy we don't have to rent out the office space anymore.

Ellen:                     Yeah, job well done, job well done.

Kevin:                   We can just, I can go back to hand gliding lessons. This is a claim that was made in Politico the other day. It was two days ago, basically speculating that after years now of work setting up the conservator, writing the standards, getting states to buy in from the National Governor’s Association, that effectively Common Core is now just a fact of life. It's in place now I believe in 42 states, the District of Columbia also has it. You've had 2 or 3 states like South Carolina and Oklahoma who either initially adopted it, then pulled out or who simply never adopted it in the first place. It's, but for the mast majority, what was the figure for the kids who are in class rooms with Common Core?

Ellen:                     I think it's about 40 million right now, that's 4 in every 5 kids. Is it 40?

Kevin:                   Yeah, 40 million. Okay, yeah, yeah, yeah, it's 80 percent so yeah that'd be 4 out of 5, so 40 million. The vast majority therefore are in states with Common Core. Does this strike you, Ellen, as kind of a bit of a preemptive declaration of victory? I mean are we actually ... What's your stance, I mean are we actually there yet?

Ellen:                     The battle is only half won I'd say.

Kevin:                   Okay General Eisenhower. All right.

Ellen:                     Yeah, so Common Core has pretty successfully been adopted by most states. I think that's hopefully the way it's going to be. The next challenge is definitely making sure that the tests that are supposed to measure how kids are doing with these standards are actually measuring what they say they're going to do. They are actually aligned to Common Core. And beyond that, there's communicating those results to parents and students and finding a way to deal with the fact that a lot of these kids who have previously been told they are doing okay, are actually really not.

Kevin:                   Yeah, yeah. That's what we've addressed on the Fly paper blog a lot of it, you know, fits the proficiently illusion. The idea that we've been, we've now been telling generations of kids, generations of parents, your kids doing fine, don't worry about it, yes of course there's an education crisis in this country but it has nothing to do with darling little Aiden. Unfor- I hate that name- unfortunately, unfortunate now eventually there's going to come a time where you set your sights a little higher and unfortunately you're not going to be able to deliver that comforting message anymore.

                                I think the other point, the one last thing we have to settle here is the political question. You're very right Ellen that implementation is important and that's on the job of ed reformers to hold people's feet to the fire. But politicians have a roll here too. Once a policy, especially one that's not produced by federal statue, like a lot of Common Core critics claim that it was, one that was voluntarily adopted by the States. Once it's put into place, that doesn't mean it's set in stone, right? Then President Bush, again through statute. Then these things went to congress, initially they could have a little more staying power but President Bush put in place tax cuts in 2001 and 2003. Those tax cuts have changed because government changed. Just off the top of my head, President Obama made Obamacare a thing that the future of Obamacare is not set in stone either. So if we think we have a good thing here, my understanding is we need to make sure we preserve it. Especially where now in 2016, we've got politicians running for president who are making it basically the point of their campaign, some of them, repeal Common Core. It seems to me there are, who was it, Bobby Jindal, Chris Christie, I mean who's running against it this? I mean there are figures who actually hate Common Core.

Ellen:                     Oh yeah. I'd say that most of the Republican cohort, right?

Kevin:                   Well it's not all, prominently Jeb Bush but if you care to speak about Common Core, it seems as though, and not a few liberals as well, if you speak about Common Core, generally speaking, you're bringing it up to condemn it. So I don't think we've turned the page yet. But hopefully we've reached if not the beginning of the end, the end of the beginning. I believe now we are going to transition to my main man David Griffith for our research minute after a quick word from our sponsor Lincoln Chafee, state trusts serving Woonsocket, Rhode Island since 1931 ... And now welcome our guest today, the Jim Webb of education reform, could we call him that ?

David:                   You could not.

Kevin:                   I could not call you the Jim Webb, how about he Jeff Merkel? The Oregon senator-

David:                   Merkley, Merkley.

Kevin:                   Merkely?

David:                   Yeah.

Kevin:                   I'm getting confused with the German Chancellor. I think they share one another's charms. It’s that native charisma.

David:                   Are you going to introduce me?

Kevin:                   Yeah, yeah, David Griffin, everyone knows who the Jeff Merkel of education reform is.

David:                   Okay, glad to be here Kevin.

Kevin:                   And he's going to share his homework, which was a study of the preliminary results of the Tennessee volunteer pre-k effectiveness study.

David:                   All right, thank you Kevin. That is the study we will be talking about today and it’s an important study. And what makes it important is its design. It’s based on a randomized control trial which is though sort of the widely known gold standard for rigorous research in education and elsewhere. So this isn't the first time such a design has been used to evaluate a pre k program. But it is the first time this method has been used to evaluate a scaled up state funded pre k program and we'll talk about the difference between those. For this study, the researchers have, basically they've been tracking the progress of about 3,000 students in Tennessee as they enter elementary school. About two thirds of these students participated in Tennessee’s pre K program. Of those 3,000, about a thousand were evaluated more intensively so for this group the researchers sort of directly administered a number of scales test that are gauged at the pre k level obviously. And also their teachers provided sort of annual ratings of their non-cognitive skills. What did they find? Well unfortunately this is yet another disappointing pre-K study.

                                The results are extremely discouraging. Like a lot of other studies, this one finds that participating in Tennessee program does give kids a head start when they enter kindergarten in a lot of measures, but by the end of kindergarten, this advantage had basically disappeared a lot like it did in the famous head start impact study. And worse, by second grade, the kids that participated in the Tennessee pre k study actually scored lower than the kids in the control group on most of the measures. So since the studies come out, lots of some pre k advocates have sought to down play it and arguing that Tennessee offered lower, has a lower quality pre k than other states. It's not clear that there's much evidence that that's actually true, and more over these results obviously fit into a pattern that we've seen elsewhere which is the clear and initial benefits followed by this really rapid fade out as the kid, the kids enter the K-12 system. So as authors Dale Farren and Mark Wipsy note, there is some as yet poorly interaction between the pre-K experience and the experience children have in subsequent grades that fails to carry forth the momentum they gain in pre k. So at least for me all of this is pretty depressing.

                                If you believe as I do that high quality pre-K does have the potential to change the lives of many underprivileged kids as it seems to have done in a lot of the early studies like the Perry Preschool study and Advocacy Darwin study. So unfortunately were operating with fuzzy definitions of pre k and high quality and it's not clear if we can scale that sort of success.

Kevin:                   Now, commentators on this study, it sounds as though you may share this, there's a certain amount of fatalism. You mention the head start study as well. Are there and I can't blame you for being fatalistic, you're a trail blazers fan, is there reasons for hope on the horizon? I mean it does seem dispiriting, I agree.

David:                   Well, so as Audrey was mentioning before we recorded this, there is this evidence from these long term studies that actually the benefits of pre k can actually disappear or become dormant for a number of years and then sort of reemerge from the data later in life. So even though they don't show up in test scores in the K-12 years, it sometimes does seem like kids do better once they are adults. The problem is, there’s a 30 year time lag between the point of which we offer pre-K and the point of which we know if those results actually occur. Right, so it's pretty difficult to make a policy I think based on benefits that won't show up for you know till 2030. I buy those studies but it also it still leaves us in a pretty tough place I think.

Kevin:                   Yeah it certainly does. A tough and lugubrious place. Thank you very much David Griffith for stopping by to bring down the Educational Gadfly show. And on that sour note, that's all the time we have for this week’s Gadfly show. Until next week-

Ellen:                     I'm Ellen Alpaugh.

Kevin:                   And I am Kevin Mahnken for the Thomas B Fordham Institute signing off.

Radio:                   The Education Gadfly show is a production of the Thomas B Fordham Institute located in Washington DC. For more information visit us online at edexcellence.net.

Matt Barnum

In a series of blog posts (IIIIIIIV), Jay Greene argues against the “high-regulation approach” to school choice. I’m going to focus on the final two posts, in which Greene argues that student achievement tests are poor proxies for school quality and that they’re not correlated with other measures of quality.

I think Greene is right to a large extent. But he undersells the value of tests.

It’s pretty clear that the ability of a school or teacher to increase students’ standardized test scores is associated with long-run outcomes. Let’s dig in to some evidence:

  • The well-known Chetty study used a rigorous quasi-experiment to show that teachers with high value-added scores (which are based on standardized tests) produced higher income, greater college attendance, and lower teen pregnancy among students. (In the comments of his post, Greene acknowledges this study but describes the effects as small. I disagree, considering we are describing the effects of a single teacher at a single grade level.)
  • A different Chetty study reports that “students who were randomly assigned to higher-quality classrooms in grades K–3—as measured by classmates' end-of-class test scores—have higher earnings, college attendance rates, and other outcomes.”
  • Hanushek finds that international academic achievement
  • ...

Writing in his always-entertaining blog a few weeks ago, Whitney Tilson gave a nice nod to Dan Willingham’s New York Times op-ed addressing the sorry state of American teacher preparation. Amid effusive praise of the piece, Whitney writes, “I think morphemes and phonemes matter too but maybe not as much as Willingham does.”  
 
This gently stated but dismissive view of the importance of reading instruction troubles me because I think it captures a viewpoint widely shared by many education reformers.
 
I don’t think it’s because there are many education reformers who reject the science here (unlike many in teacher preparation). Researchers long ago identified the reading methods that would reduce the current deplorable rate of reading failure from 30 percent to somewhere well south of 10 percent, if only schools would take that step. Teacher preparation programs that fail to impress upon elementary teacher candidates the integral connection between spoken sounds and written words are essentially committing malpractice.
  
Instead, I think the issue for some education reformers is that other reforms seem much more important. I can’t figure out why there are still perfectly reasonable, rational people who aren’t willing to embrace the 2 + 2...

The Seventy Four had a simple goal: to make the 2016 presidential election season one in which candidates could pause in their frenzy of backstabbing and baby kissing to talk about education. In a first-of-its-kind education forum, the site (with the help of sponsor and cohost the American Federation for Children) invited presidential candidates to discuss their vision for public schools. Republicans spoke in August, and Democrats were supposed to take their turn later this month.

But as Politico recently reported, the Democrats declined their invitations. It’s a missed opportunity. Worse, nobody seems to know why the candidates backed out.

Campbell Brown, the Seventy Four’s co-founder and would-be forum moderator, says it’s due to pressure from teachers’ unions (both the AFT and NEA have publically endorsed Hillary Clinton). “What happened here is very clear: The teachers’ unions have gotten to these candidates,” Brown told Politico. “All we asked is that these candidates explain their vision for public education in this country, and how we address the inequality that leaves so many poor children behind.” Representatives from the unions, unsurprisingly, won’t verify her claim. More troubling, the candidates won’t comment on their refusal to join in the debate. They’re remaining...

“The problem in American education is not dumb teachers. The problem is dumb teacher training,” University of Virginia cognitive scientist Dan Willingham recently wrote in the New York Times. Indeed, if there’s any part of the education pipeline that’s ripe for retooling, it’s the way we prepare teachers. Complaints are legion, long-standing, and not unique to policy wonks. Teachers themselves routinely bemoan how poorly prepared their training left them for the realities of classroom life. Fewer than half of new teachers described their training as “very good” in a 2012 survey by the American Federation of Teachers, while one in three new teachers reported feeling unprepared on his first day.

Thus, it can only be viewed as a great good thing that two dozen deans of education schools have come together under the banner of “Deans for Impact” and committed themselves to a common set of principles, including data-driven improvement, common outcome measures, empirical validation of teacher preparation methods, and accountability for student learning. They’re also persuading other teacher preparation programs to do the same.

At a Tuesday event at the National Press Club, the group unveiled a ...

Secretary of Education Arne Duncan deserves the many plaudits he received on Friday from President Obama and his friends in the reform community—and even from his sometime-foes in the teachers’ unions. As everyone remarked, he’s a good and decent man, a fighter for disadvantaged kids who’s passionate about his work and loyal to his team. That was certainly my personal experience with him; he was more gracious toward me than I probably deserved, considering the many swipes I’ve taken at his policy decisions over the years.

So please bear with me one more time: Even at this moment of celebration, congratulation, and reflection regarding Arne’s time at the helm, the Obama administration can’t seem to help itself. It almost seems determined to poison the well with Congress and play to the stereotype of a government unwilling to abide by constitutional limits.

I’m referring, of course, to the decision to appoint John King (another smart, committed reformer and all-around great guy) as “acting” education secretary for an entire year rather than putting him through the Senate confirmation process.

It’s certainly true that the confirmation process has slowed to an agonizing pace over the past few decades. And the Bush 43 administration also opted...

As Ohio lawmakers return to Columbus, a debate is brewing about how to measure the effectiveness of e-schools. At issue is the fact that a large fraction of their students are mobile—for example, our 2012 student mobility report found that less than half of online students stay for more than a couple years.  Some e-schools assert that it’s unfair to hold them accountable for raising the achievement of children who spend such a brief time under their supervision.

Are they right? How should we think about accountability for e-schools, or other schools with a highly mobile population? (Our mobility study revealed that urban schools also experience high rates of mobility.) Should state policymakers make accommodations for schools with a more transient student body? Or should they stand firm on accountability, regardless of the challenges of serving a mobile population?

To be sure, these are tough issues, but policymakers can look towards a few guiding principles.

First, all kids count. Every student deserves an excellent education, regardless of whether she’s brand-new to a school or has been enrolled for several years. Think of it this way: when a fourth grade student moves from one school to another, shouldn’t the...

While plenty of folks seem to think that getting rid of Common Core would be good for schools, the standards remain largely intact in most states across the nation, including here in Ohio. Before supporters start congratulating each other on victory, however, they would be wise to recognize that the real battle for Common Core has just begun. As my colleague Robert Pondiscio points out, “far too little attention has been paid to the heavy lift being asked of America’s teachers—and the conditions under which they are being asked to change familiar, well-established teaching methods.”

This heavy lifting includes selecting curricula to teach the standards (because the standards aren't a curriculum—districts choose their own). The lift gets Atlas-like when one considers the poor alignment of the curricula from which districts and teachers can choose. Since last summer, researchers have called out textbook publishers’ misleading claims of alignment with words like “sham,” “buyer beware,” “disgrace,” and “snake oil.” Slapping “shiny new stickers on the same books they’ve been selling for years” has probably lined some pockets, but it’s also left teachers high and dry—and still hefting the weight of ensuring that students master...

Hannah Putman

When trying to improve educational outcomes, it is hard not to feel the need for urgency. We want to figure out what works now and implement changes immediately—because if we wait, kids who are in schools now will miss out. Unfortunately, this pressure to act quickly may be fundamentally at odds with the ability to measure what really works, since meaningful changes in the trajectory of student achievement are not always apparent until years later. Diane Whitmore Schanzenbach of Northwestern University provides a compelling example of exactly this conundrum.
 
Schanzenbach’s thesis is that too often, education research only assesses an intervention’s immediate or intermediate outcomes without capturing its long-term benefits. This may be particularly relevant, she asserts, when judging the impact of early childhood investments.
 
Schanzenbach offers the example of two studies (both of which she co-authored) on the famous 1990s Project STAR class size experiment in Tennessee. That well-known experiment assigned students randomly to either regularly sized classes or smaller ones. Researchers behind both papers (the first from Dynarski, Hyman, and Schanzenbach and the second from Chetty, Friedman, Hilger, Saez, Schanzenbach, and Yagan) found that the smaller kindergarten classes yielded an immediate bump in student test scores for that year; but both papers report...

Since 2003, Florida has required that schools retain third graders who fail to demonstrate proficiency on the state reading test. A new study by Martin West and colleagues examines the impact of this policy by rigorously comparing the results from students who are just above or below the cutoff for retention. The first cohort to be affected by the new policy entered the third grade in 2002, and West et al. track it through high school graduation. They also track five additional cohorts, the last of which entered third grade in 2008.

Unsurprisingly, they find that the policy increased the number of third graders retained. It started with 4,800 kids in the year prior to the policy introduction (2002) and jumped to nearly twenty-two thousand the next year. The numbers retained have fallen steadily over time, however, as more students have cleared the hurdle. The study’s key finding is that third-grade retention substantially improves students’ reading and math achievement in the short run. Specifically, reading achievement improves for retained students by 23 percent of a standard deviation after one year—and by as much as 47 percent of a standard deviation after two years—when compared to students of the same age....

Pages