Standards, Testing & Accountability

The University of Kentucky may have lost the NCAA tournament, but Kentuckians can still take heart in their K–12 schools’ promising non-athletic gains. According to this new report, the Bluegrass State’s ACT scores have shot up since it began to implement the Common Core in 2011–12.

Using data from the Kentucky Department of Education, the study compared ACT scores for three cohorts of students who entered eighth grade between the 2007–08 and 2009–10 school years. The first group took the ACT—a state requirement for all eleventh graders—in 2010–11, immediately prior to CCSS implementation. They were therefore not formally exposed to instruction under the new standards. Cohorts two and three took the ACT in 2011–11 and 2012–13, after the introduction of CCSS-aligned curricula. They earned composite scores that were 0.18 and 0.25 points higher, respectively, relative to first cohort. The study authors report this gain as roughly equivalent to three months of additional learning.

The report rightly cautions against reading too much into these early findings. The short interval between Common Core implementation and the cohorts’ ACT scores reduces the effect the standards could have on student achievement. The authors also note that it is not clear whether the scoring gains...

The process of reforming charter school law in Ohio took another big step forward last week with the introduction of S.B. 148 in the Ohio Senate. Jointly sponsored by Senator Peggy Lehner (R-Kettering) and Senator Tom Sawyer (D-Akron), the bill is the result of workgroup sessions over the last nine months to craft the best legislation possible to improve charter school oversight and accountability.

The new Senate bill follows on the heels of House Bill 2, a strong charter school reform measure passed by the House last month. The Senate proposal maintains many of the critical provisions that the House bill included and adds some additional measures. Specifically, the Senate bill:

  • Strengthens House language around sponsor hopping
  • Increases transparency around expenditures by operators
  • Requires all sponsors to have a contract with the Ohio Department of Education
  • Incorporates much of Governor Kasich’s proposal related to charter school sponsor oversight
  • Prohibits sponsors from spending charter funds outside of their statutory responsibilities
  • Assists high-performing charter schools with facilities by encouraging co-location and providing some facility funding

We published a full roundup of press coverage of the rollout in a special edition of Gadfly Bites on April 16. Important highlights can be...

This post has been updated with the full text of "Wanna opt out of tests? Try this instead"

There’s a bracing moment early in the 1991 movie Grand Canyon. A tow truck driver played by Danny Glover miraculously appears to rescue a stranded motorist played by Kevin Kline, who is being terrorized by thugs on a deserted Los Angeles street. Glover’s character appears, calmly hooks up the disabled car to his rig, and appeals to the gun-toting gang leader to let him and Kline go on their way.

“I'm gonna grant you that favor, but tell me this,” the gang leader says after a tense standoff, reminding the tow truck operator that he’s calling the shots. “Are you asking me as a sign of respect? Or are you asking because I've got the gun?”

“You ain't got the gun,” Glover replies, “we ain't having this conversation.”

I think of this scene every time I read a story about the “opt-out movement”—parents and others protesting the distorting effects of standardized testing in schools by refusing to let their children take the tests. Opt-out parents believe they have a gun pointed at testing. They might be right. But the opt-out movement...

In a previous post, I referred to New York’s fierce political battle over teacher evaluations. Since then, New York lawmakers have passed the education portion of the budget—and moved Governor Cuomo’s controversial teacher evaluation proposal forward. State teachers’ unions responded by calling for parents to opt-out of standardized tests, hoping that a lack of data would sabotage the system. In response, the Brookings Institution’s Matthew Chingos has published an analysis of whether opting out will actually affect teacher evaluations. The short answer is “no,” and here’s why:

To conduct his analysis, Chingos examined statewide data from North Carolina—specifically, the math achievement of fourth and fifth graders during the 2009–10 school year. Chingos ran two simulations of the data: one that investigated a random group of students opting out of state exams, and another that investigated a group of the highest-performing students opting out. Both simulations found that the effect of opt-outs on a teacher’s evaluation score is small unless a large number of her students choose to opt out.

So what happens if a large number of students in New York opt out?[1] As the number of students opting out increases, so...

Part II of the latest Brown Center report is called “Measuring Effects of the Common Core.” Loveless creates two indexes of Common Core State Standards implementation by using data from two surveys of state education agencies. The 2011 index is based on a survey from that year, which reports how many activities—such as conducting professional development or adopting new instructional materials—states had undertaken while implementing the CCSS. “Strong” states are those that pursued at least three implementation strategies. The 2013 index uses survey data asking state officials when they plan to complete CCSS implementation. In this case, “strong” indicates full implementation by 2012–2013.

Analyzing the relationship between survey results and fourth-grade NAEP data for reading, Loveless finds little difference between “strong” states and the four states that never adopted Common Core. According to the 2011 index, strong implementers outscored the four states that didn’t adopt the Common Core by a little more than a scale point between 2009 and 13 (yet the small comparison group makes for less reliable findings). Strong states did a bit better relative to the 2013 index, but still outdid non-implementers by less than two NAEP points.

More interesting than these preliminary correlation studies, however, is...

On accountability

Atlanta cheating, Eva’s Success Academies, poverty and brain science, and measuring Common Core’s effects. 

Amber's Research Minute


Michelle:         Hello, this is your host Michelle Lerner of the Thomas B. Fordham Institute here at the Education Gadfly Show and online at Now please join me in welcoming my co-host, the coach K of ed reform, Robert Pondiscio.

Robert:            A Blue Devil at heart.

Michelle:         A Blue Devil dad maybe.

Robert:            Maybe. My daughter just went to look at Duke last week among other schools and I'm not going to speak for her, but I think she liked it. What's not to like? It's a beautiful school and one great basketball team.

Michelle:         Are you a fan of Duke or a hater of Duke?

Robert:            You know, I've got to be honest when you grow up as I did in the New York area college sports is just not the thing that it is pretty much everywhere else ...

Michelle:         Everywhere else.

Robert:            That's not New York. I'm coming to love college sports because of my daughter. No, I only followed the pro game, although Duke could probably beat the New York Knicks easily they're pretty lousy right now.

Michelle:         Well my brother is a huge duke fan and he is very happy today but he's also starting a family feud. My husband, a huge Louisville Cardinals fan and yes, my brother text mean things to Daniel every time the Cards lose and Duke wins. Which happened more often this season.

Robert:            Last year wasn't it the other way around, Louisville won, or two years ago.

Michelle:         Yeah, but when you start a family feud you don't really think about history.

Robert:            That's exactly right, gives you something to argue about at Holidays.

Michelle:         Yeah because we don't have enough.

Robert:            Don't worry you'll make more.

Michelle:         On that note let's play Pardon the Gadfly.

Ellen:               A group of Atlanta educators were just convicted of RICO violations for their role in the cities cheating scandal. What does this mean for test-based accountability?

Robert:            Nothing good. I love accountability as much as the next guy and tests are important. If it weren't for testing some schools, in particular schools that serve low-income, kids of color would just not be getting the attention and the oxygen that they would be otherwise. Did we really want to see it come to this, with elementary teachers convicted of racketeering, facing 20 years?

Michelle:         Just like those mobsters.

Robert:            Oh, man. Is it just me Michelle or does this just feel like a bridge too far?

Michelle:         Well I think there's the whole communications aspect of hauling away teachers in handcuffs is just not a good visual. I mean they did something wrong and they have now been convicted of that I don't think anyone saying they shouldn't have ...

Robert:            Absolutely.

Michelle:         Been punished. I do you think the fact that we've gotten here is worth considering. Did you read Chad Alderman's piece on Campbell's Law on this topic?

Robert:            No I did not.

Michelle:         He's basically arguing we can't just say this is Campbell's Law, that if you have any sort of accountability system people are going to find a way to game it.

Robert:            Sure.

Michelle:         I don't think that's the answer. I think that's the answer if you don't like accountability.

Robert:            Yeah, I think that's right and I've talked about it here and elsewhere. I've had my dark night of the soul about accountability. The tests we have, they work, they’re fair, they do what they're supposed to do and they're deeply unpopular. They could even threaten the entire edifice of reform. If you like. On the other hand the methods that are more popular things like portfolio assessment, performance authentic etc. etc. They are so easily game-able that if you're going to have Campbell's law about anything you're going to have it there. I'm no smarter about this than I ever have been. I simply don't know how you square this circle. We don't like test and they lead people to do bad things and the alternative methods are just squishy and insubstantial.

Michelle:         I think testing is an imperfect measure but it's the best measure we have and I'm saying this as somebody who's not great on standardized testing, who thinks I can be judged better elsewhere but testing is what we can use. I think it's worth using, I think we should not abandon testing.

Robert:            No, no, no and I'm not thinking we should either but it just really gives me pause to see again this idea that we're now seeing criminal convictions prosecutions of racketeer elementary school teachers in handcuffs, facing twenty years of hard time. I just don't think that this is a good thing for reform and a good thing, it's the type of thing that could sap the political will and energy around and ed reform.

Michelle:         Unfortunately I think you're right. Ellen, question number two.

Ellen:               A New York Times investigation of Eva Moskowitz Success Academies unearthed some polarizing practices. Are you a fan of Eva's approach?

Robert:            Wow what a great question. It's complicated; Michelle and I were talking about this earlier. That every good conversation about education sooner or later gets to, it's complicated. Let's just get right to it in the beginning, it's complicated. Yeah, I am and I'm never going to be able to forget the experience that I had teaching in a frankly chaotic South Bronx education school for five years. Once you've had that experience and you see the damage that it does to student learning and engagement you can't not look at what Eva Moskowitz is doing with Success Academy and think okay this is a good thing.

Is she polarizing? You bet you she is polarizing. She doesn't apologize for running her schools this way and lord look at those results. The test scores that they put up there are just jaw on the floor, knock your fillings out spectacular. The other expo exculpatory thing I guess you would say is look at the waiting list. I don't have the data in front of me but I think they have, for every seat there's ten people who want a seat. Whether or not we in education get and appreciation what Eva is doing, parents sure seem to.

Michelle:         I think we have to remember that Success Academy exist in a system of choice. I could understand being opposed to some of these methods for your own kid so don't send your kid to Success Academy or if this was every single public school and it didn't matter what parents wanted, this is how your child is going to be educated. I can understand being opposed to that. Guess what if you don't like it you don't have to send your kid there.

Robert:            Sure, I think this does; it's another one of those things that speak to the inevitability of choice. On the one hand would I want to see some of the practices at a Success Academy be standardized and used as standard practice? Probably not. Do I think that parents should be denied the right to send their child to one of her schools? Of course not. You know what interesting, I actually just got off the phone with Eva, not more than a few minutes ago because I'm probably going write something about this. What's interesting when you talk to her about this and about the New York Times piece, she was a little irritated, I guess I can fairly say, about what didn't make it into the piece.

Michelle:         Of course.

Robert:            In other words she ... Look I've been to her school so I think she's right about this. They do dance, they do debate, they do art, they do hands-on science etc. etc. none of that made it to piece, it was all about Eva and the stress she puts on kids and the testing.

Michelle:         In reading the piece I was reminded of some of the things that I experience in my K-12 education, which I went to a private prep school. We had to wear uniforms; we had to stand up when adults walked into the room to show our respect. It was a very, very traditional school.

Robert:            Look how you turned out.

Michelle:         Look how I turned out. It was also an all-white upper middle-class school. I don't think that we should deny this classic, not discipline light education to kids in Harlem. I think there is something positive about a very old-school view of schools and high expectations and I think parents in Harlem, parents everywhere should have the right to opt into this and they have.

Robert:            Make no mistake they want it. I've made a joke about this, my father was forever threatening to send me to military school and I wish he had sometimes. I could have use a little bit of that discipline especially when I was 14 or 15 years old.

Michelle:         Look how well you turned out.

Robert:            Not as well as you Michelle. I think in the end this piece forget success Academy, fan or not a fan about this piece. I think whether you like Eve or not she has become, I think it's fair to say the most polarizing figure in education now. A piece like this that Kate Taylor of the New York Times wrote it's a bit of roar shock test. You tell me what you think about Eva, I will tell you what you think about this piece.

Michelle:         You're right on there and the other thing I kept thinking about as I read the piece was in so many industries we value we really hold up hard work. Whether it's the military and how hard they train, whether it's these movies about inner-city schools and how hard the teacher worked. All of these industries we said yes we praise hard work, we praise discipline, we praise all of this stuff and yet there's such a push back.

Robert:            This is what it looks like folks.

Michelle:         Yup. All right on that note question number three.

Ellen:               Nature's Neuroscience journal published a study that tried to link family income and parental education to the surface area of children's brains. Thoughts?

Robert:            Wow I'm not a neuroscientist and I don't play one on television. I'm not just a little bit but I'm completely as to how to respond to this. It does remind me of a study that I read a few years ago that I think was in the journal of pediatrics that really talked at great length about toxic stress and how it changed the formation of the brain. It really for me changed the way I thought about educating children from low income communities. Not that this is the pure and exclusive province of low income kids mind you but they're all kinds of stressors that are associated with low income kids and everything from parental neglect to food scarcity etc. If you put small child under enough of these stressors or stress conditions, it can physically change the brain and make it that much more difficult to develop cognitively. I wonder if this is not of a piece with that.

Michelle:         Neerav Kingsland blogged about this over on his blog and he's talking about what we should do the taxes, should we redistribute income. I think this falls into everything with education, where if it were just a matter of giving money that would be easy and we could solve it and go home. It's not just a question of money that would be easy, we can just tax more.

Robert:            We could just give everybody a high school diploma at birth and then we'll solve the graduation problem.

Michelle:         Exactly. I think this speaks to the struggles we have across the ed reform movement of how to talk about social mobility, how to do reform. I think on one hand you have the Eva Moskowitz's out there who are, let's just work really hard and high discipline and all this stuff versus the question of can education really solve poverty?

Robert:            There was a terrific piece from your Alma mater George Mason, I believe Tyler Cowen is his name, an economist who wrote a piece in the New York Times the other day, which I would encourage everyone to read, that said it's not the inequity, it's the inequality. That's exactly right it's not inequality it's the mobility that we shouldn't be trying to solve. It sounds like what Neroff is trying to solve is the inequality problem, what we should be trying to solve is the mobility problem.

Michelle:         I think that's a really interesting point. What I would like ed reforms to consider more are the cognitive sciences. I think this study could help us with that. I think we sit here and praise Tim Shanahan all the time on the reading and the knowledge. I think ed reformers would be wise to look at the science of brains and science of how kids learn to ensure that our ed reform policies are not speaking in talking points and are actually pushing forward reforms that can work to get more kids out of poverty.

Robert:            Now that you said that let me do exactly what you said shouldn't do, which is I'll make a broader over generalization. I think that it's fair to say that we in education tend to be driven more by philosophies than science. Is that a fair thing to say?

Michelle:         I think it's fair and on that depressing note that's all the time we have Pardon the Gadfly Show. Thank you Ellen. Up next is everyone's favorite, Amber's research minute.

Michelle:         Welcome to the show Amber.

Amber:            Thank you Michelle.

Michelle:         Did you root for Duke or did you not root for Duke?

Amber:            It's terrible because I taught in North Carolina. Our family has a house in the Outer Banks of North Carolina but I was not rooting for Duke and I’m a southerner. I always go for the underdog and I just felt like what's it 1941 since Wisconsin had won and I just thought I want to root for them so I did and my husband was rooting for Duke and it made for interesting yelling at the TV.

Robert:            There you go. Wisconsin's a likable team. I will say that.

Michelle:         They beat Kentucky so I was happy.

Amber:            Yeah that's right. You know Duke is Duke, they're not going to put up a fight every single time.

Michelle:         How did you do in the office brackets?

Amber:            I did not have much luck. I unfortunately did not fill out the brackets this year; I just missed the deadline somehow. It made it a lot less fun. I'm definitely, next year I'm doing it again.

Michelle:         I feel the opposite I was like it's not as fun now that I'm not going to win.

Robert:            I was out basically in the round of 64, done, done right away.

Michelle:         I don't like how we set it up that you got more points if you picked an underdog. I think Brandon rigged it. I'm just going to out there on the air.

Amber:            Who won by the way?

Michelle:         Our producer Liz.

Robert:            Our Irish national, non-American won the bracket. That shows you how good we are at forecasting basketball here at Fordham.

Michelle:         All right what do you have for us today?

Amber:            All right the Brownson report came out recently. It's a trio of studies but I'm just going to talk about one. This is the study about Tom Loveless every year; it's called Measuring the Effects of Common Core. Obviously my ears perked on that one. He creates two indices of Common Core implementation by using data from two surveys of state education agencies. One is based on a 2011 survey that reports how many activities, so did the state conduct PD, did they adopt new instructional material, stuff like that. That states have under taken while implementing this CCS and he basically said strong states are those that have pursued at least three of these things to implement the Common Core.

                        Then he uses another index on the 2013 survey data that asks states when they plan to have implemented Common Core and he basically says strong states are those that indicate full implementation by 2012, 2013, so it's just a little wonky stuff. He analyzes the relationship between the survey data, I just told you about and innate data and he finds that from 2009 to 2013 strong implementer outscored the four states that did not adopt Common Core by little more than a scale point. Again the small, and he says this, he's fair about it, he says that the small comparison group of just four states makes it so the finding are just less reliable because they're just more sensitive to fluctuations.

On the 2013 index there was a difference of 1.518 points between the strong implementers and the non-adopters which is obviously also pretty small. There's that but what's really interesting that I don't think I saw as picked up on in the press is this little, more interesting than a correlation study. Is that he did this sort of dive deep into how teachers reported they were teaching fiction. He found that fourth grade teachers, again with strong implementation states favorite to use a fiction over non-fiction, which is what we would expect. In 2009 and 2011 but when you looked 2013 you saw this huge decline, like 12.4 decline in percentage points.

Robert:            That's a lot.

Amber:            Teachers were basically moving from more fiction to more non-fiction.

Robert:            It's working. It's working.

Amber:            This is even better right. Another little interesting factoid that on the bottom line is this non-adoption states, they had a decline too of 9.8 percentage points.

Robert:            In the amount of fiction.

Amber:            Yeah, from 2009 to 11 which you sit there and think okay so one might take away that Common Core is actually having an instructional impact regardless of whether states officially adopted the Common Core or not.

Robert:            That makes sense to me actually. Because I think one of the big messages around Common Core was, news alert kids need more non-fiction and I think that benefited the field at-large.

Amber:            Yeah there was a bleed over affect if you will.

Robert:            That's also not surprising especially if there's going to be Common Core aligned curriculum and textbooks out there. A fourth-grade textbook will have more informational ...

Amber:            People don't want to own that they're adopting Common Core anyway, they're doing it but they're whatever.

Robert:            Pay no attention to the standards behind the curtain.

Amber:            Yes but anyway I thought at the end one other point that he made just for researchers is going to be exceedingly difficult to figure out a reliable measure on state implementation because the stuff is all so fluid. You've got states that are saying I'm in, I'm out, I'm delayed, I'm paused whatever.

Robert:            You know why I'm not sure he's right about that?

Amber:            Why?

Robert:            You were a teacher, I was a teacher what's the most powerful driver of your instructional decisions?

Amber:            Me.

Robert:            Really? For those of us who are not superstars, the tests right? I tend to think that as both PARCC and smarter balance gain traction and teachers learn how to teach to those tests I think you'll probably see those affects.

Amber:            I think one thing that's the survey data, let's recall who filled out the state education survey data, it's the state education department. Some official, right?

Robert:            Yeah.

Amber:            One person filling this thing out for the entire state. We know, what does state implementation even mean?

Robert:            Give that form to the intern.

Amber:            Yeah, anyway I think it's just going be really tough to measure this stuff and it doesn't mean we shouldn't try because we should. I think it was a sensible effort. It's just going to get really, really muddy.

Robert:            Tom Loveless to be fair has not been a fan of the Common Core so for him to write this is saying something as well.

Amber:            I think and you said something similar Robert another piece of research, folks can look at this and say oh my gosh this is just terrible, this is disappointing or folks can look at it and say hey this is kind of promising it's heading in the right direction.

Michelle:         You mean people are going to spin it both sides however they want it. I'm shocked.

Amber:            Yeah you're shocked. That's what it is.

Michelle:         That's all the time we have for this week's Gadfly Show till next week.

Robert:            I'm Robert Pondiscio.

Michelle:         I'm Michelle Lerner for the Thomas B. Fordham Institute signing off. 


Rick Hess opens his book, The Same Thing Over and Over, by asking readers to imagine the following scenario:

How would you respond if asked for a plan to transform America’s schools into a world-class, twenty-first century system?

Then imagine that there is one condition: you must retain the job descriptions, governance arrangements, management practices, compensation strategies, licensure requirements, and calendar of the existing system.

Hopefully, you would flee just as fast as you possibly could.

Red tape stifles innovation, dynamism, and entrepreneurship in public schooling, while creating a culture of risk aversion and defensiveness. These latter two are hardly the features of nimble organizations that can adapt to a changing world; rather, they are the marks of decaying institutions.

Here in Ohio, state leaders are taking note. On several occasions, both Governor John Kasich and Senate President Keith Faber have expressed their desire to “deregulate” public education. That is great news. Yet the task of deregulation is not a simple one. It requires carefully distinguishing the areas where the state has a valid regulatory role from those where it should defer to local, on-the-ground decision making.

The regulatory framework that we at Fordham have advocated...

This post has been updated with the full text of "A troubling verdict."

This is how it starts: You work with these kids all year. You teach them how to do fractions or find the main idea. They struggle; they make mistakes. They get it. They forget it. You keep at it. Some days you go home with tire tracks on your back, but you come back the next day. They’re your kids, even the ones who push your buttons. Especially them.

On test day, you look over their shoulders while proctoring. You cringe. A careless mistake. Another one. You know they know this stuff. You’ve been over it enough. The one kid, he’s bright enough, but unfocused. Always rushing; always has to be done first. Use the remaining time to check your answers, you suggest. “I did,” he says.

Your finger comes to rest on his answer sheet. "Check this one."

This is how it ends: In an Atlanta courtroom, with eleven educators convicted of criminal charges in a cheating scandal dating back to 2001. Forty-four schools, 180 educators, thirty-five indictments. The ones convicted Wednesday face up to twenty years in prison. They were all found guilty under...

In his proposed budget , Governor John Kasich calls for the creation of a competency-based education pilot program. Competency-based education is premised on the idea that students only move on to more complex concepts and skills after they master simpler ones. While that sounds somewhat negative at first blush, it also means that mastering current content quickly leads to advancing sooner than the standard march from grade to grade. Kasich’s proposal would provide grants to ten districts or schools that were selected through an application process created by the Ohio Department of Education to pilot the program.

The competency-based model goes by different names in different places. In Ohio, there are schools that already utilize it but call it something different: mastery grading. (Be sure to check out how schools like Metro Early College School and MC²STEM high school, as well as districts like Pickerington, make it work.) Mastery grading assesses students based on whether or not they’ve mastered specific skills and concepts. Instead of an overall grade that takes homework completion, daily assignments, class participation, and test grades that cover multiple standards into account to formulate an average, mastery grading breaks down a student’s...

It may not be obvious at first blush, but the political fight happening in New York right now over teacher evaluations has implications for Ohio. Governor Cuomo has proposed increasing the weight of a student’s test scores to 50 percent of a teacher’s evaluation, made possible by a proposed decrease in the weight of a principal’s observations. Ohio Governor John Kasich hasn’t proposed any significant changes to teacher evaluations this year, but consider this: both Ohio and New York do a poor job of objectively evaluating teachers  who don’t have grade- and subject-specific assessments, both states allow the unfair option of shared attribution, and stakeholders in each are questioning whether teacher evaluations give rise to extra hours of assessments that aren’t meaningful for students. This leads to a big question: Is there a way to fix these problems?    

Enter Educators 4 Excellence  (E4E) and their alternative teacher evaluation framework. E4E is an organization comprised of former and current teachers. Its mission is to magnify teacher voices in policy and legislative arenas where educator views are often overlooked—despite the fact that ensuing decisions significantly impact the day-to-day lives of teachers. E4E supports teacher evaluations...