Standards, Testing & Accountability

On accountability

Atlanta cheating, Eva’s Success Academies, poverty and brain science, and measuring Common Core’s effects. 

Amber's Research Minute

 

Michelle:         Hello, this is your host Michelle Lerner of the Thomas B. Fordham Institute here at the Education Gadfly Show and online at edexcellence.net. Now please join me in welcoming my co-host, the coach K of ed reform, Robert Pondiscio.

Robert:            A Blue Devil at heart.

Michelle:         A Blue Devil dad maybe.

Robert:            Maybe. My daughter just went to look at Duke last week among other schools and I'm not going to speak for her, but I think she liked it. What's not to like? It's a beautiful school and one great basketball team.

Michelle:         Are you a fan of Duke or a hater of Duke?

Robert:            You know, I've got to be honest when you grow up as I did in the New York area college sports is just not the thing that it is pretty much everywhere else ...

Michelle:         Everywhere else.

Robert:            That's not New York. I'm coming to love college sports because of my daughter. No, I only followed the pro game, although Duke could probably beat the New York Knicks easily they're pretty lousy right now.

Michelle:         Well my brother is a huge duke fan and he is very happy today but he's also starting a family feud. My husband, a huge Louisville Cardinals fan and yes, my brother text mean things to Daniel every time the Cards lose and Duke wins. Which happened more often this season.

Robert:            Last year wasn't it the other way around, Louisville won, or two years ago.

Michelle:         Yeah, but when you start a family feud you don't really think about history.

Robert:            That's exactly right, gives you something to argue about at Holidays.

Michelle:         Yeah because we don't have enough.

Robert:            Don't worry you'll make more.

Michelle:         On that note let's play Pardon the Gadfly.

Ellen:               A group of Atlanta educators were just convicted of RICO violations for their role in the cities cheating scandal. What does this mean for test-based accountability?

Robert:            Nothing good. I love accountability as much as the next guy and tests are important. If it weren't for testing some schools, in particular schools that serve low-income, kids of color would just not be getting the attention and the oxygen that they would be otherwise. Did we really want to see it come to this, with elementary teachers convicted of racketeering, facing 20 years?

Michelle:         Just like those mobsters.

Robert:            Oh, man. Is it just me Michelle or does this just feel like a bridge too far?

Michelle:         Well I think there's the whole communications aspect of hauling away teachers in handcuffs is just not a good visual. I mean they did something wrong and they have now been convicted of that I don't think anyone saying they shouldn't have ...

Robert:            Absolutely.

Michelle:         Been punished. I do you think the fact that we've gotten here is worth considering. Did you read Chad Alderman's piece on Campbell's Law on this topic?

Robert:            No I did not.

Michelle:         He's basically arguing we can't just say this is Campbell's Law, that if you have any sort of accountability system people are going to find a way to game it.

Robert:            Sure.

Michelle:         I don't think that's the answer. I think that's the answer if you don't like accountability.

Robert:            Yeah, I think that's right and I've talked about it here and elsewhere. I've had my dark night of the soul about accountability. The tests we have, they work, they’re fair, they do what they're supposed to do and they're deeply unpopular. They could even threaten the entire edifice of reform. If you like. On the other hand the methods that are more popular things like portfolio assessment, performance authentic etc. etc. They are so easily game-able that if you're going to have Campbell's law about anything you're going to have it there. I'm no smarter about this than I ever have been. I simply don't know how you square this circle. We don't like test and they lead people to do bad things and the alternative methods are just squishy and insubstantial.

Michelle:         I think testing is an imperfect measure but it's the best measure we have and I'm saying this as somebody who's not great on standardized testing, who thinks I can be judged better elsewhere but testing is what we can use. I think it's worth using, I think we should not abandon testing.

Robert:            No, no, no and I'm not thinking we should either but it just really gives me pause to see again this idea that we're now seeing criminal convictions prosecutions of racketeer elementary school teachers in handcuffs, facing twenty years of hard time. I just don't think that this is a good thing for reform and a good thing, it's the type of thing that could sap the political will and energy around and ed reform.

Michelle:         Unfortunately I think you're right. Ellen, question number two.

Ellen:               A New York Times investigation of Eva Moskowitz Success Academies unearthed some polarizing practices. Are you a fan of Eva's approach?

Robert:            Wow what a great question. It's complicated; Michelle and I were talking about this earlier. That every good conversation about education sooner or later gets to, it's complicated. Let's just get right to it in the beginning, it's complicated. Yeah, I am and I'm never going to be able to forget the experience that I had teaching in a frankly chaotic South Bronx education school for five years. Once you've had that experience and you see the damage that it does to student learning and engagement you can't not look at what Eva Moskowitz is doing with Success Academy and think okay this is a good thing.

Is she polarizing? You bet you she is polarizing. She doesn't apologize for running her schools this way and lord look at those results. The test scores that they put up there are just jaw on the floor, knock your fillings out spectacular. The other expo exculpatory thing I guess you would say is look at the waiting list. I don't have the data in front of me but I think they have, for every seat there's ten people who want a seat. Whether or not we in education get and appreciation what Eva is doing, parents sure seem to.

Michelle:         I think we have to remember that Success Academy exist in a system of choice. I could understand being opposed to some of these methods for your own kid so don't send your kid to Success Academy or if this was every single public school and it didn't matter what parents wanted, this is how your child is going to be educated. I can understand being opposed to that. Guess what if you don't like it you don't have to send your kid there.

Robert:            Sure, I think this does; it's another one of those things that speak to the inevitability of choice. On the one hand would I want to see some of the practices at a Success Academy be standardized and used as standard practice? Probably not. Do I think that parents should be denied the right to send their child to one of her schools? Of course not. You know what interesting, I actually just got off the phone with Eva, not more than a few minutes ago because I'm probably going write something about this. What's interesting when you talk to her about this and about the New York Times piece, she was a little irritated, I guess I can fairly say, about what didn't make it into the piece.

Michelle:         Of course.

Robert:            In other words she ... Look I've been to her school so I think she's right about this. They do dance, they do debate, they do art, they do hands-on science etc. etc. none of that made it to piece, it was all about Eva and the stress she puts on kids and the testing.

Michelle:         In reading the piece I was reminded of some of the things that I experience in my K-12 education, which I went to a private prep school. We had to wear uniforms; we had to stand up when adults walked into the room to show our respect. It was a very, very traditional school.

Robert:            Look how you turned out.

Michelle:         Look how I turned out. It was also an all-white upper middle-class school. I don't think that we should deny this classic, not discipline light education to kids in Harlem. I think there is something positive about a very old-school view of schools and high expectations and I think parents in Harlem, parents everywhere should have the right to opt into this and they have.

Robert:            Make no mistake they want it. I've made a joke about this, my father was forever threatening to send me to military school and I wish he had sometimes. I could have use a little bit of that discipline especially when I was 14 or 15 years old.

Michelle:         Look how well you turned out.

Robert:            Not as well as you Michelle. I think in the end this piece forget success Academy, fan or not a fan about this piece. I think whether you like Eve or not she has become, I think it's fair to say the most polarizing figure in education now. A piece like this that Kate Taylor of the New York Times wrote it's a bit of roar shock test. You tell me what you think about Eva, I will tell you what you think about this piece.

Michelle:         You're right on there and the other thing I kept thinking about as I read the piece was in so many industries we value we really hold up hard work. Whether it's the military and how hard they train, whether it's these movies about inner-city schools and how hard the teacher worked. All of these industries we said yes we praise hard work, we praise discipline, we praise all of this stuff and yet there's such a push back.

Robert:            This is what it looks like folks.

Michelle:         Yup. All right on that note question number three.

Ellen:               Nature's Neuroscience journal published a study that tried to link family income and parental education to the surface area of children's brains. Thoughts?

Robert:            Wow I'm not a neuroscientist and I don't play one on television. I'm not just a little bit but I'm completely as to how to respond to this. It does remind me of a study that I read a few years ago that I think was in the journal of pediatrics that really talked at great length about toxic stress and how it changed the formation of the brain. It really for me changed the way I thought about educating children from low income communities. Not that this is the pure and exclusive province of low income kids mind you but they're all kinds of stressors that are associated with low income kids and everything from parental neglect to food scarcity etc. If you put small child under enough of these stressors or stress conditions, it can physically change the brain and make it that much more difficult to develop cognitively. I wonder if this is not of a piece with that.

Michelle:         Neerav Kingsland blogged about this over on his blog and he's talking about what we should do the taxes, should we redistribute income. I think this falls into everything with education, where if it were just a matter of giving money that would be easy and we could solve it and go home. It's not just a question of money that would be easy, we can just tax more.

Robert:            We could just give everybody a high school diploma at birth and then we'll solve the graduation problem.

Michelle:         Exactly. I think this speaks to the struggles we have across the ed reform movement of how to talk about social mobility, how to do reform. I think on one hand you have the Eva Moskowitz's out there who are, let's just work really hard and high discipline and all this stuff versus the question of can education really solve poverty?

Robert:            There was a terrific piece from your Alma mater George Mason, I believe Tyler Cowen is his name, an economist who wrote a piece in the New York Times the other day, which I would encourage everyone to read, that said it's not the inequity, it's the inequality. That's exactly right it's not inequality it's the mobility that we shouldn't be trying to solve. It sounds like what Neroff is trying to solve is the inequality problem, what we should be trying to solve is the mobility problem.

Michelle:         I think that's a really interesting point. What I would like ed reforms to consider more are the cognitive sciences. I think this study could help us with that. I think we sit here and praise Tim Shanahan all the time on the reading and the knowledge. I think ed reformers would be wise to look at the science of brains and science of how kids learn to ensure that our ed reform policies are not speaking in talking points and are actually pushing forward reforms that can work to get more kids out of poverty.

Robert:            Now that you said that let me do exactly what you said shouldn't do, which is I'll make a broader over generalization. I think that it's fair to say that we in education tend to be driven more by philosophies than science. Is that a fair thing to say?

Michelle:         I think it's fair and on that depressing note that's all the time we have Pardon the Gadfly Show. Thank you Ellen. Up next is everyone's favorite, Amber's research minute.

Michelle:         Welcome to the show Amber.

Amber:            Thank you Michelle.

Michelle:         Did you root for Duke or did you not root for Duke?

Amber:            It's terrible because I taught in North Carolina. Our family has a house in the Outer Banks of North Carolina but I was not rooting for Duke and I’m a southerner. I always go for the underdog and I just felt like what's it 1941 since Wisconsin had won and I just thought I want to root for them so I did and my husband was rooting for Duke and it made for interesting yelling at the TV.

Robert:            There you go. Wisconsin's a likable team. I will say that.

Michelle:         They beat Kentucky so I was happy.

Amber:            Yeah that's right. You know Duke is Duke, they're not going to put up a fight every single time.

Michelle:         How did you do in the office brackets?

Amber:            I did not have much luck. I unfortunately did not fill out the brackets this year; I just missed the deadline somehow. It made it a lot less fun. I'm definitely, next year I'm doing it again.

Michelle:         I feel the opposite I was like it's not as fun now that I'm not going to win.

Robert:            I was out basically in the round of 64, done, done right away.

Michelle:         I don't like how we set it up that you got more points if you picked an underdog. I think Brandon rigged it. I'm just going to out there on the air.

Amber:            Who won by the way?

Michelle:         Our producer Liz.

Robert:            Our Irish national, non-American won the bracket. That shows you how good we are at forecasting basketball here at Fordham.

Michelle:         All right what do you have for us today?

Amber:            All right the Brownson report came out recently. It's a trio of studies but I'm just going to talk about one. This is the study about Tom Loveless every year; it's called Measuring the Effects of Common Core. Obviously my ears perked on that one. He creates two indices of Common Core implementation by using data from two surveys of state education agencies. One is based on a 2011 survey that reports how many activities, so did the state conduct PD, did they adopt new instructional material, stuff like that. That states have under taken while implementing this CCS and he basically said strong states are those that have pursued at least three of these things to implement the Common Core.

                        Then he uses another index on the 2013 survey data that asks states when they plan to have implemented Common Core and he basically says strong states are those that indicate full implementation by 2012, 2013, so it's just a little wonky stuff. He analyzes the relationship between the survey data, I just told you about and innate data and he finds that from 2009 to 2013 strong implementer outscored the four states that did not adopt Common Core by little more than a scale point. Again the small, and he says this, he's fair about it, he says that the small comparison group of just four states makes it so the finding are just less reliable because they're just more sensitive to fluctuations.

On the 2013 index there was a difference of 1.518 points between the strong implementers and the non-adopters which is obviously also pretty small. There's that but what's really interesting that I don't think I saw as picked up on in the press is this little, more interesting than a correlation study. Is that he did this sort of dive deep into how teachers reported they were teaching fiction. He found that fourth grade teachers, again with strong implementation states favorite to use a fiction over non-fiction, which is what we would expect. In 2009 and 2011 but when you looked 2013 you saw this huge decline, like 12.4 decline in percentage points.

Robert:            That's a lot.

Amber:            Teachers were basically moving from more fiction to more non-fiction.

Robert:            It's working. It's working.

Amber:            This is even better right. Another little interesting factoid that on the bottom line is this non-adoption states, they had a decline too of 9.8 percentage points.

Robert:            In the amount of fiction.

Amber:            Yeah, from 2009 to 11 which you sit there and think okay so one might take away that Common Core is actually having an instructional impact regardless of whether states officially adopted the Common Core or not.

Robert:            That makes sense to me actually. Because I think one of the big messages around Common Core was, news alert kids need more non-fiction and I think that benefited the field at-large.

Amber:            Yeah there was a bleed over affect if you will.

Robert:            That's also not surprising especially if there's going to be Common Core aligned curriculum and textbooks out there. A fourth-grade textbook will have more informational ...

Amber:            People don't want to own that they're adopting Common Core anyway, they're doing it but they're whatever.

Robert:            Pay no attention to the standards behind the curtain.

Amber:            Yes but anyway I thought at the end one other point that he made just for researchers is going to be exceedingly difficult to figure out a reliable measure on state implementation because the stuff is all so fluid. You've got states that are saying I'm in, I'm out, I'm delayed, I'm paused whatever.

Robert:            You know why I'm not sure he's right about that?

Amber:            Why?

Robert:            You were a teacher, I was a teacher what's the most powerful driver of your instructional decisions?

Amber:            Me.

Robert:            Really? For those of us who are not superstars, the tests right? I tend to think that as both PARCC and smarter balance gain traction and teachers learn how to teach to those tests I think you'll probably see those affects.

Amber:            I think one thing that's the survey data, let's recall who filled out the state education survey data, it's the state education department. Some official, right?

Robert:            Yeah.

Amber:            One person filling this thing out for the entire state. We know, what does state implementation even mean?

Robert:            Give that form to the intern.

Amber:            Yeah, anyway I think it's just going be really tough to measure this stuff and it doesn't mean we shouldn't try because we should. I think it was a sensible effort. It's just going to get really, really muddy.

Robert:            Tom Loveless to be fair has not been a fan of the Common Core so for him to write this is saying something as well.

Amber:            I think and you said something similar Robert another piece of research, folks can look at this and say oh my gosh this is just terrible, this is disappointing or folks can look at it and say hey this is kind of promising it's heading in the right direction.

Michelle:         You mean people are going to spin it both sides however they want it. I'm shocked.

Amber:            Yeah you're shocked. That's what it is.

Michelle:         That's all the time we have for this week's Gadfly Show till next week.

Robert:            I'm Robert Pondiscio.

Michelle:         I'm Michelle Lerner for the Thomas B. Fordham Institute signing off. 

 

Rick Hess opens his book, The Same Thing Over and Over, by asking readers to imagine the following scenario:

How would you respond if asked for a plan to transform America’s schools into a world-class, twenty-first century system?

Then imagine that there is one condition: you must retain the job descriptions, governance arrangements, management practices, compensation strategies, licensure requirements, and calendar of the existing system.

Hopefully, you would flee just as fast as you possibly could.

Red tape stifles innovation, dynamism, and entrepreneurship in public schooling, while creating a culture of risk aversion and defensiveness. These latter two are hardly the features of nimble organizations that can adapt to a changing world; rather, they are the marks of decaying institutions.

Here in Ohio, state leaders are taking note. On several occasions, both Governor John Kasich and Senate President Keith Faber have expressed their desire to “deregulate” public education. That is great news. Yet the task of deregulation is not a simple one. It requires carefully distinguishing the areas where the state has a valid regulatory role from those where it should defer to local, on-the-ground decision making.

The regulatory framework that we at Fordham have advocated...

This post has been updated with the full text of "A troubling verdict."

This is how it starts: You work with these kids all year. You teach them how to do fractions or find the main idea. They struggle; they make mistakes. They get it. They forget it. You keep at it. Some days you go home with tire tracks on your back, but you come back the next day. They’re your kids, even the ones who push your buttons. Especially them.

On test day, you look over their shoulders while proctoring. You cringe. A careless mistake. Another one. You know they know this stuff. You’ve been over it enough. The one kid, he’s bright enough, but unfocused. Always rushing; always has to be done first. Use the remaining time to check your answers, you suggest. “I did,” he says.

Your finger comes to rest on his answer sheet. "Check this one."

This is how it ends: In an Atlanta courtroom, with eleven educators convicted of criminal charges in a cheating scandal dating back to 2001. Forty-four schools, 180 educators, thirty-five indictments. The ones convicted Wednesday face up to twenty years in prison. They were all found guilty under...

In his proposed budget , Governor John Kasich calls for the creation of a competency-based education pilot program. Competency-based education is premised on the idea that students only move on to more complex concepts and skills after they master simpler ones. While that sounds somewhat negative at first blush, it also means that mastering current content quickly leads to advancing sooner than the standard march from grade to grade. Kasich’s proposal would provide grants to ten districts or schools that were selected through an application process created by the Ohio Department of Education to pilot the program.

The competency-based model goes by different names in different places. In Ohio, there are schools that already utilize it but call it something different: mastery grading. (Be sure to check out how schools like Metro Early College School and MC²STEM high school, as well as districts like Pickerington, make it work.) Mastery grading assesses students based on whether or not they’ve mastered specific skills and concepts. Instead of an overall grade that takes homework completion, daily assignments, class participation, and test grades that cover multiple standards into account to formulate an average, mastery grading breaks down a student’s...

It may not be obvious at first blush, but the political fight happening in New York right now over teacher evaluations has implications for Ohio. Governor Cuomo has proposed increasing the weight of a student’s test scores to 50 percent of a teacher’s evaluation, made possible by a proposed decrease in the weight of a principal’s observations. Ohio Governor John Kasich hasn’t proposed any significant changes to teacher evaluations this year, but consider this: both Ohio and New York do a poor job of objectively evaluating teachers  who don’t have grade- and subject-specific assessments, both states allow the unfair option of shared attribution, and stakeholders in each are questioning whether teacher evaluations give rise to extra hours of assessments that aren’t meaningful for students. This leads to a big question: Is there a way to fix these problems?    

Enter Educators 4 Excellence  (E4E) and their alternative teacher evaluation framework. E4E is an organization comprised of former and current teachers. Its mission is to magnify teacher voices in policy and legislative arenas where educator views are often overlooked—despite the fact that ensuing decisions significantly impact the day-to-day lives of teachers. E4E supports teacher evaluations...

There used to be a wry and mildly provocative blog called “Stuff White People Like.” Briefly popular in its heyday, it was described by the New Republic as a “piquant satire of white liberal cultural mores and hypocrisies.” The site’s creator stopped updating it a few years back after landing a book deal. But if it were still active, “opting out of tests” might have been right up there with craft beer, farmers’ markets, NPR, and Wes Anderson movies on that list of mores. Maybe hypocrisies, too.

A list compiled by the teachers’ union in New Jersey, where PARCC testing began earlier this month, claims that there have been more than thirty-five thousand test refusals statewide. On the order of one million young New Jerseyans are supposed to take the test, yet the state data documenting how many of them opted out won’t be available for at least a month. An informal analysis of the New Jersey Education Association (NJEA)’s list, however, shows that the highest numbers of test refusals are concentrated in communities that are affluent, left-leaning, and heavily white. 

A blue state with a Republican governor, New Jersey features a mix of affluent suburbs...

Last week, I complained that Eva Moskowitz and other reformers weren’t being fair when they described schools as “persistently failing” because they didn’t get many of their students to the ambitious levels built into the Common Core. This is how I concluded:

The move to higher standards means that we need to recalibrate our rhetoric and, more importantly, our approach to school accountability. In the low-standards days, it was perfectly legitimate to call out schools that couldn’t get all or most of their students to minimal levels of literacy and numeracy. It simply doesn’t work to similarly defame schools that don’t get all of their students “on track for college and career.” It’s a much higher bar and a much longer road.

But reform critics aren’t any better when it comes to playing games with the new standards. Diane Ravitch and Valerie Strauss, for example, continue to peddle the notion that the Common Core is developmentally inappropriate because it expects all students to be able to read simple passages by the end of kindergarten. Perhaps without knowing it, they’re making the same mistake as Moskowitz and...

Since its birth in 1990, Teach For America (TFA) has been one of the most scrutinized education reform programs on record. Not without reason: TFA takes a bold, innovative approach to teacher selection and preparation. Instead of having aspiring teachers slog through the conventional education school coursework before setting foot in a classroom, TFA recruits young people from selective universities, provides a five-week training program, and places them in high-need schools, including in Northeast and Southwest Ohio. The research evidence on TFA teachers’ impact has been mainly positive—particularly in math in the higher grades. But somewhat less known is the impact of TFA in the earlier grades. This study analyzes TFA teachers’ effectiveness in grades PK–5, employing “gold standard,” random-assignment methodology. Researchers randomly assigned 2,153 students to 156 teachers—sixty-six TFA and ninety comparison teachers—in thirty-six high-poverty schools, most of which were located in the urban South. The study compares students’ reading and math outcomes from the 2012–13 school year along the Woodcock-Johnson III achievement test for grades PK–2 and state tests for grades 3–5. The main finding: Across grades PK–5, no differences in average math and reading outcomes were detected between students taught by a...

The National Conference of State Legislatures has put together a nice primer on accountability for private school choice programs. Twenty-three states, one Colorado school district, and the District of Columbia presently have such programs, including “traditional” tuition vouchers, education savings accounts, scholarship tax credits, and personal tax credits or deductions. Accountability requirements for schools participating in such programs vary. Most states require: 1) a measure of school quality (whether via student assessment data or outside accreditation), 2) determination of financial strength and sustainability, and 3) meeting minimum seat-time requirements. Once private schools are permitted to accept voucher students and public dollars begins to flow, the gamut of accountability measures—and the consequences of failing to meet them—broadens. Programs can differ by testing requirements for students (same-state assessments as their public school peers or tests of their own choosing), how and to whom test results are reported, whether outside accreditation can substitute for testing, and the level and timing of sanctions related to low performance. NCSL’s report provides an overview of the varying ways these accountability measures function in Louisiana, Indiana, and Wisconsin. While Ohio is not spotlighted, it could have been. Ohio law has some meaningful accountability built into its private...

Discussion of charter schools is everywhere in the Ohio news. Everyone has an angle, including a few unexpected ones:

Pages