Standards, Testing & Accountability

Over the last five years, prodded by the feds, states have adopted teacher evaluation systems. According to a recent report from the National Council on Teacher Quality, forty-one states, including Ohio, now require evaluations that include objective measures of student achievement. These aren’t the meat-axe assessments of yesteryear, though. These next-generation teacher evaluations combine classroom observations using new prescriptive protocols with quantitative evidence of learning gains on state tests (or another form of assessment) to determine each teacher’s effectiveness.

The national focus on teacher evaluations raises a couple of questions. First, why have states chosen to focus on teacher evaluations (i.e. what’s the problem that policymakers are trying to solve)? Second, are the new evaluations proving effective in solving the problem?

Let’s start with the why. Recall all the evidence that the single most important in-school factor for student achievement is teacher quality. If we know that good teachers make a difference, it's not surprising that we've focused on evaluating them. Such evaluations hold the potential to identify great teachers whom we can reward, retain, and/or hold up as models, struggling or developing teachers whom we can help to improve, and ineffective teachers who should be removed from...

The information yielded by standardized tests—and the analyses based on test results, like value-added—should form the basis for tough decisions regarding which schools (charter and district) or entire school systems require intervention. Parents need information about school quality, and taxpayers ought to know whether their resources are being put to good use. But at the same time, parents and policymakers alike have valid concerns about “overtesting” students, and how high-stakes tests change how schools behave.

Over the past decade, Ohio has tested social studies and science unevenly, and will continue to do so under the new assessment program set to begin in spring 2015. Under the old system, the state administered science tests in just grades 5 and 8, while math and English language arts (ELA) were assessed in all grades 3–8. Social studies was tested for just three years (2006–07 to 2008–09) in grades 5 and 8, but it was “suspended” effective fall 2009. The new state testing program continues science assessments in grades 5 and 8 and resurrects social studies testing in grades 4 and 6.

Should Ohio test in science and social studies, in addition to ELA and math assessments? And if...

All Hallows Edition

The testing pushback, a college boost for poor kids, adolescent readers, and school-supporting nonprofits.

Amber's Research Minute

"The Rise of School-Supporting Nonprofits," by Ashlyn Aiko Nelson and Beth Gazley, Association for Education Finance and Policy (Feburary 2014).

Morgan Polikoff

Election Day is less than a week away. Given the heat around major education policies—especially Common Core and teacher evaluations—there is increased attention to public attitudes about education. A number of polls from major news organizations, education groups, and universities have been commissioned over the past several months, and education pundits and advocates on all sides of current reform debates have endlessly parsed the results.

Unfortunately these pundits are mostly misguided, and public opinion polls on education don’t mean what people think they mean. What follows are three conclusions, all based on data from these various polls, and a discussion of what they ought to mean for education policy and advocacy going forward.

Conclusion 1: Americans’ views on education are incoherent.

The most straightforward conclusion from existing polling data is that Americans’ views are all over the map and, depending on the issue, either nuanced or contradictory. The clearest example of this is on standardized testing. The 2013 Phi Delta Kappa (PDK)/Gallup poll found that just 22 percent of the public thought that standardized tests have helped local public schools. But when asked about specific test-related policies—some of...

Jonathan Schleifer

Countries with high school exit exams appear to have higher levels of student achievement, as indicated by PISA and some positive evidence from other countries that have used graduation exams. But have they worked in the United States? A recent Education Next forum failed to ask this essential question.

When fourteen public school teachers came together as part of Educators 4 Excellence-New York Teacher Policy Team on how to improve the use of testing in schools, they were taken aback by the depth of research showing the harmful effects of exit exams, which twenty-six states have adopted in one form or another.

There are two relevant research questions: Do exit exams have beneficial effects on students in terms of achievement or labor-market outcomes? And do exit exams have negative consequences, particularly on historically disadvantaged populations of students? The answers are no and yes, respectively. Here’s a representative, though not comprehensive, review:

Studies that find no benefit

  • In 2008, researchers examined a nationally representative sample of students and found no impact on achievement of high school graduation exams for any subpopulation of students, including low achievers.
  • A 2010 study of California’s exit exams used a
  • ...

All the world's a stage - October 22, 2014

The benefits of live theater, how and whether to discipline, detrimental reading tests, and relative school costs.

Amber's Research Minute

The Relative Costs of New York City’s New Small Public High Schools of Choice,” by Robert Bifulco and Rebecca Unterman,  MRDC (October 2014).

Ohio is moving to new standardized tests, the PARCC assessments, which are set to commence in spring 2015. These new and vastly different tests pose big challenges. For one, unlike the paper-and-pencil exams of the past, the PARCC is designed for online administration, leading to obvious questions about schools’ technical readiness to administer the exams.[1] In addition, as Cleveland’s Plain Dealer reported recently, PARCC test results may be released later than usual in 2015—likely delaying the release of school report cards. At the same time, no one knows exactly where PARCC will set its cut-scores for “proficiency” and other achievement levels.[2] Finally, expect political blowback, too, when lower test scores are reported under PARCC, perhaps even stronger than the ongoing skirmishes around Ohio’s new learning standards.

Despite these complications, Ohioans should give PARCC a chance. Ohio needs a higher-quality state assessment to replace its mostly rinky-dink tests of yesteryear. Take a look at PARCC’s test-item prototypes; they ask students to demonstrate solid analytical skills based on what they know in math and English language arts. The upshot: PARCC’s more-sophisticated approach to assessment could put an end to...

In his recent State of the Schools speech, Cleveland Metropolitan School District (CMSD) CEO Eric Gordon referred to a 2013 column in the Plain Dealer comparing him to the ancient Greek king Sisyphus. As every school kid used to know, Sisyphus rolled a boulder up a mountain each day, only to watch it roll back down. He was doomed to spend the rest of eternity repeating this pointless task as a punishment for his greed and deceit—a kind of Greek myth Groundhog Day

The comparison of Gordon and Sisyphus is unfair. The punishment of Sisyphus, at its heart, is one constructed to impose hopelessness and despair. There is certainly much work to be done in Cleveland, but as we at Fordham have pointed out before (see here) there are also reasons to be hopeful about Cleveland’s progress. There is no room for Sisyphus in the fight to improve Ohio schools.

That being said, the English teacher in me appreciates the allusion. It even got me thinking about other ancient figures who might better symbolize the Buckeye state’s struggle to give its kids the best education—an education that all students deserve, but far too few receive. There’s the...

On Wednesday, CCSSO (the organization of state superintendents) joined with CGCS (the organization of big urban school districts) to announce joint plans to reassess and scale back testing programs. This is big news, and it’s getting lots of attention. Here are the ten big things to know about the announcement.

  1.  A direct response to testing concerns. These two leading organizations are clearly responding to the pressure to reduce or end testing emanating from the AFT, former President Clinton, Secretary Duncan, and others. They’re agreeing to audit the number and types of tests administered and develop new systems that are leaner (eliminating “multiple assessments of the same students for similar purposes”) and more integrated (“complement each other in a way the defines a coherent system of measures”).
  2. Won’t back down. CCSSO and CGCS, however, are standing firm on testing, and the most vociferous anti-testing forces aren’t happy about it (Randi Weingarten, for example, said the plan fails to address the fundamental problem of “test fixation”). The joint statement makes clear these leaders believe deeply in the value of smart
  3. ...

The enlightenment edition - October 15, 2014

Civil rights, Christopher Columbus, D.C. school spending, and teacher prep.

Amber's Research Minute

"Teacher Preparation Policies and Their Effects on Student Achievement," by Gary T. Henry, et al., The Association for Education Finance and Policy (2014).

Transcript

Michelle:       Hello, this is your host, Michelle Gininger of the Thomas B. Fordham Institute here at The Education Gadfly Show and online at edexcellence.net. Now, please join me in welcoming my cohost, the Alicia Florrick of education reform, Alyssa Schwenk.

Alyssa:           Oh wow. That was such an honor.

Michelle:       I knew you'd love that. I knew it.

Alyssa:           I mean it is kind of our topic of conversation every Monday or Tuesday whenever we get around to watching The Good Wife which airs on Sunday …

Michelle:       Let's be honest, it's never Monday morning because I can't stay up till 10 o'clock at night on Sunday so it's Tuesday or Wednesday or Thursday.

Alyssa:           Yeah. Usually, not on football season, I can watch it on Sunday. But on football season, forget about it.

Michelle:       Yeah. Football just ruins The Good Wife for me. I think we should move football to another day and this is the big policy I'm going to be pushing this year.

Alyssa:           Oh good to know but I mean if they were up to, I think, the two of us, television would end by 9 PM every night. We would have to stay up till 11.

Michelle:       That would be fantastic. I think we should promote that but in the meantime let's get back to Ed reform. Ellen, what do you have for us?

Ellen:              Fordham’s own Mike Petrelli recently argued that over the last six years, the Department of Education has been too involved in civil rights. Do you agree?

Michelle:       This is based on a special Op Ed that Mike had in NRO’s education week that they just had this weekend. Robert Pendisi on our team also had one on common core but basically, Mike takes another aim at the Secretary of Education. Alyssa, what's your take?

Alyssa:           Mike gave two examples in the piece of areas in which he believes that Arne Duncan has used the Office of Civil Rights to overreach over local control on several issues and the first, I agree with and the second, I do not.

Michelle:       Oh, I think we might be opposite here.

Alyssa:           Ooh, okay which one do you agree with Mike on?

Michelle:       I agree on the AP testing.

Alyssa:           So do I.

Michelle:       Oh see, I thought that was second in the article.

Alyssa:           I guess it was okay. Well one of the points I agree with and one that I do not.

Michelle:       Here's how we should solve the AP problem. Basically Mike is calling out the department because they are pushing for more minority students to be in AP classes which obviously is a good thing but the on the ground reality is that we're just going to be pushing more students into AP courses who aren't ready for AP course week. The goal should be putting more minority kids in AP courses and having the pass rate of AP courses staying the same. That way, we’re not incentivizing schools to just put kids in the classes they're not prepared for but we are incentivizing schools to get more students prepared and putting them in these AP courses.

Alyssa:           Yeah. I do worry very much about the unintended consequences of pushing unprepared kids into AP whether or not that's colleges deciding AP no longer accounts for college credit which is really important when you're a student who tuition is a big barrier for entry. I also think it's an opportunity for schools to do things like maybe course share or take online courses so kids who are in schools were not … There’s not 30 kids who are ready for AP but there's maybe 5 can still take those classes and have those opportunities. I think there's an opportunity there but the way that the office is being used, I'm a little concerned about.

Michelle:       Now, one thing that I did in … or that happened in my high school once upon a time when I was a young, young high school student was they just offered courses and then at the end of the course, you could opt in to taking the AP exam. There was an AP US history on top of US history. It was just history class and if at the end of the year, you felt prepared to take the AP exam, you could opt in to taking it which was an interesting model. I don't know how if that's necessarily possible in public schools. I did go to a private school and it was pretty small so it was easy. There wasn’t the scale issue that's one interesting thing that might be a good policy recommendation.

Alyssa:           Yeah, that would definitely I think be a solution.

Michelle:       We're not even going to talk about school discipline, sorry Mike. We're just going to go question number 2.

Ellen:              Monday was Columbus Day, a celebration of a controversial often misunderstood figure. Should schools give students a more accurate picture of who Christopher Columbus really was?

Michelle:       I know, Alyssa and I disagree on this one so that's a good thing. I am for Christopher Columbus Day it might just be because I am part Italian. What can I say? But I'm for it. Alyssa?

Alyssa:           I think that there's a better use of students’ time than … I think on American history, we have a tendency to kind of lionize a lot of historical figures who have kind of unsavory pasts. Christopher Columbus, obviously chief among them. I don't think a great use of students’ time is to be out of school. I think they should be in school learning about these figures, learning about different aspects of American history and really debating and discussing these things.

There's definitely a lot of weak history curriculum out there and I think now is the time to push that and have a chance for students to learn about these figures instead of just playing or doing God knows what that day.

Michelle:       First, don't take away any of my federal holidays even though Fordham does work on Columbus Day.

Alyssa:           I was going to say we were the office bright and early yesterday.

Michelle:       So I'm all for that. Would I want to create a Columbus Day if it didn't exist? Probably not but I have 2 strong feelings on this. One, we can't judge yesterday’s historical figures based on today's morals. Now, obviously Christopher Columbus did some horrendous things, but by those standards, it wasn't that bad. I'm not saying what he did is okay, I'm just saying, when we teach this stuff in school and when we do have a good history curriculum which we should, we should say here's what Christopher Columbus did and yet all these drawbacks and horrible things that he did.

I think that it's just turned into a political fight as opposed to a constructive conversation of how we should actually deal with historical figures that don't live up to today standards and morals. Obviously, you already mentioned Thomas Jefferson owned slaves but pretty much everyone historically did not respect women as equal people, which we do today hopefully. That is a more important conversation than one day off at school and should it be Christopher Columbus Day or another day, I think that conversation’s a little [mute 00:06:17].

Alyssa:           Yeah. All right, Ellen question number three.

Ellen:              With Fordham's new Metro DC school spending explorer, Mike Petrelli and Matt Richmond note that Arlington and Fairfax counties are spending much more on their high poverty schools than Montgomery County which prides itself on its strong commitment to social justice and Prince George's County with high levels of students in poverty. What's going on here?

Michelle:       Before I get on my high horse that Virginia is totally more awesome than Maryland.

Alyssa:           I knew you were going to be saying that.

Michelle:       I know, I know. I'm a lifelong Virginian. Let me explain what this awesome project this. Basically, we had the3 question of how much do DC area schools spend per people at the school level. Obviously, we know that there's going to be spending differences between districts. What's interesting in that portion was that PG County in Maryland spends so much less than the other districts that we studied. But also perhaps more interesting is that within districts, the funding levels are different even if you look at schools that should be on par with one another.

You're can have 2 public elementary schools in the same district that receive vastly different funding levels or spending levels, excuse me, Dara would [chide 00:07:27] me for seeing funding instead of spending. That's what the project is. It's really awesome. I encourage folks to look at our interactive map, but to the question at hand, Mike and Matt took a look at how districts spend for their highest needs students, the highest poverty schools. What they found is that for extra spending for low income students, Arlington hit it up out of the park with 81% followed by Fairfax County with 34 while Montgomery County which prides itself as Ellen noted on being social mobility friendly, not so much in MPG County was with 2%. A few caveats there, school don’t necessarily have a lot of control on the spending. Most of it is teacher salaries. Arlington only actually had 2 high poverty schools while PG County had 50 high poverty schools. That's a lot of nuance here but it's certainly really interesting.

Alyssa:           Yeah. I feel like that kind of undercuts your Virginia is for everyone and Virginia is the best argument but as a DC person, I was particularly interested in the spending differences between DC charter schools and DC public schools and noted that DC charter schools spent a bit more per pupil and this is obviously taking out the discrepancies in building and construction funding which is a huge issue inside DC but DC charter schools are spending more per high poverty pupil than the DC public schools are even though DC charter schools have incredibly high student poverty in most of them.

In terms of Prince George's, I was not super surprised having been around DC for a while. I think there, it's just such a tax-based issue and it's so hard to build up the tax base whereas Arlington and Fairfax have kind of a more affluent population that they're working with in general. I think that PG County stories of concern to the local area because as more and more poor families are being kicked out of DC because of gentrification, they're going from perhaps … They're going from one school in DC that's pretty well-funded to a school that isn't as well-funded. That's of concern.

Of course, there's no direct correlation between funding and student performance so there's a lot of nuance here but I think it's important to look at this and one of the things that I find most interesting about the project in total is that state average, that district average that is touted doesn’t tell the story and this map I think will be an eye-opener for certainly the parent advocacy contingent in the area.

Michelle:       I'm sure Mike's going to just love that. But yeah no I think the map is super cool. I spend, when I was looking at the beta version, almost an hour I'd say just clicking in, clicking out seeing all of the different categories. It's a very cool project to check out.

Alyssa:           Being a local, I got to look at what my district in public high school would have been back when I was a young teenager. That's all the time we have for Pardon the Gadfly. Thank you Ellen. Up next is everyone’s favorite, Amber's Research Minute.

                        Welcome to the show Dara.

Dara:              Thank you.

Alyssa:           Bravo on your DC Spending Explorer Map out today.

Dara:              We'll call that a labor of love.

Female:         We’ll call it a labor of something.

Dara:              Labor of something. We are super excited that we're able to share this with everyone. It's been literally months and months and months of work doing the analyses, getting the website out the door so we're super excited.

Female:         What is your favorite take away from this project? On the data, not on the process.

Dara:              Besides the fact that the way that schools account for the way that … The way that districts account for dollars spent is absolutely insane. I don't know if anyone has tried to actually read a school district expenditure report.

Female:         No.

Dara:              It's a bit nuts. That's why we did this project so you don't have to.

Michelle:       That's why we have researchers do this sort of thing.

Dara:              The biggest take-away I think is that there is predictable variation between districts. We know that districts spend the money that comes in so Montgomery County, lots of local funds, spends more per pupil than Prince George's County even though they're in the same state. They're receiving the same state revenue. That was predictable. What is really interesting is the variation between schools in the same district and that really is the result of district leaders making choices about what dollar goes to what school and so you can poke around on the website, click on each schools, see all their demographics and special education students and free and reduced lunch students and see how each schools spends its dollars.

Michelle:       Great. We won't have you research minute our own projects so what do you have for us today?

Dara:              Something completely different from that. Today, it's a study from this month's Association for Education Finance and Policy Journal from a team of researchers led by Gary Henry at Vanderbilt University. It asks a question that has already received a lot of attention in the past which is how does teacher preparation affect student achievement but this study is way more robust than any of the other research out there that examines similar questions.

One of the reasons is because of the way that it divided up teachers. Instead of lumping teachers into two groups, traditional versus alternative certification, instead it made many more nuanced comparisons which I’ll talk about in a second. The data consisted of over 22,000 North Carolina teachers in their first, second or third year of teaching and 1.18 million students.

To get the data, the authors use administrative data to get teacher characteristics, how long a teacher was teaching, how a teacher was prepared, characteristics of the school where they taught. They combined this with 5 years of student test score data. This is an incredible data set. The analysis used a value added model with school fixed effects. To answer the question how does teacher preparation affects student value added on state tests for eight combinations of grade levels and subjects. We've got Elementary Math, Elementary Reading, Middle School Math, Middle School Reading and High School Math, Science, English and Social Studies.

That was a big buildup. Here are the results. First, comparing teachers who were traditionally prepared to those who received alternative certification but not TFA. This is why this study is unique. First of all, it separated out alternative certification as in the day you step foot in the classroom, you don't have your full license. It separated those out from TFA. Traditionally prepared with non-TFA alternative certification, alternative entry teachers are significantly less effective than traditionally prepared teachers in Middle School Math and High School Math and Science but no different in any of the other subjects.

                        Second, traditionally prepared teachers compared to TFA teachers. TFA teachers are more effective in six of the eight categories, Elementary Math, Elementary Reading, Middle School Math, High School Math, Science and English. Third, comparing teachers prepared out of state versus those prepared in-state. Out-of-state teachers are less effective in Elementary Math and Reading and in High School Math.

Fourth, teachers who began teaching with a graduate degree or less effective in Middle School Math and Reading and more effective in High School Science than teachers who did not have a grad degree. Fifth and finally, no difference in any grade level or subject between in-state teachers who receive their certification at a private school versus a public school. One additional finding if that wasn't enough, the study confirmed previous research that showed that there is significant variation within different preparation categories. TFA teachers as a group first, second and third year TFA teachers more effective in 6 out of those 8 categories but within TFA teachers, there is significant variation.

Female:         That's fascinating. I'm excited about this. This is a cool study.

Dara:              I think so. Like I said, this is a question that has been asked a lot but because the researchers had such a enormous data set, they were able to make these much more nuanced categories for example not lumping together all alternatively certified teachers into one category.

Alyssa:           A few things based on what you said. Out-of-state prepared teachers performed worse … Or students performed worse than in-state? Does that go away with dare I say it, common core?

Michelle:       That was my question as well actually.

Dara:              I can only speculate because the student data, the five years of student data stopped with the 2009, 2010 school year. It's possible that if you have more … Teachers who are more familiar with the state standards and if the standards are common, that should theoretically be the case, then it is very possible that that variation could go away.

Michelle:       We'll see. Here's another question that I picked up. Exciting news on TFA or first up exciting news on TFA even though there are some variation when you look within TFA. It sounds like only in TFA were we seeing improvement in Reading and English which traditionally is so hard to get those scores up. What do we think could be the cause of that? I know again speculation.

Dara:              Right. I mean one of the things that's important to note is North Carolina is one of the original TFA focus areas. They have spent a very long time developing the infrastructure to train their teachers there. It's the same theoretical structure that the five or six week boot camp summer institute as you have with TFA everywhere else but because it is so well established in North Carolina, it's very possible that it's not just the way that TFA is recruiting its teacher but also because it's very well established, they know what they're doing.

One thing that the study didn't do is it didn't look beyond the third year of teaching. They would've had to go too far back in the data but the idea is that it's likely that preparation affects sort of fuss over once you get past the first couple of years, so take that as you will.

Michelle:       Alyssa, as a former TFAer, thoughts?

Alyssa:           I was very happy to hear that. I think I was doing a little bit of a happy dance right there. What was most interesting to me was that alternative certification teachers who were not TFA did not do so hot with Middle School Math and High School Math and Science which we know that those are traditionally hard subjects and those are areas where it's compelling to say this person is maybe a career changer or this person has a background in [STEM 00:18:55] subject. Let’s put them in front of the classroom. I'd love to hear your thoughts on that Dara.

Dara:              It's a constant tension between lowering the barriers to entry into the teaching profession because you have this hard staff grade levels and subjects so you lower the barriers to entry, how do you maintain quality control. This article, this study seems to speak to the fact that lowering barriers to entry via alternative certification and via allowing out-of-state teachers reciprocity with their credential is not a good thing because those teachers don't do as well.

However, it doesn't mean that you have to keep those barriers high, it just means that you need to have quality control either with entry or evaluation systems that allow for the removal of an ineffective teacher as soon as they prove themselves ineffective especially if the barriers to entry are low.

Michelle:       All right fascinating stuff. Thanks so much Dara.

Dara:              My pleasure.

Michelle:       That's all the time we have for this week's Gadfly Show. Till next week.

Alyssa:           I'm Alyssa Schwenk.

Michelle:       I'm Michelle Gininger for the Thomas B. Fordham Institute signing off.

Pages