Standards, Testing & Accountability

The 2013-14 school year marked the first year of the Third Grade Reading Guarantee (TGRG), a law that requires the retention of children not reading on grade level to be retained. This initiative was modeled after similar legislation in Florida and other states. The policy is also based on research that shows that students who can’t read on grade level by third grade are four times less likely to graduate than a child who reads proficiently. These numbers are even higher for children who live in poverty, particularly Black and Hispanic students.

In a TGRG document posted on its website, the Ohio Department of Education notes that approximately 24,000 students drop out of Ohio high schools each year. They go on to say that most of the students who drop out do not have the reading skills necessary for future success, and that the Third Grade Reading Guarantee is a way of ensuring support for struggling readers early in life.  At Fordham, we’ve long said that reading is important to long-term success, and research shows that third grade is a pivotal year. But with all this focus on third grade, we could be missing another pivotal year that’s just as deserving of our attention—ninth grade.

In the past few years, education researchers have begun to label ninth grade as the “make or break” year for students. Research shows that more students fail ninth grade than any other grade in high school, and a disproportionate number of students who are held...

Ohio’s school and district report cards were released last week, nearly a month later than originally scheduled due to inclement weather….back in February and March. No matter; they’re here now and every education stakeholder is poring over them. But to what purpose are these troves of data being put? 

Out of the gate, stories in the media focused on the “big picture” issues: urban districts (pretty bad, with some rays of hope) and dropout recovery schools (same, minus most of those rays of hope). A single grade for “overall performance” is still not being given this year but should be available in 2016. That left analysts digging through a variety of indicators at all levels. Performance index scores, value-added calculations (very confusing), graduation rates, and other factors were considered, either in isolation or in tandem, producing very different conclusions depending on how the measures were parsed or weighted by the investigators. It is tempting to say that certain foregone conclusions were bolstered by the ways in which data were considered or not considered, but perhaps it is more accurate to say that getting an analysis of such a wealth of information out the door quickly necessitates a narrowing of focus, for better or worse.

We’ve already seen some really excellent investigation of report card data this year, adding the journalist’s touch to what could just be cold recitation of numbers. We hope to see more stories making apples-to-apples comparisons between...

Since the beginning of the No Child Left Behind era, most schools in all 50 states have been given an evaluation of student performance and an overall rating. While crafting a thoughtful and nuanced accountability system is a frequent topic of discussion on The Gadfly (and is really what matters most), here I simply want to discuss the label that sums up a school's overall evaluation. Some might say it's wrong on principle to label schools. Others worry (and sometimes justifiably so) that a nuanced view of schools get lost when we attempt to boil it all down to a single school rating. Moreover, some may see these labels as nothing but a value judgment about "good" schools and "bad" schools when it's clear that parents value many different things about a school. From academics and facilities to safety and course offerings, even the "best" school might not be best for all kids. 

However, we can make objective judgments in some areas.  In addition, use of these labels is not only widely supported, it's also ingrained in federal policy through both NCLB and waivers. So the question is: If we're going to put schools into categories, what should those categories be called?  A few ideas to consider:

1.       States should avoid too few or too many categories - One of the major gripes about No Child Left Behind at the time of passage was that it treated similar schools very differently. The law initially set a bar for schools to clear called Adequate...

Fuzz-free math

CCSS myths, noncognitive skills, Dana Goldstein, and gifted ed.

Amber's Research Minute

Does Gifted Education Work? For Which Students?, by David Card and Laura Giuliano, National Bureau of Economic Research (September 2014).

School report cards arrived today. The good news is that Ohio has a waiver from No Child Left Behind’s (NCLB) “100 percent proficiency” mandate for 2013-14. Very few Ohio schools, I suspect, hit the 100 percent mark in math and reading in 2013-14. (A rough read of district and charter-school data, indicate that a couple high-achieving charters came close; for instance, in grades 3-8 Columbus Preparatory missed 100 percent proficiency in just fifth-grade reading. Menlo Park Academy, a charter for gifted students, came close too.)

A good first step to understanding state assessments is looking at student proficiency. Proficiency is a one-year snapshot of student performance, measured by state exams, not necessarily a clear indicator of the performance of their school per se. For a clearer understanding of the impact of a school on achievement, we’d want to look at student-growth measures, such as the state’s value-added data. (We’ll unpack the value-added results in more depth in the near future—so stay tuned.) But proficiency does give us a general sense of how students performed on state exams in 2013-14.

Statewide, around one-in-five students fell short of Ohio’s standard for proficiency, though there is some variation across grade and subject. (That variation could be a result of the mechanics behind the definition of "proficiency" across grades/subject, not necessairly a function of differences in actual achievement across grades.) Figure 1 shows the proficiency rates for grades 3-8 and 10 in math and reading. The chart displays the...

We seemed to have welcomed good manners back to the Common Core debate. That doesn’t mean we’ve seen more advocacy either on behalf of the standards or knocking them, only that the tenor appears to have changed for the better. At least for the time being, detractors are no longer paranoid Neanderthals, and supporters have ceased to be communists on the federal or Gates Foundation dole.

Whether this détente will prove to be ephemeral or lasting is anyone’s guess, but some credit should go to one CCSS advocate and one foe. In a Washington Times op-ed, Fordham’s Mike Petrilli and Neal McCluskey of Cato, hoping to tamp down the “raucous debate,” sought to re-ground the conversation in a number of facts.

Their piece argues, among other things, that both sides have good intentions; that much Common Core activity began before President Obama was elected, that much of that activity has been led by non-government bodies; and that federal policy—stretching from 1994 to this administration’s Race to the Top and ESEA waivers—has played a meaningful role in the standards’ adoption and implementation.

There are other clear signs of restraint. Former Arkansas Governor Mike Huckabee recently told a crowd that the Common Core fight should be dialed back. Though her union is still frustrated with implementation, AFT head Randi Weingarten penned an op-ed lauding the promise of the standards. Common Core co-author David Coleman recently denounced insulting language directed at opponents, and Glenn Beck scaled back his...

Marc Schare

Marc Schare is the Vice President of the Worthington City Schools Board of Education (in suburban Columbus), now serving his ninth year.

EDITOR’S NOTE: Marc Schare testified before the Ohio House of Representatives’ Rules and Reference Committee on August 26, 2014, opposing House Bill 597 which would repeal Ohio’s New Learning Standards. The following is from his written testimony before the committee.

We in Worthington are confused by this legislation. Perplexed really. Baffled might be the right word.

You see, the State told us back in 2009 that our “Excellent” rankings didn’t mean much anymore because Ohio’s academic content standards and cut scores were too low and that too many kids statewide were having to take high school all over again once they got to college. Fair enough, so Ohio responsibly adopted new academic content standards and recommended that we develop a curriculum based on those standards. For the next three years, teams of teachers representing over 20% of our total teaching staff met in small groups to re-write Worthington’s local curriculum. It was an enormous undertaking. The teams would methodically, standard by standard, define learning targets, compile lists of resources, determine best practices and associated professional development on a subject by subject, grade by grade basis. The result of this effort according to preliminary reports from ODE is that Worthington students using our new curriculum performed at their highest level in years.

While all this was going on, our Information Technology department was preparing to implement the...

Released on August 20, The Condition of College & Career Readiness examines the college readiness of the high school class of 2014 using ACT test scores and College Readiness Benchmarks. Approximately 1.85 million students, or 57 percent of all American graduates, took the ACT in 2014—an astounding 18 percent increase since 2010. Ohio students posted an average composite score of 22—relatively unchanged from previous years and one point above the national average. More interesting are the College Readiness Benchmarks, which indicate the chance of a student earning a B or higher in a college course in English composition, Algebra, biology, or social science. The overall report provides this data for the nation, but individual state level data is also available (Ohio’s data). It’s not a pretty picture. Of the 72 percent of Ohio’s 2014 graduating seniors who took the ACT, only one in three (32 percent) scored high enough to be deemed college ready in all four academic areas. Because not every student took the ACT, only around one in four (23 percent) of Ohio seniors can be considered college ready. If, as expected, PARCC sets its cut scores at the college and career ready threshold, Ohioans will to need to prepare themselves for the challenge that awaits as we work to make sure that more students have the skills they need to be successful on whichever path they choose after high school. Check out the report for a more detailed look at the persisting national achievement gap, top...

It’s nearly school report card time in Ohio. One thing to watch for when examining school performance is whether there are conflicting ratings. For the 2013-14 school year, schools will receive ratings along up to ten dimensions of performance, though no overall letter grade. For example, one might observe a school that receives an “F” on the state’s performance index but at the same time, also receives an “A” on the state’s value-added rating. Or vice-versa. How in the world can this happen?

Keep in mind that these two key ratings—a school’s performance index and value-added—are not the same. The performance index is an indicator of raw student achievement, weighted across a continuum of achievement levels. Value-added, on the other hand, is a statistical estimate of a school’s impact on student progress—expressed as learning gains—over time.[1] Although both measures are based on state test scores, they are different creatures: Achievement tells us more about how students perform; value-added provides evidence on how a school performs (i.e., the productivity of the school staff).

Hence, to understand the quality of a school, we really need both measures. Outside observers—parents, taxpayers, and others—should know whether a school’s students, on average, possess literacy and numeracy skills—that’s achievement. And they should know whether a school is contributing to learning over time—that’s progress.

Now back to the question of mixed ratings. How many schools in Ohio have conflicting results, particularly of the low-achievement but high-progress variety?[2]...

Hitting pause on testing, vouchers, and union solidarity

Michelle and Robert applaud Secretary Duncan’s reasonableness, question a North Carolina trial judge (but have a solution), and disparage union agency fees. Amber tells us how classroom peers affect the achievement of students with special needs.

Amber's Research Minute

Peer Effects in Early Childhood Education: Testing the Assumptions of Special-Education Inclusion,” by Laura M. Justice, et al., Psychological Science (2014): 1-8

Transcript

Michelle G:               Hello. This is your host Michelle Gininger of the Thomas B. Fordham Institute here at the Education Gadfly Show and online at EdExcellence.net. Now please join me in welcoming my co-host the Seth Meyers of education reform, Robert Pondiscio.

Robert P:                    I'm not even sure what that means and hello Michelle.

Michelle G:               Hello. I guess you unlike everyone else in America with not watching the Emmy's.

Robert P:                    No, no, I have a 16 year old daughter so of course my daughter.

Michelle G:               You know more about this than anyone.

Robert P:                    I describe this as the cultural equivalent of secondhand smoke, you're close it. You absorb some of it unintentionally but does that mean I'm focusing on it, no. Were you happy with who won?

Michelle G:               I've heard some of these reviews. I thought it was funny. I thought Seth Meyers did a pretty good job. There are some jokes that I laughed. I felt ...

Robert P:                    Okay. He's a funny guy.

Michelle G:               I felt like a real American. Usually I don't want all the award shows are doing any of that but I thought I was participating in what America does. Maybe I'll watch a football game this season.

Robert P:                    All I know is what I heard in the background blah, blah, blah, Game of Thrones, blah, blah, blah, Game of Thrones, blah, blah, blah, Game of Thrones.

Michelle G:               Isn't that all you need to know about TV?

Robert P:                    Pretty much.

Michelle G:               All right, with that we're going to play part in the Gadfly with our Com. Dev. intern Ellen. Ellen, take it away.

Ellen Alpaugh:          Last week Secretary of Education Arne Duncan declared that states with NCLB waivers could wait until the 2015 - 2016 school year to tie test scores to teacher evaluations. Is this just one year delay and nothing more or does this say something bigger about the testing over the long run?

Michelle G:               Both. Robert, you want to elaborate.

Robert P:                    Lordy, this is such a complicated question and no I don't think it goes away. I think it ... This delays it but I think a hard rain is going to fall on this. There was some polling data out last week that we talked about. A PDK poll on education next poll and you should never I suppose paint with two broader brush based on any particular finding. Look, let's be honest, testing is not popular. I was a teacher for several years and you can't blind yourself to the deleterious impact that testing is having on our classrooms.

                                    Curriculum narrowing, anxiety, lots of push back against testing. What's interesting when you look at the polling numbers is that testing itself is not necessarily unpopular. Something that jumped out to me ... At me in the ed next poll is that things like SAT testing, AP testing are really popular or as popular as a test is going to be. It's when you start looking at these accountability test in grade three through eight under whether there's no child left behind or common core.

                                    The people have lost track of why we do this. You have this kind of conundrum which is the ed reform movement is still largely popular. People like things like charter schools and choice and even vouchers but testing is really unpopular right now. Testing you could or it has created the momentum for these things at the same time it's almost threatening to turn on itself. Arne Duncan thanks for giving us a year off, buy us some time for common core and all these other good things but at some point we're going to have to decide what is exactly the role of testing in K-12 education and in ed reform.

Michelle G:               I completely agree. Yes, testing is no fun, it's awful, it's an imperfect measure, all of those things but if you look at what we support in education or what the public supports in education. A lot of it is because we have evidence that it work and we have evidence that it works because of test. Voucher programs even some school choice supporters don't like the independent evaluations that we've had on the DCPS program and the program in Milwaukee. Yet, those same folks are using those testing results to show that school choice work. You can see this across the issues. Why do we like charter schools? Probably because we're seeing some data that they are educating students better.

Robert P:                    When you say data, you mean?

Michelle G:               Results from test.

Robert P:                    There you go.

Michelle G:               It's sort of like dieting. It's not fun, no one likes eating rice cakes and celery and exercising but if you want to stay slim and fit you got to these things.

Robert P:                    Sure.

Michelle G:               It's just the way it is. It's not fun thing but guess what, it's life.

Robert P:                    Yeah, and I wrote a piece about this early this week and I describe this using Jefferson's quote from 1820's about slavery. Our relationship with testing is like holding a tiger or a wolf by the ears, you don't much like it but you can't let go.

Michelle G:               A lot of people on Twitter were liking your analogy there, bravo on that.

Robert P:                    Bravo Mr. Jefferson.

Michelle G:               All right, Ellen.

Robert P:                    A steal from the best.

Michelle G:               Question number two.

Ellen Alpaugh:          On Thursday a North Carolina trial court judge held unconstitutional a state voucher law that allowed public money to pay tuition at private and religious school. How big of blow is this for voucher proponents and how should they respond?

Michelle G:               All right, I'll say that this is a moderate blow to voucher proponents but a big blow to families in North Carolina.

Robert P:                    Especially when they're starting school and they got to write a tuition check.

Michelle G:               Yes, just over ... Almost 2,000 scholarships have been issued for this program and private schools started this week for a lot of student in North Carolina. That just puts a lot of upheaval in many families lives. That's what the first issue but the second is this is a program that was means tested. Families qualified if they were at or below 133% of the poverty level and according to the Alliance for School Choice which I worked for, full disclosure there.

                                    They hit seven of the eight accountability measures for voucher programs. It's very, very, very high on the accountability spectrum. In all intent and purposes this was a great program. Why it was ruled unconstitutional? I'm not a lawyer. The North Carolina does not have a blind amendment but this is a blow to families. I think they'll go back at it and they'll try to pass the program in the slightly different way. Perhaps changing the funding mechanism or whatever is needed but it's just a longer wait time.

Robert P:                    Sure. I'm reading for the decision here and it says that, "General assembly fails that children of North Carolina when they're sent with public tax payer money to private schools that have no legal obligation to teach them anything. I guess you could argue that but I'm not sure that's a credible argument. Look as Brandon Wright our colleague in Legal Expert says, "If that's the test, well then you just need to go back to the drawing board. Pass a law that says that private schools will give out the accountability measures, the test, etcetera and then problem solved.

Michelle G:               Yeah, and I think that proponent should have seen this coming. I looked back there's a great study that the Institute of Justice put out many years ago that I still go back to which looks at the ... State Constitutions in all 50 states and says whether school choice programs, whether vouchers or scholarship, text credit programs would be constitutional and it says yes, no depending on the program. For North Carolina it did say vouchers and scholarship text credit programs were constitutional but it did say that if a bill was to be ... Law was to be passed it should not draw from the public school funding stream which is basically what they did. No surprise in the long run.

Robert P:                    Of course there'll be an appeal.

Michelle G:               It's America, there you go. Ellen, question number three.

Ellen Alpaugh:          New York City's United Federation of Teachers supported a Saturday march against police brutality. Pitting one city union against another and angering many teacher union members. Teachers in NYC can choose not to be a member and avoid dues but all teachers still have to pay agency fees. What does such union activity say about these mandatory contributions?

Michelle G:               Mr. New Yorker?

Robert P:                    Man, the contributions is not withstanding. This is such a good old fashion New York City style food fights. Some of the stuff that's coming out with the police just the outrage from that the UFT would take this on and that Michael Mulgrew would participate in this protest is just amazing. I heard one teacher say, "Would we want cops protesting in our schools over low test scores?" The head of the PBA said something, I got it right here in front of me, "How would Mulgrew like it if police officers with the activist who oppose his efforts to shield bad teachers and undermine effective charter schools?" This is quite ... The fur is flying here.

Michelle G:               Only in New York it seems.

Robert P:                    Sure but you have to wonder what was Mulgrew thinking. This is ... Look, you can't make light with this, this is serious incident somebody died but if you're deciding where to spend your political capital and your members capital capital. I'm not sure this was the wisest decision.

Michelle G:               I think this is what happens when you work outside of your very narrow issue. On one hand you're building a strong coalition on the other hand when you go outside of your one issue for us education or for the unions education you're going to get people who's ... Your own members here are going to say, "I don't quite agree with that," and that just what happens. I think it's a decision that you have to make and in this case it looks like it was a messy one.

Robert P:                    Yeah, and Mulgrew ... Push back by saying, "Look, we have a history as a union of getting involved in these kinds of issues. You invoked union, activism around the freedom riders many, many years ago. Sure you can understand the process that got him from A to B but still the police are institution in New York City and as many police officers have been saying, "Look, you know, our sons and daughters and wives and husbands are teachers." It just feels this was a little bit of a third rail that did not need to be touched.

Michelle G:               Couldn't agree more. That's all the time we have for Pardon the Gadfly and now it's time for Amber's Research Minute. Welcome to the show Amber.

Amber Northern:      Thank you, Michelle.

Michelle G:               You were on Fox and Friends this weekend weighing in on this very issue [UFT police brutality protest]. What do you have to say about?

Amber Northern:      I was. I start out with it's outrageous. It's outrageous and teachers know ... Teachers always know, "Well, some of our donations, some of our fees go to politics," but wow this was right in their face. I loved it. I was like, "You know what, they finally get it." They finally get it because this was ... The zebra was showing it stripes, we just went off on Fox. They call me the next day and said, "You want to do it again tomorrow." My families called me like, "You are riled up," I'm like, "I know, like it just got me," and I dug into the contributions.

                                    Because Doug and I write a report about teacher union strength what seem hasn't been that long but it's ... I think it was two years ago. Anyway, and then when Doug back in the contributions and year after year after year they we giving donations to Al Sharpton's National Action Network along with plan parenthood and a host of other liberal leaning causes. Teachers need to dial like, "Hey, this is where some of our money goes to, like it or not."

Robert P:                    My money, $35 a paycheck for five years. Never joined the union but they got my money.

Amber Northern:      I think they knew this intuitively but then it was just out there blatant. I thought it was good that it happened. I didn't know how it works, you don't get a lot of time to go into the new ones. I didn't even get to talk about ...

Michelle G:               You don't get ... I can go in TV to go into new ones, I'm shocked.

Amber Northern:      I was, "I didn't get to take the agency fees," and all that stuff but anyway it was fun.

Michelle G:               You got riled up and you got your pay across.

Amber Northern:      I got riled up and I then I have a friend of mine taped it because I hadn't watched. My whole face was contorting. I was mad, I was, "Wow. It was really ugly doing that segment," but that what happens when you're riled up but anyway ...

Robert P:                    She's riled up again right now.

Amber Northern:      Riled up about our new study this week. It is a new study out in psychological science that's called peer effects in early childhood education, kind of a boring title but this is interesting study. It examines the performance of preschoolers both those without and with disabilities and how they are impacted by their peers when they're in a mainstream classroom. This is actually according to the authors and I think they're right. This is the first study of peer effects an inclusive classroom that serves preschoolers with disability.

                                    We've got a lot of peer effect research but never on the preschool level and never with kids in the mainstream classrooms. Anyway, they study the language skills of 670 preschoolers average age of four in multiple school districts in a single mid western over the course of a year about half of the kids had high EP's. Three key findings, number one there was indeed evidence of peer effects in the classroom have shown by the strong relationship between kids spring language scores and the language skills of their peers. Definitely a strong relationship between the two.

                                    Number two, the impact of peer effects varied based on whether the child had a disability. Specifically peer effects were stronger for kids with disabilities than those without. Preschoolers in classroom of kids with high language skills tended to have better language scores than preschoolers in classes of kids with lower skills. The lowest skilled kids, if you got that, made the greatest gains. This is what we've seen in other studies.

Robert P:                    Yeah, no surprise there.

Amber Northern:      Kids at the bottom make the greater gains. Kids with disabilities are more influenced by their classrooms language skills than children without. Last bottom line, children with the highest skills were not adversely impacted by the lower performing kids whether they had disabilities or not which is what everybody is always searching for, right? Like, "What about the kids on the other end of the spectrum.?" The study was correlational, it's not causal.

                                    It was one year, it's not a trend study and they also ... The instrument they use which I was kind of dug in. It was a teacher report instrument ... It's a dibble or something.

Robert P:                    It's squishy.

Amber Northern:      Which you typically have to do with young kids you have to deal one on one measure but it wasn't really standardized in a way. A little bit of clumsy there but I think it was encouraging because it showed us once again that peer effects matter and they matter greatly when we mainstream these kids. Which it's not an argument for against mainstreaming but it's interesting stuff.

Robert P:                    Persuasive because the kids at the high end so to speak no adverse effect.

Amber Northern:      They were harmed, right. Why do I want help? They weren't harmed either. They still scored at the end of the year higher than ... Their post test was still higher than a pretest. It wasn't a bad thing.

Michelle G:               Is your recommendation more research?

Amber Northern:      Wow. In my case I won't do it. Anyway, it was a needed area to do research. I think it's a neat idea because they're really striving for balance. On this half the kids with IP's. It's not like you've go 90% of the kids with Ip's you know what I mean. They're really striving to get what set up optimal affect and impact on kids. Half and half seems to be interesting, seems to be a positive outcome. I don't know if they change the percentages whether we would have see the same thing.

Michelle G:               Exciting stuff.

Robert P:                    Is it my imagination or we seeing a lot more pure effects research lately?

Michelle G:               I feel I've been reading more. It's also one of my sort of ... What do we call that, sew boxes if that's the word again. I tend to pay more intention to it just because he's interested in it. He wrote a lot about it in his book. Yeah, I don't know saying more as maybe just we just cover it more.

Robert P:                    You're paying attention. Fair enough.

Michelle G:               Either way both a good thing I think. All right. Thank you, Amber. That's all the time we have for the Education Gadfly Show till next week.

Robert P:                    I'm Robert Pondiscio.

Michelle G:               I'm Michelle Gininger for the Thomas B. Fordham Institute signing off.

Pages