The Amazin’ Mets edition
AP U.S. History, teacher professional development, the myth of the overworked American kid, and math coursework’s effect on college readiness.
Amber's Research Minute
SOURCE: Shaun Dougherty et al., " Early Math Coursework and College Readiness: Evidence from Targeted Middle School Math Acceleration," National Bureau of Economic Research, Working Paper 21395 (July 2015).
Robert: Hello, this is Robert Pondiscio of the Thomas B. Fordham Institute here at the Education Gadfly show and online at edexcellence.net. Now, please join me in welcoming my co-host, the Lucas Duda of Education Reform.
Brandon: All right, Lucas Duda.
Robert: The Duda abides.
Brandon: ML Player of the week.
Robert: How could he not be? Like, nine home runs? It's August 1st. I don't know what the date is, but it's August 1st. Why? Because it's August and they're in first place. That hasn't happened since some people who worked at Fordham, before they were born.
Robert: Well, not really, but it's been a long, long time. I'm going to enjoy every minute of this, because, as a Met fan, we know this too shall pass.
Brandon: And we are both huge baseball fans.
Robert: We are.
Brandon: Talk about it often.
Robert: I was going to call you the Noah Syndergaard before because of your long, flowing blonde locks.
Brandon: He was on my fantasy team. Thor.
Robert: Thor, that's right.
Robert: This is much more fun than talking about education. Let's just make this baseball puns. The fun just begun. We have a new voice of the Gadfly this week. Please welcome Audrey Kim to the proceedings, and Audrey, take us away.
Audrey Kim: Thank you Robert. Your recent U.S. News and World piece argues that we shouldn't buy that today's students are overworked and exhausted. Why not?
Robert: Because they're not. Simple. Next question. No, I should give the back story here. There was a column in the New York Times by a columnist who I normally like, Frank Bruni, and I still like him. He's a serious, sober guy. He writes thoughtfully, but he swallowed whole what I take to be a bit of, I don't want to call it a myth, that's overstating the case, but a bit of a non-problem. "We're working kids too hard. They're taking too many APs. We're stressing them out. They're over-scheduled." Look, I don't want to minimize this because for some kids this is surely the case. There have been stories of kids even committing suicide. Not massive numbers, but it happens. The much larger the story, I think, and this is what I wrote about at U.S. News, is exactly the opposite. The under-stressed U. S. teen. I looked at the data, it's something like 5% of American students take more than 5 APs. They are the anomaly. What percentage of students, Brandon, would you guess take no APs.
Robert: Yeah, about that. Two out of three.
Brandon: Lucky guess.
Robert: Two out of three American students don't take a single, darn AP.
Brandon: That's not surprising.
Robert: This is the problem that I describe as the worried well. In other words, people whose children or people who write columns in the New York Times, whose children go to school. Yes, this is their reality, but for the vast majority of American students, this is just not the case.
Brandon: Yeah, and we ran this by a colleague here, and she sympathized, but she was more interested in the other side of the over-stressed kid. The low-income kid who has to do a lot of things outside of school, like take care of his or her siblings or go to work. Do all these things that actually don't advance his or her academic prospects. That's kind of a bigger and really kind of more interesting and dire issue than the over-stressed affluent kid.
Robert: For what it's worth, I went looking for data on that, and it's hard to find, I don't know what number of kids have part-time jobs or are taking care of siblings after school. I suspect that it's a lot more than we think. What was interesting, there is data, since one of the other things that Frank Bruni was writing about in his column, was the over-scheduled kid. One activity after another, their scheduled within an inch of their life with youth soccer, dance lessons, whatever. There's every good reason in the data to look at that and think, "Look, those kids have really good outcomes. They're more likely to stay in school. They're more likely to not get involved with alcohol and drugs. They're more likely to grow up and finish high school and live lives of active, engaged citizens."
These are all good outcomes. Yes, there is some, small subset who perhaps over-do it. This was another interesting data point. Something like 40% of American youth do no organized activity whatsoever, so it's exactly the same thing as the AP. For every single kid who does too much, there are multiple who do nothing at all.
Brandon: Right. Which, again, seems to be a bigger issue.
Robert: Again, I don't want to say that it's not a problem at the high-end, but it's a much larger problem at the other end.
Brandon: Yeah, agreed.
Robert: Question number 2.
Audrey Kim: A new study from the New Teacher Project finds that, although districts are investing in teacher development, teachers aren't getting much better. What do you make of this?
Robert: I don't know about you Brandon? Having been a teacher for several years, I can tell you what I make of it, which is that, forgive my language, professional development sucks.
Brandon: Yeah, I haven't been in the classroom beyond being a sub for a year, but I don't think it is extremely effective in any job. It seems more important to hone training before someone enters, or in their early years, than to do these continuous. things like going to a seminar or taking a night class every year or two years.
Robert: A lot of it is choose your own adventure. It doesn't end up being very, very good. This is a bit of a wasteland. Here's a list of all of the goods. I went to in five years as a full-time teacher. Ready? Here's the list ... Okay I'm done. It really was a bit of a waste of time. What the New Teacher Project report find that is sobering is this is bewilderingly expensive. I think the figure was something like $18,000 per teacher, per year.
Brandon: For public schools.
Robert: Right, and what good outcomes do we see from that? None whatsoever. The vast majority of teachers show no improvement whatsoever.
Brandon: Like 3 out of 10 did or something?
Robert: Yeah. From memory, I think, 3 out 10 improved, 2 out of 10 got worse, the rest, no difference whatsoever. The logical question that TNTP is asking is, "What are we spending all this ... What are we getting for all this investment in it.?" The answer seems to be not a lot.
Brandon: The thing was that, for charters, I think the number was 7 out of 10 teachers improved, but it also cost, I think, $33,000 per. The interesting thing there though, right, I brought up earlier that investing in teachers early, be it before they start or in their early years, that seems like that would produce better gains and charter teachers are, on average, younger than public school educators. We're involved in a book right now, and I think the average number of years teaching for charter school teachers is something like 8, and for public schools it's 15.
Brandon: It's a younger workforce ... With less experience.
Robert: It is younger, but I haven't poured over the TNTP report in detail, but if I understand it correctly that most of the gains come at the front end and then you level out. Then that's exactly what you'd expect to see.
Brandon: Okay. Sure. Yeah.
Robert: It's still sobering. Ed reform, I think, at large, we put a lot of value on the idea of professional development, on improving teachers, on teacher quality at large. I've made a joke of this myself over the years saying, "You go to school with the teachers you have, not the teacher you wish you had," so we have to find a way to make them better. This report suggests that's a really, really heavy lift.
Robert: Question number 3.
Audrey Kim: The college board has just released a new set of AP U.S. History standards in response to criticism that the 2014 standards presented a negatively biased view of American History. What's your take, and do these new standards present a more fair view of our nation's history.
Robert: What do you think Brandon?
Brandon: The expert opinion on the latter question seems to be yes. Pretty much everyone who has reviewed the new ones think the liberal bias is kind of gone. The majority of the people who reviewed the old one believed that it was there. People on both sides of the aisle. Step back and see what the college board did. It's pretty impressive actually. You have this big organization taking a bunch of flack for a program that they made, that almost everyone has an opinion on, regardless of whether they have kids in the actual class. For instance, you aren't going to see this in AP Chem. They got the flack. They stepped back, and they really put a ton of time into improving them, and it seems like they have.
Robert: Yeah. You know what's interesting, too, I think the perception, I don't know this, but I suspect that a lot of the perception will be, "Oh the college board caved to conservative critiques." I just had a conversation with Trevor Packer, the head of the AP program. He was here visiting us a few weeks to tell us about these changes. What impressed me most is, yeah that may be the narrative that they were paying attention to conservative, political critiques, but when you really start to unpack what they did. They were paying attention to teachers, and I don't have this data in front of me, so this is from memory. Now, to be fair, the majority of teachers did not perceive a bias, but among teachers who did perceive a bias after teaching the previous framework, they were four times more likely to perceive a liberal bias than a conservative bias. That was a revelation to the folks who had issued the framework and folks like Trevor Packer, so yeah, I give them credit. I do think they made an earnest attempt to get it right, and all the initial feedback from the pundit class, at least, is favorable. I'll be paying attention a year from now, when they go into the field and ask teachers what they think.
The other interesting thing about this is, we've forgotten in all of this what the big idea here was in re-vamping the AP History framework to begin with. It wasn't about who gets in, who gets left out, and what the bias is. It was an attempt to get AP History students to engage more with primary source documents, to do more historical work as opposed to just making it ... Repeating facts and taking a multiple choice test, and that seems successful as well.
Brandon: Yeah. Agreed, agreed.
Robert: And that's all the time we have for Pardon the Gadfly. Next, here's Amber with this week's research minute.
Brandon: Amber. How are you?
Amber: Hey Brandon. Doing great.
Brandon: Good, good. Are you a baseball fan?
Amber: I am. I like the Orioles and the Nats.
Brandon: You like both.
Amber: That seems odd. I know, that seems like a traitor, right? You don't like the Orioles and the Nats, but I don't know, Orioles cause of Cal Ripken. I was always a Cal Ripken fan, so I still like them, and the stadium is phenomenal. They used to have these great pretzels with about 10 pounds of cinnamon on them. That was the whole reason, really, that I really liked going to the games. Then, they stopped ... The pretzel vendor left, so now they have these crummy little stale pretzels. Aren't half as fun for me, so anyway that's really what it is. It's the pretzels, it's not really the baseball for me at the Orioles.
Brandon: The ball-park experience.
Amber: Yes, the experience. I normally don't eat junk food, but you have to when you go to the games. Anyway, it's all good. I'm a baseball fan.
Brandon: Cool, cool. What do we have this week?
Amber: We got a new NBER study, doing a lot of those, examines the impact of math acceleration on students prior to their graduating. This is a little bit wonky, but a lot of people know that we had these Algebra-For-All policies in the 90s, where we really trying to push all kids into Algebra 1 because we thought we have an access problem here. We need to get these kids on track to college readiness. Then they kind of fell out of favor because a lot of the kids weren't being successful in the harder courses, and there was some questions whether we were watering down the courses.
Now we've kind of got the next generation of policies trying to solve this problem. These analysts looked at Wake County public schools in North Carolina, and they had implemented a more targeted acceleration policy in 2010-2011 that funneled particular students into a higher sequence of math courses. It's not college for all, it's college for these kids that we're going to identify. They were identified based on an algorithm, which basically used the history of that kid's test scores that predicted their probability of passing a standardized Algebra test. The threshold they used corresponded to the 25th percentile of the district's skill distribution, which essentially means about 75% of students would be placed on the accelerated track, so pretty big number.
Analysts were able to compare the performance of nearly identical students who scored just above and just below the threshold. They followed students from the end of 5th grade, when they received their probability score, through high school. All right, quick findings. Number one, acceleration had no clear effects on end-of-grade math test scores. Number two, it did, however, induce lower grades at the middle school level, so we saw the kid's grades go down in the middle grades. Yet, it raised the probability of taking, and passing, geometry in 9th grade by over 30 percentage points, including for those disadvantaged kids. Number four, most students accelerate in middle school do not remain that way in high school. In other words, there are three or four sequence of courses they were supposed to take, but a lot of them don't make it to the end. They get, was "de-accelerated" a word? Maybe. For those that do manage to stay in that accelerated pipeline, they end up earning lower grades in those courses.
For instance, 40% of students accelerate into pre-Algebra in 7th grade, continue on to Geometry in 9th grade, so that's a significant drop from 7th to 9th grade. Anyway, the bottom line is, they say we have a leaky pipeline, right, and that these acceleration policies alone aren't enough to keep kids on track with rigorous coursework. There's a little bit of discussion, okay like, now what do we do? Everybody says ... What do you think? What do you think that people think the answer is? I mean, like okay, if kids can't do it, then maybe we should give them ... Tutoring, right?
Brandon: Okay. Sure.
Amber: Then there was some other evidences they site that said, "Oh by the way, when we looked at the impact of tutoring, that didn't really seem to help them either." It was kind of a bleak report at the end, where you're like, "Okay, so we have this more targeted way of identifying these kids and trying to open up access, but the results were really mixed. On the one hand, we didn't see a big bump in these test scores. On the other hand, there were some positive impacts around Geometry, and so on and so fourth. At the end of the day it just seemed like a lot of these kids weren't able to stay and stick into that acceleration pipeline and were dropping out at pretty high rates."
Then when you say, "Okay well maybe they need more resources and supports." The supports didn't seem to be helping either. Honestly, I don't know what the answer is. You know?
Brandon: Yeah. The findings leave me stumped. I don't really know what to say at all.
Amber: Then they say, well ... These guys are rock stars, one of them is one of our EEPS, Shawn Doherty, but one of the things it says, "You know, well maybe we would see impacts if we looked at long-term outcomes," which seems kind of, you know, a long shot. They are going to check back in and look at these kids after high school and see if there is any college acceptance outcome or any, kind of like, things like that. You know, maybe we're just not following them long enough. Maybe we're going to be surprised by some of the long-term impacts of such a strategy. Kudos because I think there's a real understanding that the Algebra-For-All sounded good, it's kind of like the 2014 rhetoric around NCLB, but when you really kind of dig in, there are a lot of perverse incentives.
Brandon: Interesting. Kind of bleak.
Amber: Kind of bleak. Kind of bleak, but I mean I think we ... You've done a lot of research around gifted education.
Amber: I think acceleration sounds pretty good.
Brandon: For certain kids, yeah.
Amber: For certain kids, right? Maybe they try to target the kids here? Who knows. Maybe their target was off? Maybe they needed to set a higher threshold for how to identify those kids. I don't know. Acceleration just seems like it's not the new answer to all of our woes either for some of these struggling kids.
Brandon: For sure, for sure. Okay. All right. Thanks Amber.
Amber: You're welcome.
Robert: That's all the time we have for this week's Gadfly show. 'Til next week.
Brandon: I'm Brandon Wright.
Robert: And I'm Robert Pondiscio for the Thomas B. Fordham Institute, signing off.