Evidence-based products won't sell themselves

Getty Images/Ridofranz

In my last post, I described a demand-side approach to bringing evidence-based practices into schools by developing programs and processes that help educators ask the right questions and find new solutions that work for them. Now I’d like to tackle the supply side: the creation and marketing of tools, especially curriculum, that can help drive evidence-based change in the classroom.

While the Golden Age of Education Practice might still be a figment of my imagination, the Golden Age of High Quality Curriculum is here and now. That’s because a number of promising curricula have come onto the market in recent years, driven by the opportunity window opened by the Common Core. (And no, despite what you may have heard, that window has not yet closed. Most state standards are still very close to the Core.)

Here’s how Jared Myracle, Chief Academic Officer of the Jackson-Madison school district in Tennessee, puts it:

[A] few years ago, there weren’t many curricula aligned to the new standards. Fortunately, that has changed—education leaders describe a “curriculum renaissance.” I’ve seen that firsthand: All four curricula that we adopted in Jackson-Madison County became available in the last few years.

He’s right: According to EdReports, thirteen products have met its high standards for alignment and usability in elementary or middle school English language arts; ten in math. These include programs by nonprofit upstarts, like Great Minds’ Wit and Wisdom and Eureka Math K-5, EL Education Language Arts, and Open Up Resources Math, authored by Illustrative Mathematics, and as well as offerings from for-profits and legacy publishers, including American Reading Company’s ARC Core, Pearson’s ReadyGen and MyPerspectives, and McGraw-Hill’s StudySync

To be sure, reasonable people may dispute whether validation from EdReports, or other entities that review instructional materials, makes a curriculum “evidence based.” These new curricula haven’t been around long enough to have gone through experimental-design studies and demonstrated impacts on student achievement. But they are aligned to state standards, which themselves are informed by evidence, and have also been found by educators to be teachable in real-world settings. That’s something.

This raises a key question—one recently posited by my friend Rick Hess—about what we mean when we reference “evidence based practices.” Do we mean discreet teacher behaviors, like how best to instruct students on phonemic awareness? These are the sorts of practices you’ll find in the What Works Clearinghouse practice guides, or the National Reading Panel reports.

But another approach is to bundle a bunch of evidence-based practices and insights together into a coherent and teachable whole. When this is also aligned to academic standards—the what plus the how—we call that a high quality curriculum. Of course, there’s a risk that curriculum designers will stretch the evidence and make claims that can’t be defended; who doesn’t say that their pet curriculum is based on the best evidence? So until we can subject new curricula to experimental studies and check on the effectiveness of the whole package, the best we can probably do is rely on expert and educator reviews, like those from EdReports.

If you build it, they won’t come—not without a marketing budget

So let’s stipulate that schools and districts have an increasing number of high quality curricula to choose among, the kind that do a solid job incorporating evidence-based practices and that track the academic standards states have adopted. Surely these programs are sweeping the country, right?

Sorry. As far as I can tell, only 10–15 percent of districts are using the good stuff today. And while that portion may rise somewhat with time, as textbook adoption cycles come around, or as states follow Louisiana’s lead and make buying the good stuff easier, we’d be nuts to think that the non-profit upstarts in particular will be able to compete without a serious investment in sales and marketing.

Even medicine, which many evidence-based-practice enthusiasts look to with envy, has a challenge with the adoption of new and proven methods. And that’s even though doctors number in the hundreds of thousands, not millions; they operate with a high degree of professional autonomy; and they received lengthy, rigorous pre-service training in scientific fields.

That’s why, as you may have noticed, pharmaceutical companies spend zillions of dollars advertising to doctors and patients—because medical practice doesn’t automatically change because of new findings in respected journals, either.

But just running more ads in Education Week or the ASCD SmartBrief won’t be enough to get great curricula adopted by schools. The new providers need massive sales forces, too. Consider this passage from a great Atul Gawande piece in the New Yorker on the challenge of changing doctors’ and nurses’ practices. (Thanks to Ben Riley of Deans for Impact for pointing it out to me; I’ll return to more of this article in a future post.)

I once asked a pharmaceutical rep how he persuaded doctors—who are notoriously stubborn—to adopt a new medicine. Evidence is not remotely enough, he said, however strong a case you may have. You must also apply “the rule of seven touches.” Personally “touch” the doctors seven times, and they will come to know you; if they know you, they might trust you; and, if they trust you, they will change. That’s why he stocked doctors’ closets with free drug samples in person. Then he could poke his head around the corner and ask, “So how did your daughter Debbie’s soccer game go?” Eventually, this can become “Have you seen this study on our new drug? How about giving it a try?” As the rep had recognized, human interaction is the key force in overcoming resistance and speeding change.

The legacy textbook firms have sales reps all over the country—often former superintendents with personal relationships with the people they are marketing to, with budgets for steak dinners, golf games, and such. Not to mention, less salaciously, big-time budgets for free samples of books, which can get expensive quickly given the large size of many districts’ selection committees.

So if funders and advocates want to see a dramatic uptick in the uptake of high quality materials, especially those produced by the non-profits, they need to face these facts. We can’t just invest in building great curricula—we need to invest in marketing them, too.  To my knowledge, only Great Minds has committed serious money to constructing this sort of sales force, and it’s one reason why Eureka Math is grabbing significant market share. Other new entrants will need to do the same if they want to play in the big leagues.

There is another approach, which is get legacy publishers to create and sell high quality curricula. As noted above, some of that’s already happening, according to EdReports, at least in some cases. Or the non-profits can dance with the devils, as Illustrative Mathematics did with its new deals with McGraw Hill and Kendall-Hunt. Rather than compete with the legacy companies, team up with them. Unleash their armies of sales reps to promote good stuff.

This too is not without risk, though. Stories are legion of high quality titles getting abandoned by corporate publishers when not enough copies sell fast enough.

There are also big question marks about how traditional publishers support the professional development needs of districts. Districts and schools risk “implementation failure” if they adopt new curricula that tee up major shifts in practice for teachers, as the high quality curricula do, without supporting teachers with implementation. Historically, publishers have included professional development “free with the purchase” of a curriculum, and the running joke among districts is that it’s worth about what they pay for it. Shallow “spray and pray” PD won’t help teachers improve their practice, and it remains to be seen whether traditional publishers will build quality services around this need.

So the challenge of getting high quality, standards-aligned, evidence-infused instructional materials into schools is daunting. But let’s end on a positive note: Ten years ago, most of these materials didn’t even exist. We’ve made real progress thanks to the hard work of thousands of developers and educators over the past decade. Compared to many other reforms in education, getting more schools and districts to adopt one of the growing numbers of strong programs is eminently doable.

Michael J. Petrilli
Michael J. Petrilli is the President of the Thomas B. Fordham Institute.