Evaluating the Content and Quality of Next Generation Assessments
Evaluating the Content and Quality of Next Generation Assessments examines previously unreleased items from three multi-state tests (ACT Aspire, PARCC, and Smarter Balanced) and one best-in-class state assessment, Massachusetts’ state exam (MCAS). The product of two years of work by the Thomas B. Fordham Institute, two rock-star principal investigators, and almost forty equally stellar reviewers, the report uses a new methodology designed to answer policymakers’ most pressing questions: Do these tests reflect strong content? Are they rigorous? What are their strengths and areas for improvement?
As our benchmark, we used the Council of Chief State School Officers’ Criteria for Procuring and Evaluating High-Quality Assessments. We evaluated the summative (end-of-year) assessments in the capstone grades for elementary and middle school (grades 5 and 8). (The Human Resources Research Organization evaluated high-school assessments.)
Here’s just a sampling of what we found:
- Overall, PARCC and Smarter Balanced assessments had the strongest matches to the CCSSO Criteria.
- ACT Aspire and MCAS both did well regarding the quality of their items and the depth of knowledge they assessed.
- Still, panelists found that ACT Aspire and MCAS did not adequately assess—or may not assess at all—some of the priority content reflected in the Common Core standards in both ELA/Literacy and mathematics.
Overall, programs received the following marks on content and depth across math and ELA.
Our reviewers spotted areas of strengths and improvement for all four programs:
- ACT Aspire’s combined set of ELA/ Literacy tests (reading, writing, and English) require close reading and adequately evaluate language skills. Its math test items are also generally high-quality and clear. In ELA/Literacy, reading items fall short on requiring students to cite specific textual information in support of a conclusion, generalization, or inference and in requiring analysis of what has been read.
- MCAS’s ELA/Literacy tests require students to closely read high-quality texts, and both math and ELA assessments include a good variety of item types. While mathematical practices (such as modeling and making mathematical arguments) are required to solve items, MCAS does not specify their connections to the content standards.
- PARCC’s ELA/Literacy assessment includes appropriately complex texts, require a range of cognitive demand, and include a variety of item types. In math, the test is generally well-aligned to the major work of the grade. PARCC would better meet the criteria by increasing the focus on essential content at grade 5.
- Smarter Balanced’s ELA/Literacy tests assess the most important skills called for by the Common Core standards, and its assessment of writing and research and inquiry are especially strong. In math, the test is also generally well-aligned to the major work of the grade. In ELA/Literacy, a greater emphasis on academic vocabulary would further strengthen Smarter Balanced relative to the criteria.
All four tests we evaluated boasted items of high technical quality, and the next generation assessments that were developed with the Common Core in mind have largely delivered on their promises. Yes, they have improvements to make, but they tend to reflect the content deemed essential in the Common Core standards and demand much from students cognitively. They are, in fact, the kind of tests that many teachers have asked state officials to build for years.
Now they have them.
|State Use of Next-Generation Assessments, 2016-17|
|Source: Education First Consulting, LLC|
|PARCC (9 states)||CO, DC, IL, LA, MD, NJ, NM, NY, RI|
|Smarter Balance (17 states)||CA, CT, DE, HI, ID, IA, MI, MT, NV, NH, NC, ND, OR, SD, VT, WA, WV|
|ACT Aspire (3 states)||AL, AR, SC|
|MCAS (1 state)||MA|
|Used to Use PARCC or Smarter Balance But Dropped in the last two years (7 states)||AR, ME, MS, MO, OH, WI, WY|
|Never Used These Assessments (15 states)||AK, AZ, FL, GA, IN, KS, KY, MN, NE, OK, PA, TN, TX, UT, VA|
To speak with a researcher about the report, contact Alyssa Schwenk at 202-223-5452.
In the Media
Quality in Adversity: Lessons from Ohio's best charter schools
Though charter schools are fiercely debated in Ohio, too rarely are the voices of charter leaders actually heard. This report from the Thomas B. Fordham Institute surveys the leaders of the highest-performing Buckeye charters to take stock of their views on sector quality, accountability, and replication and growth.
The survey, conducted by the nonpartisan FDR Group, was fielded to the principals of 109 charter schools, yielding a 70 percent response rate.
We hope that Quality in Adversity will help lift these leaders' voices, so that their firsthand knowledge can overcome counterproductive rhetoric and entrenched positions.
In the Media
Policy Brief: Pathways to Teaching in Ohio
Have you ever wondered what it takes to become a teacher in the Buckeye State? Wonder no more! In this policy brief, we outline the entire process—from acceptance to a preparation program all the way to advanced licensure. We even take a look at alternative pathways and out-of-state educators.
America's Best (and Worst) Cities for School Choice
More than twelve million American students exercise some form of school choice by going to a charter, magnet, or private school—or opting for homeschooling—instead of attending a traditional public school. Countless others use district-wide lotteries, attendance waivers, or interdistrict transfers to attend public schools other than the ones in their neighborhoods. But some cities are significantly more “choice-friendly” than others…and some are downright hostile.
Using nearly fifty markers of “choice friendliness,” this report shows which of thirty American cities are the best and worst for school choice.
What makes a city choice-friendly? We examined three factors:
|New York City||12|
|Kansas City, MO||18|
First, POLITICAL SUPPORT:
This area measures the views of various important individuals and groups when it comes to school choice. How supportive are the city council, mayor, superintendent, and governor? How about the teachers' union, parent groups, and the local media?
Second, the POLICY ENVIRONMENT:
It’s nearly impossible for choice to flourish without supportive policies and practices for both providers and parents. This area includes topics like whether charter schools have adequate funding and access to district facilities, as well as the ease with which parents can learn about, and apply to, schools of choice.
Third, the QUANTITY AND QUALITY of choices:
How many choices exist for students—and if they are any good—obviously matters a great deal. This area includes factors like whether students are allowed to attend schools in nearby districts. And if so, can the district choose to say no by opting out? What percentage of public schools in the city are schools of choice? And what is the quality of charter schools relative to their district counterparts?
As you’ll read, some cities’ overall rankings come as no surprise—the gold, silver, and bronze go to choice hot-spots: New Orleans, Washington, D.C., and Denver. But New York City, a roaring engine of choice under Mayor Bloomberg, is now sputtering and fails to crack the top ten (blame the “de Blasio effect”). And how did Atlanta—marred by a cheating scandal—end up surprising us in the ninth position?
There's lots more to learn in this most comprehensive look at urban school choice to date. You can download the full report here or dive into the state profiles on their own.
In the Media
Is Detente Possible? District-charter school relations in four cities
Whether you think the end game of the current “mixed economy” of district and charter schools should be an all-charter system (as in New Orleans) or a dual model (as in Washington D.C.), for the foreseeable future most cities are likely to continue with a blend of these two sectors. So we wanted to know: Can they peacefully co-exist? Can they do better than that? Can they actually collaborate in the service of students, families and the public interest?
We teamed up with Public Impact to address these very questions, in the context of five cities that have among the best conditions for district-charter collaboration: Boston, Cleveland, Denver, the District of Columbia and Houston.*
As we examined these evolving relationships, we found markedly different forms of engagement reminiscent of how international relations often play out. From Washington, D.C.’s “superpower summit” through Boston’s “protectionism under pressure,” the shifting district-charter interplay highlighted in this report may begin to point the way to a new world order in public education.
Download the full report, foreword and executive summary, individual city profiles, and recommendations to learn more:
* Houston is a lesson in “Isolationism” as each sector mostly pursues its own course with minimal contact between them (so much so that we omitted it from full discussion in the report).
If you have questions about the book, please email David Griffith.
In the Media
Policy Brief: Ohio Community School Governance
A thorough overview of Ohio's charter/community school governance framework
Failing Our Brightest Kids: The Global Challenge of Educating High-Ability Students
If you have questions about the book, please email Brandon Wright.
In the Media
Schools of Thought: A Taxonomy of American Education Governance
Report by Dara Zeehandelaar and David Griffith, with Joanna Smith, Michael Thier, Ross Anderson, Christine Pitts & Hovanes Gasparian
Foreword by Amber M. Northern and Chester E. Finn, Jr.
Questions of education governance are often considered moot by policymakers, who typically assume that the governance challenges plaguing their local schools are both universal and inevitable. Given the ubiquity of everything from local school boards to state superintendents, this seems to be a logical assumption. However, a closer examination of state and local education governance arrangements reveals that, despite some common features, the structures and processes that govern education in each state actually vary significantly. Those wanting to put governance into service on behalf of needed reforms are wise to start with a clearer understanding of not only the arrangements they’re presently working within, but also of the remarkably different arrangements that have arisen in other jurisdictions.
To that end, this study creates a taxonomy of education governance systems. To do this, we first classify governance relative to three main components: the degree to which decision-making authority lies at the state versus the local level; the degree to which decision-making authority is distributed among many institutions versus consolidated in a few; and the degree to which the public can participate in the policymaking process. (Each component consists of approximately 12 discrete indicators.) We score states on each component, then combine them into eight “governance types” named for the characteristics they have in common with some of history’s most famous political leaders and theorists. We supplement our typology with qualitative data that explores how different approaches to governance constrain or facilitate the work of schools and districts on the ground.
Here’s the bottom line—to see how we got here (and for more on what the groups really mean), download the full report.
Addendum as of September 3, 2015: The authors thank the team at the Iowa Department of Education for notifying us that all elected boards in the state (not just state and local school boards, but all elected boards) have partisan and gender balance requirements. (The authors apologize for missing the necessary statutes, as data for these indicators were gathered from state education codes only.) Iowa's Degree of Participation type should therefore be classified as "participatory," and its governance type as Jeffersonian.
If you have questions about the book, please email David Griffith.
In the Media
Establishing a baseline: Ohio’s education system as it enters a new era
If you have questions about the book, please email Aaron Churchill.
Who Should Be in Charge When School Districts Go into the Red?
School districts across the land are contending with rising education costs and constrained revenues. Yet state policies for assisting school districts in financial trouble are uneven and complex. Interventions are often haphazard, occur arbitrarily, and routinely place politics over sound economics.
This brief presents a menu of sensible state responses when districts are insolvent or nearly so, arranged into a tiered sequence of interventions.
1. Collaborative Supports
District leaders receive low-impact assistance in managing their finances. Supports might include convening a budget review committee to identify unnecessary expenditures or assisting district finance officers to develop more accurate projections of future revenue. The goal is to work with leaders to recognize and rectify the causes of distress.
2. Financial Management
At this stage, experts are no longer advisory; they now oversee and manage a district’s financial matters. The goal is immediately to improve district finances so as to avert costly bailouts down the line, while building the capacity of district leaders to manage once the experts leave.
3. Administrative Control
Otherwise known as a “state takeover.” Outside experts manage the entire district, not just its finances. A state-appointed administrator and/or governing commission replaces or supersedes the superintendent and board and operates with additional powers. Changes in district management can be accompanied by an emergency loan if necessary, although any major financial assistance should hinge on complete administrative control. The goal here is to remove ineffective leaders and prevent district bankruptcy and closure.
Few districts need drastic measures. Quiet technical assistance is often enough to help local leaders project revenue accurately and adjust expenditures to match. External advisors can also give district leaders political cover to make unpopular decisions. The fear of greater consequences is motivating too, which is welcome news since most states aren’t equipped to run districts. For districts on a catastrophic course, however, “takeover” is warranted—and from the perspective of students and taxpayers, it is even essential.
Download the brief for information on what each stage entails, as well as profiles of districts and states that have successfully implemented interventions (and those that didn’t).
If you have questions about the book, please email Victoria Sears.