Standards, Testing & Accountability

Ohio’s report card release showed a slight narrowing of the “honesty gap”—the difference between the state’s own proficiency rate and proficiency rates as defined by the National Assessment of Educational Progress (NAEP). The NAEP proficiency standard has been long considered stringent—and one that can be tied to college and career readiness. When states report inflated state proficiency rates relative to NAEP, they may label their students “proficient” but they overstate to the public the number of students who are meeting high academic standards.

The chart below displays Ohio’s three-year trend in proficiency on fourth and eighth grade math and reading exams, compared to the fraction of Buckeye students who met proficiency on the latest round of NAEP. The red arrows show the disparity between NAEP proficiency and the 2015-16 state proficiency rates.

Chart 1: Ohio’s proficiency rates 2013-14 to 2015-16 versus Ohio’s 2015 NAEP proficiency

As you can see, Ohio narrowed its honesty gap by lifting its proficiency standard significantly in 2014-15 with the replacement of the Ohio Achievement Assessments and its implementation of PARCC. (The higher PARCC standards meant lower proficiency...

School report cards offer important view of student achievement -
c
ritical that schools be given continuity moving forward

The Ohio Department of Education today released school report cards for the 2015-16 school year. After a couple tumultuous years, today’s traditional fall report card release reflects a return to normalcy. This year also marked the first year of administration for next-generation exams developed jointly by Ohio educators and the American Institutes for Research (AIR).

“This year’s state testing and report card cycle represents a huge improvement from last year,” said Chad L. Aldis, Vice President for Ohio Policy and Advocacy at the Thomas B. Fordham Institute. “Last year’s controversy made it easy to forget the simple yet critical role state assessments and school report cards play. They are, quite simply, necessary, annual checkups to see how well schools are preparing students for college or career.”

“The state tests are designed to measure the extent to which our children are learning so that our students can compete with students around the country and around the globe,” said Andy Boy, Founder and CEO of United Schools Network, a group of high-performing charter schools in Columbus....

A report recently released by the Economic Studies program at the Brookings Institution delves into the complex process behind designing and scoring cognitive assessments. Author Brian Jacobs illuminates the difficult choices developers face when creating tests—and how those choices impact test results.

Understanding exam scores should be a simple enough task. A student is given a test, he answers a percentage of questions correctly, and he receives a score based on that percentage. Yet for modern cognitive assessments (think SAT, SBAC, and PARCC), the design and scoring processes are much more complicated.

Instead of simple fractions, these tests use complex statistical models to measure and score student achievement. These models—and other elements, such as test length—alter the distribution (or the spread) of reported test scores. Therefore, when creating a test, designers are responsible for making decisions regarding test length and scoring models that impact exam results and consequently affect future education policy.

Test designers can choose from a variety of statistical models to create a scoring system for a cognitive assessment. Each model distributes test scores in a different way, but the purpose behind each is the same: reduce the margin of error and provide a more accurate representation of...

The Olympic edition

On this week’s podcast, Alyssa Schwenk, Brandon Wright, and David Griffith discuss alternative teacher licensing in Utah and opt-out consequences in Florida. During the research minute, Amber Northern examines the lack of college readiness in Baltimore.

Amber's Research Minute

Rachel E. Durham, "Stocks in the Future: An Examination of Participant Outcomes in 2014-15," Baltimore Education Research Consortium (August 2016).

Ohio leaders have started an important conversation about education policy under the Every Student Succeeds Act. One of the central issues is what accountability will look like—including how to hold schools accountable for the outcomes of student subgroups (e.g., pupils who are low-income or African American). Ohio’s accountability system is largely praiseworthy, but policy makers should address one glaring weakness: subgroup accountability policies.

The state currently implements subgroup accountability via the gap-closing measure, also known as “annual measureable objectives.” Briefly speaking, the measure consists of two steps: First, it evaluates a school’s subgroup proficiency rate against a statewide proficiency goal; second, if a subgroup misses the goal, schools may receive credit if that subgroup shows year-to-year improvement in proficiency.

This approach to accountability is deeply flawed. The reasons boil down to three major problems, some of which I’ve discussed before. First, using pure proficiency rate is a poor accountability policy when better measures of achievement—such as Ohio’s performance index—are available. (See Morgan Polikoff’s and Mike Petrilli’s recent letters to the Department of Education for more on this.) Second, year-to-year changes in proficiency could be conflated with changes in student composition. For example, we might notice a jump in subgroup proficiency. But is...

Everyone is entitled to their own opinion, Daniel Patrick Moynihan famously quipped, but they are not entitled to their own facts. This idea animates "The Learning Landscape," a new, accessible, and engaging effort by Bellwether Education Partners to ground contemporary education debates in, well, facts.

A robust document, it’s divided into six “chapters” on student achievement; accountability, standards, and assessment; school finance; teacher effectiveness; charter schools; and philanthropy in K–12 education. Data on these topics can be found elsewhere, of course. Where this report shines is in offering critical context behind current debates, and doing so in an admirably even-handed fashion. For example, the section on charter schools tracks the sector’s growth and student demographics and offers state-by-state data on charter school adoption and market share (among many other topics). But it also takes a clear-eyed look at for-profit operators, the mixed performance of charters, and other thorny issues weighing on charter effectiveness. (Online charters are a hot-button topic that could have used more discussion). Sidebars on “Why Some Charters Fail” and case studies on issues facing individual cities lend the report heft and authority, along with discussions on authorizing, accountability, and funding. In similar fashion, the chapter on standards and...

In recent years, more and more districts have encouraged students to take Advanced Placement (AP) courses because they’re more challenging and can earn them college credit. And according to the College Board, this encouragement has translated to more course taking: “Over the past decade, the number of students who graduate from high school having taken rigorous AP courses has nearly doubled, and the number of low-income students taking AP has more than quadrupled.”

Enter a new study that examines what role grade-weighting AP courses might have played in this uptick in participation (for example, a district might assign 5.0 grade points for an A in an AP course but 4.0 grade points in a regular class).

The authors conducted a survey of over nine hundred traditional public high schools in Texas, inquiring whether they had weighting systems for AP courses; if so, when they began; and what changes have occurred in their systems since then. Twenty-eight schools that had increased their weights made up the “treatment group,” including rural, urban, and suburban schools scattered around the Lone Star State. The control group was drawn from traditional public schools with school-level data available before any weight changes occurred. It was then...

Dear Mark and Priscilla,

Please allow an aging education reformer to offer some unsolicited advice regarding the work of the new Chan Zuckerberg Initiative.

Almost twenty years ago, I wrote a long public letter to Bill Gates that drew lessons from earlier philanthropic efforts in K–12 education—including many billions of dollars wasted by the likes of Ford, Rockefeller, and Annenberg. In it, I offered suggestions for the most useful work that the then-new Bill & Melinda Gates Foundation might do in this realm, particularly by advancing the (also new) concept of charter schools.

In fact, Gates has done—and continues to do—good work in the charter sector. Much of what his foundation has undertaken in the K–12 realm, however, has fallen prey to the classic temptation to try to reform school districts. You—Mark—apparently succumbed to that same temptation when you committed $100 million to the renewal of public education in Newark, by way of both district and charter schools. Smart fellow that you are, you’ve acknowledged that the charter part of this generous gift has done some good (whereas the district part, not so much). You’ve probably read Dale Russakoff’s provocative book about what went wrong in Newark; she notes that you...

It used to be that when people talked about urban school success stories, Catholic schools were at the center of the discussion. Twenty years ago, Cardinal John O’Connor, then archbishop of New York, all but dared public school leaders to send their hardest-to-teach students to archdiocesan schools. “Send me the lowest-performing 5 percent of children presently in the public schools,” O’Connor declared, “and I will put them in Catholic schools—where they will succeed.”

Such was the audacity of urban Catholic school leaders back then. We were confident. Our schools routinely outperformed neighborhood public schools. Our results were stronger—and longer-lasting—and our success came at a bargain price.In fact, it was the historic success of urban Catholic schools that fed the reform movement in general and the charter school movement in particular. Catholic schools were proving what was possible, and entrepreneurial young education leaders were quick to seize the opportunity to do the same in the public sector.

Over the past two decades, that confident leadership has been shaken by declining enrollment and financial struggles. Some in the reform sector and elsewhere have even taken to writing off urban Catholic schools as a relic of a bygone day.

At the same time, efforts from...

Editor's note: This post reproduces a letter sent to Secretary of Education John King on July 29. 

Dear Mr. Secretary:

I am writing to suggest two very specific changes to the proposed rule that your department published on May 31, 2016, regarding the implementation of the Every Student Succeeds Act (ESSA), and its provisions for school-level accountability.

  1. For the “academic achievement” indicator mandated by ESSA, do not require states to use proficiency rates.
     
  2. Allow states to provide evidence that their proposed “other indicators of student success or school quality” are related to improved graduation, college completion, employment, civic engagement, and/or military readiness rates as alternatives to achievement.

These two tweaks will maintain the law’s strong focus on results while allowing states to develop accountability systems that are maximally fair and useful to educators, parents, and the public.

Recommendation #1: Make states report proficiency rates, but don’t require their use for the “academic achievement” indicator.

As Morgan Polikoff and dozens of scholars and policy analysts explained in a letter submitted to your department on July 22, proficiency rates are extremely poor measures of school quality. Other approaches—such as proficiency indices or scale scores—would meet ESSA’s mandate for measuring proficiency without encouraging schools...

Pages