As more states adopt accountability measures in response to the No Child Left Behind Act, debates about their effectiveness have ensued. New analysis of state accountability systems finds that students in states that reward or sanction schools for their academic performance make greater achievement gains than those in states that do not.
In a forthcoming article in Education Next, Hoover fellows Margaret E. Raymond and Eric A. Hanushek present the detailed analysis of student performance they conducted. They found that better state accountability led to greater gains on the National Assessment of Educational Progress in math from 1996 to 2000.
Raymond and Hanushek's article "High Stakes Research" will be published in the summer issue of Education Next, a Hoover Institution publication. Press copies will be available April 7, 2003 on the Education Next web site: www.educationnext.org.
The controversial nature of high-stakes testing has become a lightning rod for policy debate. In this contentious environment, sound measurement and analysis are critically important to a balanced study of the effectiveness of the policy. Instead, flawed and misleading education research has been picked up by the national media and turned into front-page news.
A prime example is the recent report by Arizona State University researchers Audrey Amrein and David Berliner that purports to demonstrate that high stakes testing may actually harm rather than help students.
Unchallenged, Amrein and Berliner's study was catapulted to national prominence when it made the front page of the New York Times in December 2002.
Raymond and Hanushek also evaluated this related research and found serious flaws in Amrein and Berliner's research.
"The findings are astonishing," Raymond and Hanushek said. "Once correct statistical techniques are applied to the data they used, the results are opposite to nearly every one of their conclusions."
Raymond and Hanushek's analysis showed that test scores actually improved at a faster rate than in no-accountability states in almost all of the states where Amrein and Berliner claimed to find decreases. In New Mexico, Oklahoma, and West Virginia, where Amrein and Berliner found decreases, Raymond and Hanushek found that high-stakes testing was introduced too early to make a valid before and after comparison.
The fatal flaw of Amrein and Berliner's methods, assert Raymond and Hanushek, is their point of comparison.
"If one wants to assess the effect of high-stakes testing, the obvious comparison is between states that adopted accountability systems and those that did not. Amrein and Berliner's decision to compare the gains in high-stakes states with the national average violates a most basic principle of social-science research."
Unlike the Amrein and Berliner study, the new results were reviewed at a high-profile conference and were subject to a blind peer review for publication in a Brookings Institution volume, No Child Left Behind? The Politics and Practice of Accountability, slated for release in the fall.
Margaret E. Raymond is a research fellow and the director of CREDO, an education policy research group at the Hoover Institution that promotes the development of rigorous evidence on program performance. She has evaluated leading education programs such as Teach for America and the Edison School Project.
Eric A. Hanushek, the Paul and Jean Hanna Senior Fellow at the Hoover Institution, is a member of the Koret Task Force on K–12 Education as well as a research associate at the National Bureau of Economic Research. He is a leading expert on educational policy, specializing in the economics and finance of schools. His books include Improving America's Schools, Making Schools Work, and Educational Performance of the Poor.
The Hoover Institution, founded at Stanford University in 1919 by Herbert Hoover, who went on to become the 31st president of the United States, is an interdisciplinary research center for advanced study on domestic public policy and international affairs, with an internationally renowned archives.
