Nicholas Lemann.
The Big Test: The Secret History of the American Meritocracy.
Farrar, Straus and Giroux. $27.00
Peter Sacks.
StandardizedMinds: The High Price of America’s Testing Culture.
Perseus Books. $26.00
Attacks on standardized testing are nothing new. In fact, it is surprising that the tests, given their grim task of ranking people according to ability, enjoy as much public acceptance and support as they do. Authors Nicholas Lemann and Peter Sacks would like to change that.
In Lemann’s view, selective college admission based on the Scholastic Assessment Test (SAT) has generated a smug, self-perpetuating elite, inimical to equal opportunity. Lemann finds the very idea of aptitude testing offensive, because, as he argues plausibly, tests never tap innate abilities, only developed mental abilities much affected by exposure to schooling and middle-class upbringing. Sacks agrees, but develops his own, more broad-based critique, not confined to the SAT. According to Sacks, all forms of multiple-choice testing fail to gauge individual potential. In his view, achievement tests are especially pernicious, because they foster a narrow, test-oriented style of teaching, harmful to the education process itself.
Lemann’s book is an entertaining read, chock-full of colorful characters and incidents, but as an argument, it doesn’t add up. He shows that some individuals who played key roles in the origins of testing held crackpot or elitist ideas. For example, Educational Testing Service (ETS) founder Henry Chauncy hoped to develop tests not just of scholastic ability, but of creativity, practical judgment, persistence, etc. Another early proponent of the SAT, Harvard president James Conant, wanted to limit the number of students attending college. But these villains seem too ineffective to be really sinister. Nothing much came of Chauncy’s enthusiasm for quantifying every conceivable human trait, and in the 1940s, the masses went to college anyway, thanks to the GI Bill. Lemann’s style of argument here is pure guilt by association, and as unconvincing as such arguments usually are.
Lemann sets great store by the fact that colleges adopted the SAT "outside the purview of politics and open debate." This is not as telling as he thinks. The pre-SAT system, based on recommendations from headmasters of private New England boarding schools, was certainly not founded in democratic debate. But more importantly, as Lemann shows, the sat was vigorously contested, as one might expect, by the soon-to-be-displaced elites who had benefited from the traditional methods of recruitment. From the evidence Lemann presents, it is apparent that colleges turned to the SAT out of rivalry with each other, as each institution feared getting stuck with a mediocre student body. Standardized testing gained importance in college admissions through a competitive, pluralistic process, not a conspiratorial one.
In Lemann’s view, SAT scores should play only a minimal role in college admissions, because they explain only about 16 percent of the variation in first term freshman grades. Technically correct, but ignoring the importance of "restriction of range," Lemann is wrong to dismiss this as a "slender achievement." Most colleges choose from the applicant pool in part by scores. It is within that selected group — and in some colleges, this is a very selective group indeed — that scores are of diminishing utility in predicting freshman grades. The results might look very different, as admissions offices are keenly aware, were students with much lower scores admitted. Of course no test can measure a student’s motivation; colleges know that they must use other sources of information to assess this quality, so critical to success in any endeavor.
Lemann frequently alludes to the charge that testing is culturally biased against minorities. Despite its lack of merit, this is an accusation still frequently hurled at the SAT. To his credit, Lemann acknowledges, however obliquely, that, at least with respect to black-white differences, which have been intensively studied, the SAT predicts the first term grades of black college students as well as it predicts the grades of white ones.
The most exasperating weakness of Lemann’s critique of the SAT is that he never shows a better method of selecting the students who would most excel at college. His one proposal — achievement testing based on a national curriculum — is so far off the mark that one must question his seriousness. Until we learn methods to improve the schooling of disadvantaged children, achievement tests will not result in a different pattern of class recruitment. The same privileged kids with well-educated parents will outperform everyone else, just as they do now. There is nothing odd about this; it would be odd if all the efforts of parents who already have education and therefore value it — speaking English correctly at home, taking children to the library and museums, showing concern about homework and grades, and a hundred other things — did not have a measurable impact on student performance. Lemann’s idea is that making "decent early education" a federal responsibility will quickly negate the effects of parental upbringing; that’s a leap of faith running counter to existing evidence.
Compared to Lemann, Sacks makes more worthwhile criticisms, if not precisely of standardized tests, then of the way we sometimes use them. Like Lemann, Sacks frequently overstates his case, and relies heavily on anecdotal evidence, but he also draws on a broader range of social science research on education.
Like Lemann, Sacks views the SAT as a major stumbling block to the economic advancement of every young person whose parents do not drive a Volvo. But in contrast to Lemann, who sees achievement measurements as a desirable alternative, Sacks views multiple-choice achievement testing as equally if not even more flawed and harmful. According to Sacks, not only do achievement tests fail to measure learning, but even worse, they tend to narrow and "dumb down" classroom teaching.
Sacks is a proponent of what he calls "authentic" or "performance assessment," which he does not clearly define, but which focuses on, for example, "writing, speaking, building, drawing, solving, synthesizing and analyzing." One suspects that Sacks likes "performance assessment" because it is murky and subjective, and thus will hinder the ability of large educational institutions to rapidly compare thousands of individuals. But Sacks may be right that some students are best motivated by a teaching style oriented toward projects with real world applications, such as building an electric car, or designing an "herbal soft drink" container. And he may be right that standardized testing leads some teachers to neglect the excitement of intellectual exploration in favor of drill in poorly designed "test prep" booklets. On the other hand, not all students find projects such as making peanut butter, or writing an essay from the perspective of a growing fetus, to be a good use of their time. One cannot help wonder why parents do not have more choices in education, why students must suffer one-size-fits-all schooling.
A puzzling omission in Sacks’s book (and Lemann’s too) is any discussion of the achievement tests administered by ETS, the so-called SAT II. Since these widely used tests measure acquired knowledge, they escape Lemann’s objections to the fetish some people make of aptitude. If the ETS tests are, as one suspects, quite good indicators of subject mastery, Sacks has a responsibility to say so.
The most worthwhile point that Sacks makes pertains to "high stakes testing," in which substantial rewards and penalties are attached to perforsents disturbing evidence that teachers adjust their instruction to raise scores on particular tests, but that the gains evaporate once a different brand of test is administered. Public frenzy over test scores can lead to intensive coaching and other more dubious, even unethical practices. These may explain, for example, the short-lived "Tacoma miracle," a one-time spike in scores in spring 1995 that helped catapult Rudy Crew to the post of chancellor of New York City schools.
But what should we do about schools in which children do not learn? Surely decency requires that local school boards acknowledge failure, and provide parents with more options. Sacks ought to see this, but somehow doesn’t. Yet conservatives in particular should give careful thought to Sacks’s argument that "high stakes testing" is too easily manipulated and may end up merely punishing low-income students. Considered from both a philosophic and a pragmatic angle, should the first choice of conservatives be "incentives" doled out from on high by education bureaucrats, using "indicators" that remind one of Soviet central planning? Would not vouchers and charter schools, which afford parents choice, do more to encourage the experimentation needed to find out which approaches truly work best with which children?
For reasons that have little to do with the arguments in Lemann’s and Sacks’s books, standardized aptitude testing has already begun to play a declining role in college admissions. Liberal discomfort with racial disparity has always been a danger to standardized testing, yet for a long time, the SAT, despite its hefty "disparate impact" on black Americans, largely escaped liberal assault. The explanation is that universities have tended to, as Lemann puts it, "simply bend the rules of the meritocracy for Negroes." Most colleges (whether they acknowledged it or not) have employed dual systems favoring the admission of certain groups of minority students, despite lower test scores than other applicants.
Attacks on the SAT are particularly timely now, however, precisely because racial preferences in admissions are very much under challenge. One court case, Hopwood v. Texas (1996), ended the dual admissions system in public education in that state. Another strong blow was dealt in 1996, when California voters rejected preferential admissions in a popular referendum (Proposition 209).
But "race-neutral" standards in admissions, a seeming victory for "meritocracy," do not signify the firm entrenchment of standardized testing. Quite the opposite is true. In both Texas and California, to bolster the minority presence on campus, admissions offices have de-emphasized test scores, going back to the more traditional method of selecting students — according to class rank, grades, and recommendations. This has not been an easy transition, and it is important to understand why. Standardized test scores, unlike grades, are indeed "standardized." Similar scores generally reflect similar aptitudes, or in the case of achievement tests, similar mastery of subject matter. Scores may not measure a student’s "merit" (even if that kind of abstract language, with its implications concerning human worth, is sometimes used), but they are fairly reliable indicators of preparation to do demanding college-level work, which is why admissions offices began to use them in the first place.
Is excellence in education threatened by a lesser role of test scores in admissions? Few would deny that voters can reasonably require a state university system to serve the children of the taxpayers who fund the system. But admission of a legislated percentage of graduating high school students (whether the top 4, 10, or 20 percent), when high schools are of varying quality, means admitting more students who are unprepared for college level work. This in turn requires either spending on remediation programs, flunking out large numbers of students, or diluting educational standards — and very possibly some combination of all three. If we are lucky and smart, greater reliance on high school transcripts will stimulate outreach efforts by state universities to improve curriculum and teaching at the high school level. The risk is that public institutions of higher education will relax their own standards of excellence. In that case, the attempt to increase opportunity for some will come at the cost of lessening the educational achievement of others — a destructive form of social leveling which, if widely practiced, would serve no worthwhile national purpose.
It is interesting that the same animus that led Lemann and Sacks to write their books — resentment against a system that seems to reward those already advantaged by middle-class birth — can lead to such strikingly different prescriptions. Lemann advocates a national curriculum with achievement measured, presumably, by standardized tests. Sacks is at least more consistently hostile to standardized testing; but then, he believes that an "A" earned in one school indicates as much learning, and capacity to learn, as an "A" in any other. Most of us will part company with him there.
People involved in education reform often divide between those who favor state-mandated "standards" and those who place their hopes in vouchers and charter schools. The standards movement tends to rely on government action and to highlight failure; the voucher and charter schools movement tends to rely on private initiative and to emphasize choice. Lemann’s book assumes, rather than proves, the malignance of aptitude testing. But Sacks does succeed in raising doubts about the meaningfulness of many commonly used achievement tests, especially in a high-stakes setting. Reading the two books together suggests that effective education reform depends on preserving, and building on, the decentralization and experimentalism that currently characterize American education.
