EDUCATING BY NUMBERS: Standards, Testing, and Accountability in Education

Wednesday, January 9, 2002

Will standards-based testing and accountability improve our nation's education system? In January 2002, President Bush signed into law the Elementary and Secondary Education Act of 2002. The act calls for a mandatory annual test in reading and math for every child in the nation in the third through eighth grades. Schools that fail to improve their students' scores may be held accountable, possibly losing some federal funding. Supporters of the act say that standards-based testing and accountability are the best ways to monitor and improve the nation's schools. Opponents say that such a regime is largely a political ploy that will do more harm than good. Who's right?

Recorded on Wednesday, January 9, 2002

Peter Robinson: Today on Uncommon Knowledge, a novel idea for America's schools: testing, standards and accountability.

Announcer: Funding for this program is provided by the John M. Olin Foundation and the Starr Foundation.


Peter Robinson: Welcome to Uncommon Knowledge. I'm Peter Robinson. Our show today: testing every school child in America.

In January, President Bush signed into law the Elementary and Secondary Education Act of 2002 or as he prefers to call it, the Leave No Child Behind Act of 2002. The Act mandates testing on every school child in America in reading and math from the third to the eighth grades. Schools that perform very badly may be held accountable, possibly losing some federal funding. Proponents of the Act say that mandatory testing is the best way to monitor our schools and hold them accountable. Opponents of the Act say, phooey. Testing is just a political ploy and it might actually do our children more harm than good. Who's right?

With us today, two guests. Elliot Eisner is a Professor of Education at Stanford University. Williamson Evers is a Fellow at the Hoover Institution and a former Education Advisor to President George W. Bush.

Title: Educating by Numbers

Peter Robinson: A poll by the organization Public Agenda, taken in the year 2000: ninety percent of employers, ninety-two percent of professors, eighty-one percent of teachers and eighty-three percent of parents believed that establishing standards for what students are expected to learn, helps to improve academic performance. That's on the one hand. Here's the other hand. The former educator, now journalist, Alfie Kohn, "If there has ever been a more profoundly undemocratic school reform movement in U.S. educational history then what is currently taking place in the name of standards, I haven't heard of it." Who are you with, the American people or Alfie Kohn? Elliot?

Elliot Eisner: Well it's more complicated than that and any answer that I can give you directly to your question would be inadequate. Depends what you mean by standards.

Peter Robinson: Okay. We'll elucidate that as we go along. Bill?

Williamson Evers: I think I'm with the American people on this.

Peter Robinson: You took the easy way out. All right, Congress enacted, President Bush has signed something called the Elementary and Secondary Education Act. He prefers calling it the No Child Left Behind Act. Main provisions: it increases federal spending on education by some forty percent to over twenty-six billion dollars and it will require all students to be tested annually in reading and math in grades three through eight. Now I will ask you both to elucidate a great deal in the course of this program. But first tell me how it is that Senator Edward Kennedy and President Bush can both have hailed this bill? Bill?

Williamson Evers: It's because the public believes that standards and tests will be a way to improve our public schools. And both of them are in politics.

Peter Robinson: Elliot?

Elliot Eisner: Well the reasons for their endorsing the bill I think is because the American public believes that testing and standards will improve the quality of education.

Peter Robinson: Okay. So they're looking at that poll data that I quoted to you.

Elliot Eisner: I think that's right and I think that that conception is inadequate. It's inadequate as a conception of what counts for quality education and it is inadequate as an understanding of what schools need to become in order to be genuinely educational.

Peter Robinson: Okay. Let me ask you to listen to a quotation and comment on that. Former Education Secretary, William Bennett, "The standards, testing and accountability," that's the mantra, standards, testing and accountability, those three words, "of the legislation make it a modest improvement over current law." Modest improvement? How do you rate the legislation?

Elliot Eisner: The legislation is a plus, 26.5 billion dollars for education is a plus but, you know, you cannot buy quality education. You have to develop quality education. We have been through this before with standards and objectives and I could trot you through the history of education in the twentieth century to show where we've tried to do essentially the same thing without the results that we had anticipated, whether you want to start…

Peter Robinson: Let's look at standards, testing and accountability in turn, beginning with standards.

Title: Putting Standards to the Test

Peter Robinson: Once again, the former educator, now journalist, Alfie Kohn, "Considerable research has demonstrated the importance of making sure students are actively involved in designing their own learning, invited to play a role in formulating questions and so on. The more students are excluded from this process, that is Congress makes up standards or state legislatures make up standards, the more students are excluded from this process, the more alienated they tend to become, thus standards may actually have the effect of dumbing down classrooms." Your position roughly?

Elliot Eisner: They could have that effect. It's called de-skilling. If you think about the assignment of goals coming from elsewhere for teachers, the role of the teacher is as technician, namely to implement practices that will realize educational goals that other people prescribe. And…

Peter Robinson: So this is the teacher as a figure on the assembly line, the big boss up above decides what the teacher's supposed to do to each child as it whizzes by and the teacher just sits there turning screws…

Elliot Eisner: That's a characterization of it. Plato said a slave is someone who executes the purposes of another. The provision of professional discretion, making it possible for teachers to exercise professional judgment in terms of what youngsters need, I think is a part of being a professional.

Peter Robinson: Bill, you have just heard standards trashed.

Williamson Evers: Well I think standards, which tend to be not written by presidents or congressmen…

Peter Robinson: Under this legislation…

Williamson Evers: They're not written by presidents or congressmen.

Peter Robinson: Who establishes the standards, not the President, not Congress, state legislatures?

Williamson Evers: They're done at the state level but they're delegated to people who put them together.

Peter Robinson: So just walk me through. So each state legislature has or state government, governor and legislature has discretion about establishing standards and they'd likely--they could choose to permit standards to be established by each of their own school districts or a State Board of Education, it's up to them?

Williamson Evers: Well there are fifty states and there's going to be some variety but the basic thing that is done is some group of people, community leaders and teachers, get together under the auspices of the state government and they establish a catalogue of topics for each grade that the children are supposed to learn in the different subject matters.

Peter Robinson: Okay, now to Elliot's point. This establishing standards, rules coming down from on high will alienate students?

Williamson Evers: Elliot's right that there should be a certain amount of professional discretion in teaching in the classroom. And I think he has been right in the past to criticize excesses of mechanistic application. But I think perhaps he's bending the stick too far here. Standards that are reasonably general, that say the student should know something about quadratic equations, a student should know material about the American Civil War, things like this, are essentially benchmarks or checklists or whatever of topics that help people writing textbooks, that help teachers, that help…

Peter Robinson: This sounds very sensible.

Williamson Evers: …people. And then the teacher has to apply that in the classroom knowing something about the students in the class, knowing about her strengths, what she can best do in dramatizing this and conveying information.

Peter Robinson: There was some poll, I can't remember where it was but it made the popular press that high school students couldn't name the years of the Civil War to within several decades. I mean, isn't it just sensible on the face of it to say that by the time you graduate from ninth or tenth grade, you ought to know when the Civil War took place?

Elliot Eisner: That's an un-endable list.

Peter Robinson: Un-endable list?

Elliot Eisner: Un-endable list to think about education that way. Now when we talk about standards, if we're talking about values, that this is something that we embrace, we want students to learn to think critically about issues in American society. That's a value that I think is important if that's what we mean…

Peter Robinson: Critical thought?

Elliot Eisner: Critical thought.

Peter Robinson: But he's talking about catalogues of things that kids ought to know by the time they reach a certain grade. Right?

Williamson Evers: That's right.

Peter Robinson: Okay.

Elliot Eisner: That's a catalogue that you can't complete and that's not, in my view, the way we should think about education. We should be thinking more about the kinds of questions students are able to ask after studying a unit in the social studies than perhaps as much as certainly as we ask them to have answers about what it is that they've studied.

Peter Robinson: Elliot isn't budging on this notion of standards but it just makes common sense to me. Let me press him a little bit more.

Title: Making a List and Checking It Twice

Peter Robinson: There may be dickering over what's on the list and what isn't on the list but it just seems as though you ought to be able to come up with a reasonably concrete set of items, they ought to know about the Civil War, they ought to know the place in American history of Abraham Lincoln and George Washington and so forth. No? You'd reject any effort in that direction?

Elliot Eisner: I would reject any effort that comes up with highly specific lists of particulars which will paralyze teaching and teachers and which will lead to emphasis on rote memorization of material that is not connected to any larger idea. What I'm talking about, as being needed in American education is the creation of a culture of schooling in which the intellect matters, in which curiosity is peaked.

Peter Robinson: Bill, you've written about what you term progressives and traditionalists. In other words, one thing we seem to be uncovering here is just fundamentally different philosophies of education. Can you just describe what you mean when you say progressive and traditionalists?

Williamson Evers: Let me talk about what I think our colleague here Elliot is saying. He's saying that the teacher should be an artist. The teacher should go into the classroom and paint, in a sense, a picture there. He should do things that are fun to him or to her as a teacher, enlist out of that energy, out of that excitement, get the children interested. Part of what he wants and he's described this himself in his writings, is a license to explore a chance at idiosyncratic exploration. And the…

Peter Robinson: And the contrary view, the traditionalist view is…

Williamson Evers: …so he's against the idea that there be specific enumerations, a specific catalogue of a sense of things that are culturally necessary to operate successfully in the society and to be a cultivated person.

Peter Robinson: Okay. In your view, you write about the traditionalist view believing in a systematic sequential teacher led instruction.

Williamson Evers: Yeah I do think that's--that is what the traditionalists say.

Peter Robinson: It is a matter of getting information into kids' head. Okay, so let me ask you this. If you have on the one hand the progressive view, broadly speaking, what Elliot is describing here and the traditionalist view, which in your judgment, dominates American teaching as a profession in the year 2002?

Williamson Evers: It's very eclectic in reality.

Peter Robinson: Is it?

Williamson Evers: Yes, because you can still think progressives are in the leading institutions, they run the teacher colleges.

Peter Robinson: Okay. So let me ask this question then. If then that's the case, how is it that the government by mandating testing is in effect choosing one philosophy as education over another an essentially attempting to ram it down the throats of an educational establishment that just doesn't believe in it?

Williamson Evers: Well I think what you have here is you do have some differences between the general public, between governors and state legislatures and the educational establishment. But even among progressives…

Peter Robinson: Do you buy that that the education--that there is just this divide between what the public wants and…

Elliot Eisner: I think there's a blanket characterization, which really doesn't hold.

Williamson Evers: The key thing to understand here is that even some progressives think and I would name John Dewey as one of them, that there is a body of knowledge in the culture that you want children to learn. And what those progressives argue for is fashioning problem solving situations that will lead the children to learn this material.

Elliot Eisner: Amen. Amen. Agree exactly. John Dewey said, as a matter of fact, that one of the largest problems with progressive education, which was taking place in the 1930's, was inadequate attention to the sequential development of subject matter. So the organization of subject matter is terribly important. But the question is how you get kids into it so that they're not drilled and killed by trying to stuff their heads with facts, which are going to be tested sometime in May.

Peter Robinson: Okay. Now from standards to testing.

Title: Testing is a _____ Idea (Fill in the Blank)

Peter Robinson: The idea here is again, it seems to me, common sense. You establish standards then you enforce them by giving the kids tests. And the tests have consequences. If students flunk certain tests, consequences follow including consequences being held back until they do master the material. As I say, that seems to me on the face of it a sensible approach. Elliot?

Elliot Eisner: Well you have to ask what tests are for. In the measurement community, there's something called predictive validity where you take scores and it allows you to make predictions about what people will do in other circumstances. The tests that are being employed have very little predictive validity outside of other test scores. But what we ought to be looking for if we want to evaluate the quality of education, we need to look at what kids do in classrooms. We need to do what teachers do with youngsters. We need to do--we need to deal with the kinds of thinking and the sorts of tasks that they engage in. The problem is more complicated than the public understands. And what I think we do is to get into a highly reductionistic approach to education, we cram their heads with facts, we test them that we're going to, in effect, bully them into performance.

Williamson Evers: I think, you know, I work on tests myself specifically, for example, here in California, I work on the history test that's given to all school children in the high school and soon to be also a middle school grade. And I work on the math test that's given grades two through eleven. These questions that we put on the test reflect certain kinds of, you know, in history, what do they know about the Women's Suffrage Movement. In math, what do they know about dividing fractions, things like this? Now I think this is something we want to know if the children know. And we want to know by adding up the score, something about how well the teachers are doing. And the more test results you have, the more accurate a picture you'll have of what's going on in the place you're looking at.

Peter Robinson: So there may be crudities here and there but over time and as you aggregate test scores, they actually give you very valuable information.

Elliot Eisner: You know something; I was in Nebraska a couple of years ago giving a lecture on educational evaluation. After the lecture was over, a woman came up to me to the podium and said, Professor Eisner, I really liked your lecture but, you know, we're here from Nebraska and we raise cattle in Nebraska and one thing that we've learned is that you can't fatten cattle by putting them on a scale. You have to pay attention to the diet. You have to pay attention to the environment. You got to pay attention to how they're related to. That's what we need to do in schools. We need to pay attention to what goes on, where the rubber hits the road…

Peter Robinson: By putting the cattle on the scale, you can find out in a hurry which farmers are fattening the cattle and which are not. You can discover which techniques work in fattening the cattle. It does provide very valuable information.

Elliot Eisner: But it also can distort the program of education youngsters have access to. It can effect priorities.

Williamson Evers: People can misuse it just like they can misuse anything.

Elliot Eisner: Well we have to take that into account I think…

Williamson Evers: Absolutely.

Elliot Eisner: …when we make educational policy and we ought not to lead the public to believe that these indicators are adequate for making judgments about the quality of education because you can raise test scores and diminish the quality of education students receive.

Williamson Evers: He's saying essentially we should operate like connoisseurs in looking at what's going on in the classroom. And there's a reason why. In it's report of several months ago, the National Research Council of the National Academy of Science suggested that Professor Eisner's approach of connoisseurship will not lead to scientifically accurate accounts of what's going on in the classroom and whether children are learning. And I think his approach leaves us in the following situation. If the teacher is just to be a kind of artist and if the student is to join in and become a fellow artist, how can we suggest which is a good textbook to buy. What is an effective reading practice? We are at a loss and different people will claim different things about connoisseurship under this circumstance. I believe he is looking at a kind of policy nihilism here.

Peter Robinson: We've tackled standards and testing so let's go last to accountability.

Title: A is for Accountibility

Peter Robinson: Federal government just committed itself to spending twenty-six and a half billion dollars but that's a very small fraction of the amount that the nation, state governments, local government, we spend tens of billions of dollars a year on education, public education, year in and year out. And the notion is simply this; taxpayers would like to be able to have some idea what they're getting for their money. They are being taxed here in California, what is it, what percentage of the general fund goes to education each year? It's half isn't it? Some huge percentage, billions of dollars out of my pocket, out of every Californian's pocket. And when I listen to you, it sounds as though all I'm allowed to do is listen to an individual teacher say well I think the kids are doing well. It becomes that subjective.

Elliot Eisner: No, reasons are given, judgments are made. If you think, for example, the way we employ professors at Stanford, we use our connoisseurship. We look at presentations that they make. We read the material that they write and publish and we make judgments about its quality. And we discuss those judgments with our peers, etc. And we finally arrive at a judgment to offer a position. Now that's a kind of a sophisticated higher education approach but the point is that teachers are employing connoisseurship when they make judgments about when to intervene, when not to intervene, what kind of support to provide. I mean, this is a ubiquitous process in human life and what we ought to be doing is cultivating the ability of teachers to make those kinds of judgments.

Peter Robinson: Let me turn the question just a slightly different way; stay on the same topic of accountability. So if this notion of standards and testing and accountability is the wrong way to go, what reform would you make?

Elliot Eisner: You begin with teachers and getting teachers together with a principal that can exercise some leadership and can promote the kind of deliberation that makes it possible for changes to be made in schools, which are not trivial changes.

Peter Robinson: And what's the lever for a governor or a state legislature to go after teachers? Is it money? Is this more spending?

Elliot Eisner: There certainly can be incentives that make it possible for teachers to come together during the summer to work on materials, to work on projects, to work with each other during the school year. What we're trying to do is to buy a silver bullet that will take these schools which we already know are not doing well and we're saying, here's the test scores and now what you've got are two or three years to get better otherwise we're going to send your students out to a private enterprise.

Peter Robinson: Accountability, Bill?

Williamson Evers: I think if we don't take the temperature of the patient, if we don't take the blood pressure of the patient, we don't know what's going on and we don't know what the disease is. I think it's true that there is a tendency in education to look for silver bullets and look for magic and look for a fad or a panacea. And I think that Professor Eisner is right to caution us to be careful in this and to not lose sight of the intellectual aspects of this and not lose sight of important content. But I think he's bending the stick too far.

Peter Robinson: Final topic. Where is all this going?

Title: Choose Your Future

Peter Robinson: Now the current legislation that the President signed, the penalties imposed on schools that perform badly are not exactly draconian. There's some thought that they might lose some federal funding and parents might be permitted some school choice but only if their kids are in a school that's performing egregiously badly. They may choose another school, public school, in the same school district, which is a pretty limited version of choice. I put to you Margaret Spellings who was then Governor Bush's education aide in Texas. "The whole school choice movement will flourish when people have data about how schools are doing." So she seems to be suggesting that this is a first step, that the information in itself will create a political dynamic that will give parents greater choice about which schools their kids attend. What do you think?

Williamson Evers: Well I think it's important to get the information per se. I think that it's important to be doing business in daylight here. I think it's worth having a spotlight shine on what exactly is going on, where students are not really learning and where teachers are falling down and need some additional training or need to be encouraged to go into some other line of work. And I think that that--the future of our children, not having them cheated by having an inadequate education, I think is the key thing here.

Peter Robinson: But under this current legislation, parents who discover that their children are indeed being cheated because they're stuck in a bad school don't really have much recourse.

Williamson Evers: We need to work on that problem. But…

Peter Robinson: So do you suspect or hope or both that the information will itself create a kind of appetite on the part of parents that will lead to some political solutions in the direction of greater choice?

Williamson Evers: I think that, in fact, we're moving in a direction of greater pluralism, whether or not the standards and accountability aspect of it is driving it or affecting it. And secondly, I think the standards and accountability are worth defending on their own merits, whether or not they happen to be useful to the choice people because I think it's important to know where you're falling down on the job.

Peter Robinson: Okay, so we have these tremendous amounts of information being produced. More will be produced. It'll find its way into the hands of parents and Bill says this is a good thing.

Elliot Eisner: Well it could be a good thing if parents interpreted the material in a way that's valid. But my concern is that they will take these very, very partial indicators of educational quality and regard them as a summum bonum of educational practice and they are not. So they need other kind of information. They need baseline data. They need work samples. They need communications from the school to the community. Again, I think what we try to do is to reduce this into some kind of mechanical mechanism…

Peter Robinson: Do you see a danger, Elliot, that the parents will harass already harried teachers? That is to say, that teachers who are already in very difficult situations in inner cities…

Elliot Eisner: They may indeed.

Peter Robinson: …suddenly their parents will come armed with testing data…

Elliot Eisner: Well not only in the inner city but it's not unheard of that schools will create situations where teachers may, in fact, cheat by having students absent who are going to be taking these exams, who are not likely to do well.

Williamson Evers: I hope that parents do come armed with test data to the school board and say, why isn't my child getting an education?

Peter Robinson: Okay. Last question. Seven years from now when President Bush is finishing his second term, should he get a second term, will the nation have recoiled from this experiment in standards, testing and accountability or will it have tended to lead to still more information and still more school choice? Is this a seed that will flower or is it something we'll want to throw in the rubbish? Bill?

Williamson Evers: Well I think it will continue. I think that the public wants knowledgeable, cultured citizens. And we don't have this when we have substantial numbers of kids that are illiterate.

Peter Robinson: So this is only the beginning. Elliot?

Elliot Eisner: If the history of education in America is any indication, this is going to blow over. And I would hope that there would be developed a deeper understanding of what the educational process is about in communities around the United States so that better questions can be asked and better expectations can be formed for what schools should try to accomplish.

Peter Robinson: It'll blow over. Elliot, Bill, thank you very much.

Peter Robinson: I'm Peter Robinson for Uncommon Knowledge. Thank you for joining us.