EDUCATION WEEK
September 27, 2000
Standardized Testing and Its
Victims
By Alfie Kohn
Standardized
testing has swelled and mutated, like a creature in one of those old
horror movies, to the point that it now threatens to swallow our schools
whole. (Of course, on "The Late, Late Show," no one ever insists that the
monster is really doing us a favor by making its victims more
"accountable.") But let's put aside metaphors and even opinions for a
moment so that we can review some indisputable facts on the subject.
Fact 1. Our children are tested to an extent that is
unprecedented in our history and unparalleled anywhere else in the
world. While previous generations of American students have had to sit
through tests, never have the tests been given so frequently, and never
have they played such a prominent role in schooling. The current situation
is also unusual from an international perspective: Few countries use
standardized tests for children below high school age—or multiple-choice
tests for students of any age.
Fact 2. Noninstructional factors explain most of the variance
among test scores when schools or districts are compared. A study of
math results on the 1992 National Assessment of Educational Progress found
that the combination of four such variables (number of parents living at
home, parents' educational background, type of community, and poverty
rate) accounted for a whopping 89 percent of the differences in state
scores. To the best of my knowledge, all such analyses of state tests have
found comparable results, with the numbers varying only slightly as a
function of which socioeconomic variables were considered.
Fact 3.
Norm-referenced tests were never intended to
measure the quality of learning or teaching. The Stanford,
Metropolitan, and California Achievement Tests (SAT, MAT, and CAT), as
well as the Iowa and Comprehensive Tests of Basic Skills (ITBS and CTBS),
are designed so that only about half the test-takers will respond
correctly to most items. The main objective of these tests is to rank, not
to rate; to spread out the scores, not to gauge the quality of a given
student or school.
Fact 4. Standardized-test scores often measure superficial
thinking. In a study published in the Journal of Educational
Psychology, elementary school students were classified as "actively"
engaged in learning if they asked questions of themselves while they read
and tried to connect what they were doing to past learning; and as
"superficially" engaged if they just copied down answers, guessed a lot,
and skipped the hard parts. It turned out that high scores on both the
CTBS and the MAT were more likely to be found among students who exhibited
the superficial approach to learning. Similar findings have emerged from
studies of middle school students (also using the CTBS) and high school
students (using the other SAT, the college-admission exam). To be sure,
there are plenty of students who think deeply and score well on tests—and
plenty of students who do neither. But, as a rule, it appears that
standardized-test results are positively correlated with a shallow
approach to learning.
Fact 5.
Virtually all specialists condemn the practice of
giving standardized tests to children younger than 8 or 9 years old. I
say "virtually" to cover myself here, but, in fact, I have yet to find a
single reputable scholar in the field of early-childhood education who
endorses such testing for young children.
Fact 6. Virtually all relevant experts and organizations
condemn the practice of basing important decisions, such as graduation or
promotion, on the results of a single test. The National Research
Council takes this position, as do most other professional groups (such as
the American Educational Research Association and the American
Psychological Association), the generally pro-testing American Federation
of Teachers, and even the companies that manufacture and sell the exams.
Yet just such high-stakes testing is currently taking place, or scheduled
to be introduced soon, in more than half the states.
Fact 7. The time, energy, and money that are being devoted to
preparing students for standardized tests have to come from somewhere.
Schools across the country are cutting back or even eliminating programs
in the arts, recess for young children, electives for high schoolers,
class meetings (and other activities intended to promote social and moral
learning), discussions about current events (since that material will not
appear on the test), the use of literature in the early grades (if the
tests are focused narrowly on decoding skills), and entire subject areas
such as science (if the tests cover only language arts and math). Anyone
who doubts the scope and significance of what is being sacrificed in the
desperate quest to raise scores has not been inside a school lately.
Fact 8. Many educators are leaving the field because of what
is being done to schools in the name of "accountability" and "tougher
standards." I have no hard numbers here, but there is more than enough
anecdotal evidence—corroborated by administrators, teacher-educators, and
other observers across the country, and supported by several state surveys
that quantify the extent of disenchantment with testing— to warrant
classifying this as a fact. Prospective teachers are rethinking whether
they want to begin a career in which high test scores matter most, and in
which they will be pressured to produce these scores. Similarly, as The
New York Times reported in its lead story of Sept. 3, 2000, "a growing
number of schools are rudderless, struggling to replace a graying corps of
principals at a time when the pressure to raise test scores and other new
demands have made an already difficult job an increasingly thankless one."
It also seems clear that most of the people who are quitting, or seriously
thinking about doing so, are not mediocre performers who are afraid of
being held accountable. Rather, they are among the very best educators,
frustrated by the difficulty of doing high-quality teaching in the current
climate.
*
Faced with inconvenient facts such as these, the leading fall-back
position for defenders of standardized testing runs as follows: Even if
it's true that suburban schools are being dumbed down by the tests,
inner-city schools are often horrendous to begin with. There, at least,
standards are finally being raised as a result of high-stakes testing.
Let's
assume this argument is made in good faith, rather than as a cover for
pursuing a standards-and-testing agenda for other reasons. Moreover, let's
immediately concede the major premise here, that low-income minority
students have been badly served for years. The problem is that the cure is
in many ways worse than the disease—and not only because of the preceding
eight facts, which remain both stubbornly true and painfully relevant to
testing in the inner city. As Sen. Paul Wellstone, D-Minn., put it in a
speech delivered last spring: "Making students accountable for test scores
works well on a bumper sticker, and it allows many politicians to look
good by saying that they will not tolerate failure. But it represents a
hollow promise. Far from improving education, high-stakes testing marks a
major retreat from fairness, from accuracy, from quality, and from
equity." Here's why.
The tests may be biased.
For decades, critics have complained that
many standardized tests are unfair because the questions require a set of
knowledge and skills more likely to be possessed by children from a
privileged background. The discriminatory effect is particularly
pronounced with norm-referenced tests, where the imperative to spread out
the scores often produces questions that tap knowledge gained outside of
school. This, as W. James Popham argues, provides a powerful advantage to
students whose parents are affluent and well-educated. It's more than a
little ironic to rely on biased tests to "close the gap" between rich and
poor.
Guess who can afford better test preparation. When the stakes
rise, people seek help anywhere they can find it, and companies eager to
profit from this desperation by selling test-prep materials and services
have begun to appear on the scene, most recently tailoring their products
to state exams. Naturally, affluent families, schools, and districts are
better able to afford such products, and the most effective versions of
such products, thereby exacerbating the inequity of such testing.
Moreover, when poorer schools do manage to scrape together the money to
buy these materials, it's often at the expense of books and other
educational resources that they really need.
The quality of instruction declines most for those who have least.
Standardized tests tend to measure the temporary acquisition of facts
and skills, including the skill of test-taking itself, more than genuine
understanding. To that extent, the fact that such tests are more likely to
be used and emphasized in schools with higher percentages of minority
students (a fact that has been empirically verified) predictably results
in poorer-quality teaching in such schools. The use of a high-stakes
strategy only underscores the preoccupation with these tests and, as a
result, accelerates a reliance on direct-instruction techniques and
endless practice tests. "Skills-based instruction, the type to which most
children of color are subjected, tends to foster low-level uniformity and
subvert academic potential," as Dorothy Strickland, an African-American
professor at Rutgers University, has remarked.
Again, there's no denying that many schools serving low-income children
of color were second-rate to begin with. Now, however, some of these
schools, in Chicago, Houston, Baltimore, and elsewhere, are arguably
becoming third-rate as testing pressures lead to a more systematic use of
low-level, drill-and-skill teaching, often in the context of packaged
programs purchased by school districts. Thus, when someone emphasizes the
importance of "higher expectations" for minority children, we might reply,
"Higher expectations to do what? Bubble-in more ovals correctly on a bad
test—or pursue engaging projects that promote sophisticated thinking?" The
movement driven by "tougher standards," "accountability," and similar
slogans arguably lowers meaningful expectations insofar as it relies on
standardized testing as the primary measure of achievement. The more that
poor children fill in worksheets on command (in an effort to raise their
test scores), the further they fall behind affluent kids who are more
likely to get lessons that help them understand ideas. If the drilling
does result in higher scores, the proper response is not celebration, but
outrage: The test results may well have improved at the expense of real
learning.
Standards aren't the main ingredient that's in low supply.
Anyone
who is serious about addressing the inequities of American education would
naturally want to investigate differences in available resources. A good
argument could be made that the fairest allocation strategy, which is only
common sense in some countries, is to provide not merely equal amounts
across schools and districts, but more for the most challenging student
populations. This does happen in some states—by no means all—but, even
when it does, the money is commonly offered as a short-term grant (hardly
sufficient to compensate for years of inadequate funding) and is often
earmarked for test preparation rather than for higher-quality teaching.
Worse, high-stakes testing systems may provide more money to those already
successful (for example, in the form of bonuses for good scores) and less
to those whose need is greatest.
Many public officials, along with like-minded journalists and other
observers, are apt to minimize the matter of resources and assume that
everything deficient about education for poor and minority children can be
remedied by more forceful demands that we "raise the bar." The implication
here would seem to be that teachers and students could be doing a better
job but have, for some reason, chosen not to do so and need only be bribed
or threatened into improvement. (In fact, this is the tacit assumption
behind all incentive systems.) The focus among policymakers has been on
standards of outcome rather than standards of opportunity.
To make matters worse, some supporters of high-stakes testing have not
just ignored, but contemptuously dismissed, the relevance of barriers to
achievement in certain neighborhoods. Explanations about very real
obstacles such as racism, poverty, fear of crime, low teacher salaries,
inadequate facilities, and language barriers are sometimes written off as
mere "excuses." This is at once naive and callous, and, like any other
example of minimizing the relevance of structural constraints, ultimately
serves the interests of those fortunate enough not to face them.
• Those allegedly being helped will be driven out. When rewards
and punishments are applied to educators, those who teach low-scoring
populations are the most likely to be branded as failures and may decide
to leave the profession. Minority and low-income students are
disproportionately affected by the incessant pressure on teachers to raise
scores. But when high stakes are applied to the students themselves, there
is little doubt about who is most likely to be denied diplomas as a
consequence of failing an exit exam—or who will simply give up and drop
out in anticipation of such an outcome. If states persist in making a
student's fate rest on a single test, the likely result over the next few
years will be nothing short of catastrophic. Unless we act to stop this,
we will be facing a scenario that might be described without exaggeration
as an educational ethnic cleansing.
Let's be charitable and assume that the ethnic aspect of this perfectly
predictable consequence is unintentional. Still, it is hard to deny that
high-stakes testing, even when the tests aren't norm-referenced, is
ultimately about sorting. Someone unfamiliar with the relevant
psychological research (and with reality) might insist that raising the
bar will "motivate" more students to succeed. But perform the following
thought experiment: Imagine that almost all the students in a given state
met the standards and passed the tests. What would be the reaction from
most politicians, businesspeople, and pundits? Would they now concede that
our public schools are terrific—or would they take this result as prima
facie evidence that the standards were too low and the tests were too
easy? As Deborah Meier and others have observed, the phrase "high
standards" by definition means standards that everyone won't be able to
meet.
The tests are just the means by which this game is played. It is a game
that a lot of kids—predominantly kids of color—simply cannot win. Invoking
these very kids to justify a top-down, heavy-handed, corporate-style,
test-driven version of school reform requires a stunning degree of
audacity. To take the cause of equity seriously is to work for the
elimination of tracking, for more equitable funding, and for the universal
implementation of more sophisticated approaches to pedagogy (as opposed to
heavily scripted direct-instruction programs). But standardized testing,
while bad news across the board, is especially hurtful to students who
need our help the most.
|