|
No child left behind acid tests Original Source Link: (May no longer be active)
http://online.wsj.com/article/SB115379210632316123.htmlhttp://online.wsj.com/article/SB115379210632316123.html
Acid Tests
Jul 24, 2006
By Charles Murray
Test scores are the last refuge of the No Child Left Behind Act (NCLB). They have to be, because so little else about the act is attractive.
NCLB takes a giant step toward nationalizing elementary and secondary education, a disaster for federalism. It pushes classrooms toward relentless drilling, not something that inspires able people to become teachers or makes children eager to learn. It holds good students hostage to the performance of the least talented, at a time when the economic future of the country depends more than ever on the performance of the most talented. The one aspect of the act that could have inspired enthusiasm from me, promoting school choice, has fallen far short of its hopes. The only way to justify NCLB is through compelling evidence that test scores are improving..
The case that NCLB has failed to raise test scores had been made most comprehensively in a report from the Civil Rights Project at Harvard University, released just a few weeks ago. The Civil Rights Project has an openly liberal political agenda, but the author of the report, Jaekyung Lee, lays out the data in graphs that anyone can follow, subjects them to appropriate statistical analyses, and arrives at conclusions that can stand on their scholarly merits: NCLB has not had a significant impact on overall test scores and has not narrowed the racial and socioeconomic achievement gap..
The Department of Education will undoubtedly produce numbers to dispute the findings of the Civil Rights Project, which brings me to the point of this essay. Those numbers will consist largely of pass percentages, not mean scores. A particular score is deemed to separate "proficient" from "not proficient." Reach that score, and you've passed the test. If 60% of one group -- blondes, let's say -- pass while only 50% of redheads pass, then the blonde-redhead gap is 10 percentage points.
A pass percentage is a bad standard for educational progress. Conceptually, "proficiency" has no objective meaning that lends itself to a cutoff. Administratively, the NCLB penalties for failure to make adequate progress give the states powerful incentives to make progress as easy to show as possible. A pass percentage throws away valuable information, telling you whether someone got over a bar, but not how high the bar was set or by how much the bar was cleared. Most importantly: If you are trying to measure progress in closing group differences, a comparison of changes in pass percentages is inherently misleading..
Any test that meets ordinary standards produces an approximation of what statisticians call a "normal distribution" of scores -- a bell curve -- because achievement in any open-ended skill such as reading comprehension or mathematics really is more or less normally distributed. The tests that produce anything except a bell curve are usually ones so simple that large proportions of students get every item correct. They hide the underlying normal distribution, but don't change it. Thus point No. 1, that using easy tests and discussing results in terms of pass percentages obscures a reality that NCLB seems bent on denying: All the children cannot be above average. They cannot all even be proficient, if "proficient" is defined legitimately. Some children do not have the necessary skills. Point No. 2 goes to the inherent distortions introduced by the use of pass percentages: Because of the underlying normal distribution, a gain in a given number of points has varying effects on group differences depending on where the gain falls..
For our purposes, you need know only this: If the real difference between two groups, measured as it should be with means and standard deviations, remains constant, the size of the pass-percentage gap between two groups changes nonlinearly in a mathematically inevitable way. In other words, if there really is a constant, meaningful difference between groups, you can generate a curve that predicts how the point gap will change as tests are made easier or harder or as students become more or less competent..
Question: Doesn't this mean that the same set of scores could be made to show a rising or falling group difference just by changing the definition of a passing score? Answer: Yes.
At stake is not some arcane statistical nuance. The federal government is doling out rewards and penalties to school systems across the country based on changes in pass percentages. It is an uninformative measure for many reasons, but when it comes to measuring one of the central outcomes sought by No Child Left Behind, the closure of the achievement gap that separates poor students from rich, Latino from white, and black from white, the measure is beyond uninformative. It is deceptive.
|
|
