The Chronicle of Higher Education reports that a new study on that most dreaded of standardized tests, the SAT, will soon be published in the Journal of Econometrics. According to the Chronicle, the study’s author “argues that SAT scores can ‘launder’ an applicant’s background, such as race and family income.” “‘If colleges do not wish to consider a student’s background as an admissions qualification,'” he says, “‘you have to be very careful ….'”
I look forward to the article as I look forward to all articles on the SAT. I’m sure that one day I’ll find one that is fair and thoughtful, but so far the ones I’ve seen that have received pre-publication publicity have tended to be more political propaganda than anything else, even those that appear in respectable journals. Still, you can’t blame a woman for hoping.
SAT bashing has been popular over the past few years, mainly because the issue is seen as related to the controversy over race-based admissions. Nicholas Lemann’s The Big Test: The Secret History of the American Meritocracy attempted (unsuccessfully in my humble opinion) to portray the SAT as somehow sinister. Peter Sacks’ Standardized Minds: The High Price of America’s Testing Culture and What We Can Do to Change It was even more critical. Using neo-Marxist jargon, he called standardized tests, including the SAT, “a highly effective means of social control … serving the interests of the nation’s elite” and a tool for elites to “perpetuate their class privilege with rules of their own making.” At one point, the Clinton administration’s Office of Civil Rights at the Department of Education issued draft “guidelines” that would have strong-armed colleges and universities into de-emphasizing or eliminating the SAT. (They were withdrawn when an uproar ensued.)
James Bryant Conant and Henry Chauncey no doubt took a few revolutions in the grave when they heard Sacks’ accusation. They created the SAT for precisely the opposite purpose–to take privilge away from the WASPish families of the Eastern Establishment and put it into the hands of talented young people, regardless of background. And they were pretty successful. More than one Nebraska farm girl or governemnt clerk’s son from Newark has beaten out a scion of wealth and privilege for a seat at Harvard or Yale precisely because of the SAT.
Sacks called the close relationship between social class and standardized test score the “Volvo effect.” He cited a study, which he claimed found that 50% of the variability in individual SAT scores is explained by parents’ level of education. If so, that would be extraordinary. But the study he cited wasn’t about individual test scores at all, but rather school-wide average test scores. As a statistical matter, it is irrelevant to the point Sacks was trying to convey.
The error was one statisticians call an “aggregation fallacy.” Groups and individuals are different. Focusing on groups will tend to obscure important individual differences that happen to be evenly distributed among the various groups. In other words, even if individual SAT scores tend to be one-tenth social class and nine-tenths random luck, schools in wealthier neighborhoods would tend to do a bit better than those in poorer neighborhoods if only because the more important “random luck” factor would be pretty evenly distributed among all schools. Under such circumstances, the school-wide data would indicate–quite misleadingly–that social class accounts for up to 100% of the variation, while the individual data would credit it with only 10%.
Does that mean that there is no relationship between socio-economic class and SAT scores? Of course not. For whatever combination of nature and nuture, the children of successful, well-educated parents do tend to do better than less-privileged children. But the effect is not as overwhelming as Sacks argues. The notion that only those with Volvos need apply to the Ivy League is nonsense.
Moreover, insofar as the children of successful parents do score more highly, the test is measuring something real and not something that will disappear if the SAT is abolished. Such students, on the whole, don’t just tend to get better scores, they tend to do better in college too. Ignoring SAT scores just because the children of high-achievers tend to do well would be like ignoring height in basketball players just because the children of tall people tend to be tall.
Comments are closed.