In the New Yorker, well-known writer Malcolm Gladwell has an article presenting his own law school ranking system. Gladwell is highly critical of the US News rankings, and believes his approach is superior (fuller, subscription-only version here):
There’s no direct way to measure the quality of an institution — how well a college manages to inform, inspire, and challenge its students. So the U.S. News algorithm relies instead relies instead on proxies for quality — and the proxies for educational quality turn out to be flimsy at best. …
[The U.S. News rankings don’t] include price. Both its college rankings and its law-school rankings reward schools for devoting lots of financial resources to educating their students, but not for being affordable. Why? [Robert] Morse admitted that there was no formal reason for that position. It was just a feeling. … U.S. News thinks that schools that spend lots of money on their students are nicer than those that don’t, and that this niceness ought to be factored into the equation of desirability. …
[G]iven that the rising cost of college has become a significant social problem in the United States in recent years, you can make a strong case that a school ought to be rewarded for being affordable. So suppose we… re-rank law schools based on … a three-factor ranking, counting value for the dollar at 40%, LSAT scores at 40%, and faculty publishing at 20% . …
Like Gladwell, I have been very critical of the US News ranking system. In a 2006 post, I too made the point that the US News formula perversely rewards schools for inefficient spending (if School A and School B are exactly the same, except that A spends more money, US News rewards A for its inefficiency).
I therefore welcome Gladwell’s elimination of this error. I also welcome his focus on student and faculty quality. These are by far the most important elements of any law school. Obviously, I am also happy that George Mason ranks very high in Gladwell’s survey (19th), ahead of big-name schools such as Cornell, NYU, and Berkeley. That’s more than 20 slots better than our latest US News ranking. I don’t believe that we actually deserve to rank ahead of all the schools that Gladwell ranks lower than us. But I do think that we are underrated by US News. Our success in Gladwell’s ranking reflects our high student LSAT scores (a median of 164 in 2010), and the high productivity of our faculty, which is also evident in Brian Leiter’s ranking of faculty quality. GMU also does well on Gladwell’s tuition measure, though the University has forced us to raise tuition in recent years.
At the same time, I have some reservations about Gladwell’s heavy emphasis on tuition. I think it both overvalues tuition relative to other factors and may unintentionally confuse applicants by heavily emphasizing a factor they can easily judge for themselves, thereby making it more difficult for them to assess a school’s ranking on other dimensions. For example, the no. 2 ranked law school by Gladwell’s methodology is BYU. It beats schools like Harvard and Yale (ranked 3 and 4) because of its much lower tuition. Let’s assume that choosing BYU over Yale or Harvard saves the student a total of $50,000 over three years. If choosing Harvard instead would have given the student a $3000 inflation-adjusted boost in his or her average annual income, he or she will still suffer a net loss of $40,000 over a thirty year career in practice; that figure goes down slightly if we include interest payments, but not much, since there will also be foregone interest income on the lost extra earnings. And the $3000 estimate probably understates the true size of the Harvard premium. It also takes no account of the prestige of the student’s job, how interesting the work is, and how well-respected the degree is outside the region where the school is located. Many applicants will care about at least some of these considerations in addition to pay.
BYU is a very good school. I’m not saying it’s always wrong for applicants to choose it over a rival that is superior on conventional measures of school quality. If you want to practice in Utah and/or attend a distinctively Mormon institution, it’s perfectly rational to choose BYU over a school with a significantly stronger faculty and student body. If you reject a much stronger school primarily to save money, however, it’s probably an unwise decision. Tuition differences are an excellent reason to choose one school over another if the quality gap between them is modest, but not if it’s very large.
The second problem with Gladwell’s emphasis on tuition is that it’s a factor that applicants can easily rate for themselves. It’s simple for applicants to find out what tuition each school charges, and compare one to another. The really relevant number here, of course, is not the average tuition that Gladwell uses in his study, but the financial aid package the school offers to the individual applicant in question, which is yet another limitation of Gladwell’s approach. From an applicant’s standpoint, it doesn’t matter how much the other students at School X are paying; what’s important is how much they’re going to charge you.
On the other hand, it’s more difficult for applicants to assess such factors as student and faculty quality on their own. A ranking system that includes both tuition and these other factors in its formula makes it harder for students to use the system to assess the more difficult to understand factors, since a large component of each school’s ranking will be determined by tuition instead. If I’m considering attending BYU, what I would like to see is a clear measure of what I’m getting for my tuition payment relative to what I would get at the other schools, not a ranking that confounds tuition with quality in a way that makes it hard to tell whether a school’s ranking is driven by tuition (a variable the applicant already knows about) or quality. A good ranking system, therefore, should focus on factors that are difficult for applicants to assess on their own. The point is not that tuition is unimportant. It obviously does matter. It’s that prospective students can assess it for themselves.
Despite these criticisms, Gladwell’s system is a useful addition to the rankings market. A ranking can still provide useful information even if it has significant flaws. Law school applicants would be wrong to base their decisions solely or primarily on Gladwell’s methodology. But it’s entirely reasonable to consider it along with other information, including other rankings systems.
UPDATE: Looking around the internet, there is a division of opinion between commentators who interpret Gladwell as attacking US News while presenting what he thinks is a superior system, and those who believe that he’s hostile to rankings as such. The fact that there is this much disagreement over what he means is a sign that the article isn’t as clear as it probably should have been. In any event, Gladwell clearly does seem to believe that his approach is better than the US News system, which suggests that he thinks some ranking systems are better than others, even if all are ultimately flawed. In my view, we cannot and will not dispense with rankings entirely, so it makes good sense to analyze their strengths and weaknesses, regardless of whether Gladwell agrees or not.