So much figurative blogosphere ink has been spilled over the US News law school rankings that I hesitate to try to add anything. However, there is one flaw in the US News system that hasn't received as much attention as it should. As Brian Leiter explains, approximately 11% (9.75% for instructional spending, 1.5% for other spending) of a school's rankings depends on its per-student expenditure of money. This may not seem like a lot, but, given the high degree of clustering in the other components of the formula, it can actually make a substantial difference to a school's final ranking.
In other words, if School A and School B are exactly equal in the quality of their students, faculty, facilities, etc., but School A spends twice as much money per student to get this result as School B, then A will come out well ahead in the in the US News rankings. A is actually rewarded for being far less efficient in getting educational value for its money than B! Thus, the US News system gives schools an incentive to engage in wasteful expenditures. This is particularly unfortunate in the case of public law schools, where some of the funds expended are taxpayer money. And even private law schools receive many direct and indirect government subsidies as well.
There is no reason to believe that including expenditures provides useful information to applicants or others interested in the school's quality. To the extent that the money the school spends translates into real improvements in quality, these can be measured directly by including ratings for the quality of the faculty, student body, and facilities. Many of these factors are already included in the US News formula and the rest certainly can and should be (some are in fact measured in the Leiter rankings).
I have to admit that George Mason Law School has a lot less money than most of our competitors, so we have a special interest in getting this part of the US News system eliminated. But this, to my mind, is one of those cases where an argument is correct despite the fact that the person making it could have self-interested motives.
UPDATE/CLARIFICATION: It's true that US News does not publish a separate expenditure ranking. However, as Brian Leiter explains in the first link above, they DO factor in expenditures in the formula that determines schools' overall rankings. As a result, some schools place ahead of others solely because they spend more money per student without actually increasing quality.
Related Posts (on one page):
- Final Four Update:
- How the US News Law School Rankings Reward Wasteful Spending:
- U.S. News Movers:
Hire me at a sufficient salary to increase your rankings as desired. Half a mill should do -- if not, I'll be easy to negotiate upwards. We'll just keep at it until you get the rankings you desire.
U.S. News agrees with you -- they don't publish the expenditure figures in their magazine.
How they measure employment at graduation (6%) and after nine months (12%) is not readily apparent to me. Do law schools these days actually follow up graduates, take surveys to see how many are employed?
Pretty much like any case ever decided by a court.
Yes they do. I just spoke with my school's career counselor and she told me to be on the look out for an employment survey in my mail. She asked that I fill it out and return it quickly as employment statistics are an important aspect of the school's ranking.
It's the bar review courses that do that training, not the law schools.
Is Anon44 correct? If a law school raises tuition by $10,000 while simulaneously handing out a $10,000 scholarship to each student, did expenditures per student just jump by $10K, thereby boosting the school's ranking? If so, that's nutty.
As to whether students learn to pass the bar in law school or in bar prep, our experience in Texas doesn't really bear that out. Some have opined that UT generally draws a little more academic student than Baylor. SMU draws from approximately the same pool as Baylor. Baylor's bar pass rates consistently outperform UT, which consistently outperforms SMU. Given these facts, it would appear that the law school attended is a significant factor in bar passage rate. It wouldn't be the craziest idea to consider that the Baylor students may be learning something about the bar exam while they're in law school.
Just roughing some ideas out of my head, here are some possible reasons why higher expenditures might indicate better quality:
1 - Conspicuous consumption; higher expenditures indicates more robust fiscal health (and, further along, better long-term prospects, less susceptibility to short-term risks, etc.)
2 - Higher expenditures may indicate a greater dedication to faculty recruitment or facilities development (and the expenditures may reflect a quality increase before the reputation scores, which lag behind, catch up)
3 - There is an apparent relationship between expenditures and quality, at least as reflected in the variety of extant rankings, even if the predictive power of expenditure is not perfect (although this may not signal causation, surely it suggests correlation, and applicants care more for signals of quality, it is administrators who should care for the causal relationships)
Obviously, none of these are flawless indicators, but then neither are admission rates (yield protection), LSAT/GPA (numbers-hunting), faculty reputations (law porn), employment date (post-graduation research assistant positions), etc.
Making the obviously false claim the heart of the post suggests a certain Socratic Irony -- maybe we were supposed to realize that you intended us to see the obvious flaw and to work backwards to your true point, perhaps that expenditure is one of the best measures of law school quality (a message that a faculty member from a state school is politically barred from posting directly, so it is here encoded)?
Very sly professor. Very sly indeed.
The "placement" factors are even more ludicrous and depend entirely on the honesty of the law schools reports. Our career services made a huge effort to find out and we have a respectable (and I think honest) number, but it is slightly lower than some competitors higher on the list. Does this hurt us--well, if usnews uses "scaling" it probably does. The ABA allows graduates to be counted as having been hired if they are working in a non-law job or are working part-time. Apparently some schools have hired otherwise unemployed graduates for one day on "census day" to do scut work in the library, moving books on shelves, etc. That way they get to report near 100% placement. And, under the definition, flipping hamburgers part-time at MacDonalds counts.
The biggest weight goes to peer and lawyer assessment. This may be moderately accurate for the top of the list, but I doubt very many federal judges or lawyers have much of an idea about more than a handful of law schools outside the top 20 or 40. Lawyers or professors in the northeast (where, apparently, a large percentage of the lawyers who rate are located) have little idea (and likely no idea) of the relative strengths of all of the law schools in, say, the northwest or in the midwest. How do they rate most of the schools (the ones that are not in the top 20) outside the area where they practice--likely by looking at last year's list. How can that be useful information!
Finally (as far as this comment is concerned) if one looks at the professor assessment rank and the overall rank, there is a pretty strong correlation. There are a few exceptions--in the top 50 most notably three religiously affiliated law schools, Brigham Young, Fordham and Notre Dame. Each has substantially higher LSAT scores than many schools ranked above them--Fordham in 32nd place would rank in 17th place if LSAT scores were the basis for ranking. (George Mason is another school which does relatively poorly in "peer assessment" I suspect unfairly so.) But, in general, what other professors think probably decides the fate of any law school as far as the US News rankings go. I don't doubt that a given professor may know something or even a lot about a few schools, but I doubt if there are any who know anything useful about more than 20 law schools--other than how they ranked last year! And, as to accuracy, I once heard a senior faculty member at Virginia announce that there were only two decent law schools on the west coast--Stanford and Boalt! (Take that Eugene Volokh!) If this faculty member happens to be chosen as one of the "raters" by US News, what useful information can he provide about any other law school on the west coast.
To suggest, as I've seen some do, that students should ignore aggregate rankings like US News and engage in extensive research is wasteful. These rankings are mostly self-enforcing, so a student _generally_ does well to heed them, and heeding them is mammothly less expensive to the student than performing the extensive research necessary both of the criteria for school quality and of the attributes of all of the law schools to examine school quality under those criteria.
In light of all of this, you'd think that the US News rankings would be fairly popular here, where micro-economics and law and economics seem to be more fondly thought of than on the average blog.
What McKaskie forgets is that perception is, for many, the most important determinant of the value of a law school degree. That perception is what it is, even if it is inaccurate. That perception gets you into the door of the job you want, or it doesn't. Sure, you might get in there with a less impressive credential, but why take the chance? So the fact that your average lawyer might be wrong about where XYZ law school ranks is irrelevant.
Someone once told me (perhaps apocraphally) that there was an experiment done where a list of law schools were sent out for ranking and Princeton's did quite well, even though Princeton obviously has no law school. The person told me this as evidence that rankings are bunk, but what it really suggested was that Princeton should get into the law school business, because it would do quite well.
Unless you are a purest, it doesn't matter whether you will get a better legal education at GMU or Harvard (and it would be hard to determine even if it did matter because it depends on what classes you take and what professors you get). What matters for most prospective students is that no one will turn you down for an interview with a Harvard degree, and many will turn you down with a GMU degree. Given the 75K cover charge, wouldn't you want to keep as many employment doors open as you can?
Here's one. And unlike others, the designer's school is actually ranked lower than in U.S. News.
Wasteful? A legal education costs $150k. I'd hardly call researching what will probably be the second biggest investment in a student's entire life "wasteful."
That said, the extensive research isn't even that time consuming, since a wide variety of sources have already aggregated a lot of this data and have it freely available online, and finding that stuff is as simple as going to any law school discussion board (even the less popular ones) and asking for links.
Oh really? So if a student wants to work in California after graduation and is trying to decide between WUSTL and Wisconsin she should just take WUSTL because it's ranked higher in U.S. News? What about a student who wants to work in New York deciding between Fordham, Emory, and Iowa?
U.S. News is completely useless to both of those students -- U.S. News isn't measuring employment placement in California, or New York, or any particular geographic region.
Also, if the rankings are self-enforcing, why does empirical research show very significant differences between U.S. News rank and regional placement?
Less expensive? U.S. News costs $10 and every alternative ranking I've seen (at least those worth considering) is available for free online. And even if the latter weren't free, once again, we're talking about a three year time committment and $150k investment here. Are you really saying people shouldn't gather information on a law school's career prospects (or faculty quality if that's what they primarily care about, or whatever) when that much time and money are at stake?
First of all, your survey is clearly susceptible to at least one of the criticisms directed at US News. People often complain that US News does not accurately reflect school quality; your survey does not purport to measure school quality, it purports to measure placement at "elite" firms. Moreover, schools can tailor their behavior to your rankings just as schools (allegedly) tailor their behavior toward the US News rankings (possibly, for example, by over-emphasizing placement at the employers you consider elite and by competing for the attention of those firms). Of course, if "elite" firm placement is all that matters, this should be no trouble; however, I know of very few people who would contend that "elite" firm placement is all that matters.
Second, and this is my fault, because by "significantly different results" I should have made clear that, in the context of my post, significantly different is a judgment that is contingent upon the differences and how those differences enter into usage. After all, your survey produces a top 15 that contains 14 of the schools in the US News survey according to your table 3, the only difference being that Texas is left out and Vanderbilt, hardly a TTT (to use terminology from your realm of the internet) is let in. Is that statistically significantly different? Sure. Is that significantly different in terms of meaningful usage? Maybe, but it certainly isn't some mammoth shake-up.
So, again, I've yet to see a survey that produces results strikingly different from the US News results and that is not itself susceptible to criticisms like the US News is susceptible to.
Third, I'd hazard that you need to re-read my post before attacking it, for I wrote not that researching was wasteful, but that "ignor[ing] aggregate rankings like US News and engag[ing] in extensive research is wasteful." It may still be the case that my statement is wrong, but you were there rebutting some claim other than mine.
Fourth, you suggest that the extensive research is not time-consuming. However, I must take issue with the premise underlying both your instant rebuttal and your rankings - that "elite" firm placement is the most important or only meaningful factor to look at. As I wrote, the research is extensive both because you need to research the attributes of the various schools and because you need to do an examination of what it takes to make a desirable law school. Obviate the second question by limiting everything to one factor addressing one aspect of the issue certainly does make research easy, but I'd hazard that is, for many applicants, an unrealistic view on the situation.
Fifth, I meant what I wrote when I said that "a student _generally_ does well to heed [the US News rankings]." Generally paying attention to or giving consideration to the US News rankings does not mean slavishly following them in selecting where to matriculate; it means using them where appopriate. After all, what about the student who wants to work in New York choosing between NYU, Fordham, and New York Law School? Sure seems the US News ranking might help there, even under your premises? As you surely know, exceptions and anecdotes don't prove rebuttals of assertions of trends and generalities.
Moreover, the US News rankings aren't even useless to your purported students - even for them, they provide a place for those students to begin compiling a list of the law schools that might be beneficial to apply to initially, they reveal to the students their opportunity costs (i.e., what aspects of their law school choice they are ignoring in their quest to be concerned only with regional placement at "elite" firms), etc.
Sixth, that there are significant differences between the US News rankings and regional placement does not show that the rankings are not self-enforcing. The fact is that those US News rankings are by far the dominant force in the rankings game and that virtually everyone else is just reacting to them. Many, many applicants take the US News rankings to be meaningful. Accordingly, the favored law schools are able to attract marginally more attractive students than they otherwise would, they are able to garner more public acclaim than they otherwise would, etc. Again, even your rankings are not all that different than the US News rankings.
Finally, I am amused and bemused by your conclusion: "Are you really saying people shouldn't gather information on a law school's career prospects (or faculty quality if that's what they primarily care about, or whatever) when that much time and money are at stake?" If that is what I had meant to say, that is what I would have said. As I explained above, it is that ignoring the US News rankings as meaningless and intended to re-create the wheel is wasteful and needlessly expensive and that "or whatever" is question-begging to the extreme (for the "or whatever" is the most difficult part of the rankings debate).
Clearly, you've put quite some time into your analysis of the data, and I don't doubt that compiling the data is useful. The hard and controversial part of comparing law schools, however, is not the descriptive part (collecting data) but instead the normative part (deciding what factors are important and how to relatively weigh the various factors).
However, on the issue of expenditures per student, I am not entirely persuaded that the spending should be described as wasteful. The signaling effect may be important. One finds an analogue in the advertising-as-information literature. There we learn that it is rational for a consumer to purchase a brand (say, Brand B) about which he has no information other than the fact that Brand B advertises a lot. Why? Because, by doing a lot of advertising, Brand B is signaling a degree of financial success and stability, meaning that lots of other customers like it, and that the producers stand behind it with significant assets. So, if the consumer likes Brand B, it is likely to be around for repeat purchases; and if Brand B happens to make him ill, there is a deep pocket to sue. (Of course there are other easily imagined reasons that the decision is rational.)
It strikes me that a consumer of law school education could make a similarly rational choice on the basis of the assets a school is able to spend per student. In fact, the decision might be even more rational in the case of a law school than in the case of Brand B, because the consumer could take the signal to mean that many alumni have achieved financial success, and that they happily support their alma mater, either through gifts to endowment or through the annual fund.
(Note: To say that the choice is rational is not to say that the choice is correct, or that it is possessed of universal appeal. Many consumers, selecting a brand in the supermarket, will be driven by factors besides what others think, and many potential lawyers, selecting a law school, will be driven by factors besides the wealth and sentimentality of the school's graduates.)
To the extent that these expenditures (under 2) cause a genuine improvement in the educational experience, though, this doesn't seem to be a problem. Ilya says that all this will be measured by other quality factors, but I don't think this is necessarily so. A student might reasonably prefer a school that can provide him access to a more varied set of experiences (e.g., more clinical offerings) that are orthogonal to measures of quality. But, yes, if that extra money is being spent on glossy magazines for alums and booze for the students it means nothing.
This is unfair to public schools (I assume) and poorer private schools, but that doesn't necessarily make it invalid.
The analogy you draw beteween the signaling effects caused by advertisements and the signaling effects caused by per-student expenditures is an interesting one. I have a question, however: are per-student expenditures like advertisements?
It would seem that it is the rankings themselves that are analogous to the advertisements. The rankings are effective advertisements if and only if the law schools can ascend in the rankings, and/or be in the top 10 or 15 or 25 or 50 [or fill-in-the-blank-category] of all law schools. The law schools' need to have their rankings be good "advertisements" would encourage the phenomenon of "gaming" the per-student expenditures that has been discussed earlier in this thread.
Thus, per-student expenditures would seem to be mainly used not as signaling effects themselves, but rather as a factor to be manipulated in order to ensure the positive signaling effects the rankings create.
You may be right. Or it may be that what is valuable to a firm in a first year associate is not as valuable to a firm in a lateral move. Or it may be that the attorneys from higher-ranked law schools are less inclined for whatever reason (better initial choices, better later options) to move laterally into law firm positions. Or it may be that the search costs of identifying the people who will be the later lateral hires are marginally much, much higher than the current hiring search costs. Etc.
I'm glad that I'm not paying nearly that much, and am getting a great education in Utah rather than back east. I like the lifestyle and attitude, and will happily leave the big firm jobs to those who worry alot about things like rankings. Skiing was great today!
I gasped a bit, too. Think I went thru school for around $5K, at least that was my student loan debt when I got out. (All in 1975 or so dollars).
My earlier comments were more in the nature of a lament–that such a faulty ranking system has such power over law schools and law students. I have no illusions that usnews will go away or that students, because they perceive the faults of it, will ignore it. It will continue to play a significant role in law applicant decisions as to where to go to law school, and it will induce law schools to make changes (legitimate or otherwise) so as to “game” the results even though it will not improve a schools academic program and may even harm it. (But, what is more important, the best education or the best rating. Obviously the latter!)
Even though the usnews listings are, overall, flawed, one item on the usnews report does convey some useful information to prospective students. Probably the rankings of the top ten or even the top thirty law schools are, in general, relatively accurate. One could quibble about whether a particular law school is ranked a little high or a little low in these top categories, but the relative rankings are probably “useful” information. Further, one of the factors reported, LSATs at the first and third quartile level, is useful information, even for lower ranked schools. But little else that usnews reports and includes in its ultimate evaluation speaks directly to the “quality” of education that a student will receive at any particular law school.
I single out the LSAT figures for three reasons. First, they are the only truly objective factors included in the rankings–and are almost impossible to “game.” Second, they say something important about student perception of the various law schools (since applicants will, other factors being equal, go to the “best” school that they can get into–and the schools perceived as “best” will get the highest LSAT applicants). Finally, I think one of the most important factors in a good law school education is who one’s classmates are. If most of one’s fellow students are very bright, the education will be enhanced. Even if a faculty is “eminent” it doesn’t mean that the instruction is necessarily of equal quality. (I have a niece who graduated magna cum laude from Harvard LS who has horror stories about some of her professors. But she and most of her classmates could learn on their own.) While an individual with, say, a 170 LSAT may or may not be a good student, the odds are that out of a hundred students with such a score, a large number will be excellent compatriots. (Maybe 50% of students with a 170 LSAT will do very well even at a top school and 90% will do okay and only a few will be marginal or flunk out. Conversely, out of a hundred law students with 150 LSATs a few (maybe 10%) will do well–and one out of a hundred may turn out to be superb. But a large percentage will do poorly and a significant percentage will flunk out. Thus most law schools, to maximize the number of successful students and minimize the marginal ones, will (absent unusual factors) admit as many high LSAT students as possible. Since this is information reported by usnews, it is useful for prospective students. And, in very broad terms it tracks the overall rankings of usnews. For example, no third tier school has LSAT scores that come close to first tier schools. But there are anomalies. For example, based on LSAT scores, Fordham should be ranked 17th, not 32nd. And in the lower part of the first tier and in the second tier there are a number of other similar anomalies.
The usnews rankings and the perceptions that existed before usnews got into the game are also very important for hiring. Most major law firms recruit primarily at “top schools.” The same is true for prestigious law clerk positions. (Major law firms based in a particular city may also do some recruiting at nearby local law schools, but would be highly unlikely to look at lower ranked schools not in their immediate neighborhood.) Thus, regardless of the intrinsic quality of the education, a law school applicant is well advised to go to the best law school he or she can–assuming locational or financial considerations are not critical to the choice. (I don’t think major law firms look at usnews to decide where to recruit, but the schools at or near the top of the list are the ones that the firms have regarded as the best for decades–and are schools from which most of the partners graduated.) I teach at a “second tier” law school. At the end of the first year, the students at the top of the class often apply to a top ten law school. I don’t discourage them–not because they will receive a better education (they may or may not) but because the job opportunities are so much better.
But, what gets lost in the “ratings game” is whether a particular law school makes an effective effort to educate their student body. This may be valuable for many students (not necessarily applicants with a 175 LSAT and a 4.00 undergraduate gpa from Harvard or Yale–but there aren’t very many such students). A faculty member, who may not be an eminent scholar, but who is willing to help a student hone research and writing skills so that the student knows how to do a workmanlike job may be very valuable. (Some very bright students don’t have such skills without some guidance.) But, there is nothing in what usnews reports which even attempts to rate whether a particular law school is good , indifferent or positively bad in this respect. An applicant might barely get into a fairly highly ranked school, and may even graduate, but might have become a better lawyer if he or she had gone to a lower ranked school which would have done a much better job at teaching him or her basic lawyer skills. Usnews makes no attempt to make this sort of evaluation–probably because it is virtually impssible unless enormous resources are devoted to the project/.
As to admitting fewer first year students and accepting transfers in the second year, I think a few top rated schools do that, possibly to "game" the system but more likely for for other reasons. Boalt, for example, often admits 40 or 50 second year transfers. But, I think it would be difficult for a lesser ranked school to do very much of that (at least to be able to count on getting an adequate number of transferees to make up for fewer first year students). A student might be willing to sacrifice a lot to go from a third tier school to a Yale or even a Vanderbilt. But, the "dislocation" costs of moving to a new law school, where one is not already in a position to get the best classes, get on law review, get to know professors and go through the hassle of moving would make it unlikely that a student in a third tier school would transfer to a second tier school. For example, Lewis and Clark is a second tier school, and Willamette, about 60 miles away is in the third tier. I doubt that many top students at Willamette would want to transfer after the first year even if Lewis and Clark offerred a second year slot. A few might--if they were from Portland and had a place to live there--but I don't think Lewis and Clark would pick up many students this way. After all, being at the top of the class at Willamette is probably a pretty good guarantee to be on law review and to get notice from local judges and firms. Getting on law review at Lewis and Clark would be a much more iffy proposition and the transferee would not be known to faculty. And being a graduate of Lewis and Clark (though a perfectly good school) doesn't get one all that much up in the hiring game. So, I don't think this sort of "gaming" is going to occur very often. (It might when two schools are close to one another, such as Houston is to Texas Southern, but those situations are not common.)
The reason a top school accepts transfers may not be because to the potential for "gaming" the LSAT scores, but the fact that if a student does well at one law school they are highly likely to do very well at another, even a very much higher ranked school. Actual experience with transferees to high ranked schools has shown that it is a much better "entrance" qualification than LSAT or ugpa.
Frankly I think that law schools (and undergrad programs) should have their student bodies judged on the basis of their 10 worst scoring students (bottom 5% would be an ok compromise). This makes gaming the system much harder still and would be, coincidentally, fairer to schools which have consistently high standards.