LSAC Study on Law School Gaming Resources for US News Rankings

My law school’s Librarian and Associate Dean, Billie Jo Kaufman, sent around this latest newsletter from the Law School Admissions Council re a new study on how law schools move resources around in order to maximize USNWR rankings (thanks Billie Jo!).  Here is part of the LSAC executive summary:

Fear of Failing:  the Effects of U.S. News & World Report Rankings on U.S. Law Schools, by Michael Sauder, and Wendy Espeland.  Love ’em or hate ’em, the rankings are a fact of life for law schools.  This report looks at the effect of the USN rankings on legal education, and it’s not peripheral.

From the Executive Summary:

One general effect of the USN rankings on law schools is that it has created pressure on law school administrators to redistribute resources in ways that maximize their scores on the criteria used by USN to create the rankings, even if they are skeptical that this is a productive use of these resources. This redistribution is illustrated by two examples mentioned consistently by the administrators interviewed: (a) increases in marketing expenditures aimed toward raising reputation scores in the USN survey and (b) increases in merit scholarships intended to improve the statistical profile of incoming classes. A more subtle form of resource redistribution is also described in this section: the adoption of strategies by some schools to “game” the rankings.

Some forms of this redistribution comes in the forms of brochures and other publications designed to enhance a school’s reputation, which is a full 40% of the ranking.  Many administrators acknowledge that many these enhancing publications are probably not read before being recycled (go green!), but peer pressure makes them spend upwards of $100,000 to produce and distribute these publications anyway.  Library volume count, by the way, represents 0.75% of the weight of an over all score.  That explains much more as to why academic law libraries are hardly ever enhanced with an eye to the rankings.

Schools game the system by giving out financial aid to students with high LSAT scores as a way of bringing in a higher quality metric to the entering class and student body as a whole.  Some schools create part-time or probationary programs to keep the academically weaker students off the books.  Student quality (LSAT, GPA) is 12.5% of the full ranking.  Things such as need-based scholarships, enhancements to programs, improved quality to areas of the school not affecting the rankings all tend to take a back seat at the most rankings-obsessed schools.  The pressure on admissions and career services offices is tremendous.  Schools put up with it because potential students do take rankings seriously.  Rankings even affects faculty recruitment and the ability of faculty to publish in quality journals.  While none of this is a surprise, the report is a nice compilation of how schools have adapted their budgets and practices to account for their survey information.  There is also a nice table that breaks down the ranking standards and weights.

The report is available on the LSAC site.  Because of the way the site is constructed, there is no direct URL.  However, anyone interested in finding it should start at the main LSAC page, click on Research Data from the menu bar running across the top of the screen, roll over the graphic for Research and select Grant Reports.  The report should be available from the page that comes up.  It’s numbered GR-07-02.  [MG]

One question I have long had about LSAT scores is whether they tend to favor the youngest applicants.  My brother the neuro-psychiatrist points out to me that IQ scores on average tend to go down starting quite early in adulthood, and suggests that particularly the logical games section of the LSAT functions like an IQ test.  If that were true, and younger takers of the LSAT had higher scores, other things equal, and if USNWR put enough pressure at the margin for higher LSAT scores (even for small increments, and I suspect the rankings do indeed create such marginal pressures), then the result might favor younger law classes.

And yet, if age is the thing that matters, the students being taken younger today will turn into those same “less smart” people tomorrow, so apart from the LSAT scores themselves, what is the advantage in this?  Do we as law schools think that in those younger years, those higher scores mean we can teach more advanced stuff in school?  Do we think that this compensates for having students who already know something of the world that lawyers regulate and intermediate?  I, for one, think that professional school is more professional, not less, if students are not coming direct from undergraduate studies – and I do not favor in the least a “trade school” model of legal education.  Rather, a thoroughly intellectual law study that takes advantage of the fact that students are not entirely operating in their imaginations about the world of work.

Why do I think think this matters?  Because, if true, it tends to squeeze out students who have spent significant years doing work in the real world – as (again, if true) merely an artifact of age.  I think offhand of one person – Harvard undergraduate, did very well there in economics, near perfect SATs – who has spent the last eight years first learning Arabic, and then reporting for various wire services from the Middle East.  Well, he can see the writing on the wall in journalism, and wants to go to law school.  From the standpoint of the tests, however, he was smarter and a better bet as a law student when he first got out of Harvard undergrad.  He’s in Cairo but does not think, probably correctly, that coming back to do Kaplan would make that much difference and, frankly, it’s an idiotic idea.  (My sense of this is anecdotal, of course, but I can think of various people like him – I don’t mean folks who might be considered “interesting” even if not “smart” in the usual terms – I mean people who have all their lives been “smart” in the sense of top schools, top test scores, but find that after significant time doing something real, test performance is not what it used to be.)

In the past, someone like him would be considered an interesting possibility for Harvard or Yale law, and the LSAT score would be a consideration, but taking him would not damage the school’s USNWR scores.  Now, the USNWR rankings mean that if the school decides that someone like him means smart + worldly experience + willingness to do something (in those years) other than practice for the LSAT while being an analyst on Wall St, well, it will cost the school.  Its LSAT composite goes down.  If I’m right about the underlying assumption of age and LSAT score, on the margin, taking these older, experienced people, even the ones who, earlier in life, have hit the standard “smart” benchmarks, is a real hit to the school’s rankings.

Anecdotally, I think this is happening.  No data to really know, but I suspect it has affected the composition of law classes, in terms of age and experience outside of school.  Anyone know of data that would tend to confirm or disconfirm this?