Thanks to Eugene for inviting me to participate and talk about one of my favorite topics: legal education. Bill Henderson (Indiana) and I are dissecting the components of the U.S. News rankings in a series of papers, the first of which is now available on SSRN, and which we presented at a symposium at Indiana on rankings last spring. (There were a number of excellent papers presented there and anyone interested in the topic should read them all.) In this post I’ll give a quick summary of our data and methodology. Later today I’ll discuss the first of our findings, the strong evidence of market segmentation in legal education.
U.S. News began ranking law schools in 1987 with a “top 20” list based on a survey of law school deans, a method it repeated in 1988 and 1989. In 1990 and 1991 the magazine used a variety of data to generate a top 25 list. Starting in 1992, U.S. News began giving data on all the ABA-accredited law schools, although only the top 25 were ranked numerically.
It is hard to compare ranks across time because of a variety of data issues. One of the consistent inputs, and a particularly important one, is the median LSAT of entering law students. Bill and I decided to tackle this part of the rankings first, largely because it is something to which faculty members seem to pay a great deal of attention. (This past year U.S. News switched to using the 25th and 75th percentiles of entering class LSATs, in part because the magazine suspected schools were fudging the median numbers they reported. Since the 25/75th percentile numbers are reported to the American Bar Association, U.S. News thought that schools might be more honest in their reporting of those numbers.)
(A good discussion of some of these issues is in Alex Wellen's New York Times magazine story, which discusses our research, from this past Sunday.)
We used data on the law schools themselves (public/private, religious/secular, the number of large law firms (those on the American Lawyer 200 list) interviewing on campus, the percentage of the entering class in the full time program, average student loan debt, and so forth), data from the U.S. News rankings (academic reputation, lawyer-judge reputation) and the metropolitan statistical area (MSA) in which the school was located (how many law schools were in the same MSA, large firm jobs in the MSA, demographic trends in the MSA).
There are huge problems with the U.S. News rankings but there is no question that they are important. For example, a friend recently told me that she had been called by a law review about one of her manuscripts. The articles editor apologized for rejecting the manuscript and explained that the rejection had been made without reading the paper because the editors had mistakenly misclassified my friend’s school as being in a lower tier law school. Now that they realized their error, the editor told her, they wanted to consider the article on the merits. I don’t know how widespread this type of screening is, but that it occurred at a well-ranked, but not top journal is at least moderately disturbing.
The U.S. News rankings also have consequences for students – several admissions directors have told both Bill and me that they are noticing a compression in the LSAT scores of applicants, with a strong clustering around the median scores reported in U.S. News. Whether applicants are relying on the overall rankings, or as Brian Leiter suggests, on individual components of the rankings, U.S. News is clearly having an impact on legal education.
The question I would like to throw out for Volokh Conspiracy readers is not whether U.S. News rankings are good or bad in aggregate, but how the competition for students with LSAT scores above each school’s median (and, now, 25th and 75th percentiles) is affecting legal education.
Later today I will post a summary of our results on the segmentation of the market for legal education between the “top tier” and everyone else.
Just like I would have advised a "normal" student (e.g., no special skills or characteristics) with a 1000 SAT not to apply to Penn as an undergraduate, because it would be a waste of time and money, I needed "someone" to tell me whether it was a waste of time for someone "like me" to apply to Penn Law. So, I looked at the median rankings and saw, lo and behold, that I was exactly the Penn Law median. I applied, got in, and attended.
I also applied to Temple Law as a "safety school." I was well above the median there, and assume I would get in (I was right).
Had I seen that I was 10 points below the median, I likely would not have applied to Penn. Why should I? My undergrad GPA was very good, but not 4.0, and likely not any better than the other median applicants. I am not under the illusion that I am "special" in a way that a Law School would care about. I think I'm "smart", but the more elite the pool is, the less smart I am in comparison.
What you describe as a "compression" is more likely just the expression of information. Had I not known Temple's median, I may have also applied to Villanova and Widener, because my safety school would have felt less safe. Had I had a much lower LSAT, and not known Penn's median, I would have likely applied there too, considering it a "reach", but not knowing that it was next-to-impossible.
It simply makes no sense to apply to lots of schools you are either over or underqualified to attend. In my view, information helps the right people find the right schools, with little wastage in application fees.
However, the answer to the question of "good" or "bad" depends on the identity of the customer in the market.
If students are the schools' customers, students would pay a premium to schools that do the best job of turning qualifications into job opportunities. The extent a school depends on the quality of students to create marketable graduates would be seen as a weakness of the school. A "great" school would turn out Manhattan Biglaw associates from 2.5 GPA, 63 LSAT kids.
If large law employers are the schools' customers, then you'd expect the highly valued school to turn out consistently high level employees, which can be done most efficiently (cheaply) in the current system by starting with the best GPA and LSAT kids. Customers would pay a premium to get students of a guaranteed high caliber delivered efficienlty via exclusive admissions.
It's more than moderately disturbing, it's a threat to academic inquiry. The journal editors are arrogantly asserting that only papers from their approved list of scholars will be considered for publication and that everyone else - regardless of the quality of their ideas - is considered second-class and unworthy of consideration. Is it common for legal journals to fail basic academic standards like the one here? More fundamentally, what's wrong with blind review and why are law journals doing it?
I found the US News ranking very unhelpful because the statistics apply only to students who are just graduating from undergrad and going directly to law school. My LSAT score would compare favorably to other applicants. My UGPA would only hurt a law school thinking about accepting me. And my GGPA was fine though the law school would get no benefit from it.
I just applied at schools that looked interesting, didn't seem too focused on new graduates and were "top tier." I think it turned out fine, but I'll let you know in two more years.
Its interesting to consider if any well known lawyers from history would have been held back by the LSAT.
I completely disagree with this often-voiced sentiment. Put simply, $60 bucks a pop is not that much money. What’s more valuable to your average high-school or college student, a copy of the latest video game or a chance to go to a great “reach” college or law school? I went to a mediocre public school in central Texas and, with no expectations of getting in anywhere all that prestigious, applied to 13 schools. East coast, west coast, mid-west etc… My friends, most of whom applied to a single school (UT or A&M), thought it was a complete waste of money, but I ended up getting into a couple of Ivy's. When I applied to law schools I repeated the process, applied to 9 and ended up at a top 3 school (thereby greasing the wheels for the rest of my professional life). Strangely enough, I was rejected from a couple of schools that according to the US News numbers were more in my range of GPA and LSAT scores. I would recommend applying to as many “reach” schools as you can. You might just get lucky and all you have to lose is a few hundred bucks (and really, you probably spend more than that at Starbucks each year…)
It may seem obvious to some that such distortion is bound to be bad. I'm less sure, however. It seems likely that some students are being reallocated among the schools, in the sense that some students might have ended up at different schools if there was not this incentive to view LSAT scores in this way. But is this reallocation really going to affect legal education in general?
I am enough of skeptic about the LSAT to think it will not really matter much to the educational experience of the students. For example, I suspect that students ordinarily could not tell the difference between classes comprised of fellow students with more or less compressed LSAT distributions respectively.
So, although it all seems silly and wasteful to me, I'm not sure I see much overall harm being done--holding aside the general point that whatever resources are being wasted on this competition could be used in more productive ways.
After graduation, I did temp work for a couple of law firms and taught high school. I never gave up going to law school, but I thought it would do me some good if I actually worked full time before applying. I still don't regret the choice that I made. Perhaps foolishly, I thought that work experience would help out my application. Good grades, solid recs, solid work experience: what else could I do? Oh yeah, score better on the LSAT. I studied even more, did well on the practice tests, took the test again and scored a 160. Damn.
I applied to 10 schools. I was accepted by only 2. One was Tulane (which had briefly dropped below the top 50 in U.S. News) and the other was a U.S. News third tier school. All the schools that rejected me were first-tier U.S. News schools (not necessarily elite like Harvard or Stanford, but first-tier nonetheless). I know that my essay wasn't atrocious, so what else can I chalk up these rejections to other than subpar LSAT scores for first tier schools?
Fortunately, I ended up grading onto law review at Tulane. I guess that shows ultimately how important silly tests like the SAT and LSAT are.
How do LSAT scores ultimately affect legal education? My educated guess is that they cause alot of folks who shouldn't really be going to law school to go to very good ones. More than a few undergrads who really have little or no interest in law study for and take the LSAT for lack of anything else to do after graduation. If they score well, and did well in undergrad, a first-tier law school will take them over someone who has demonstrated legal skills and interest (e.g., by working as a paralegal) but whose LSAT scores are subpar for the first tier. Considering that some elite law firms won't interview even law review members from below first-tier schools, people at lower-tier schools are effectively blackballed from working at such places. And such people can forget about snagging plum professorships, since prestigious clerkships and elite firm jobs are practically prerequisites.
(Yeah, I suppose you could distinguish yourself as an attorney and laterally move into a firm, but that can be tough to do.)
I just think it's sad that many of the high-paying and prestigious jobs in the law are effectively reserved for first tier law school grads, and getting into a first tier law school is mostly determined by one's LSAT score.
To any ambitious person about going to law school: better study hard for that LSAT!
"The question I would like to throw out for Volokh Conspiracy readers is not whether U.S. News rankings are good or bad in aggregate, but how the competition for students with LSAT scores above each school’s median (and, now, 25th and 75th percentiles) is affecting legal education."
What are schools doing in the competition for students with LSAT above each schools' median? I assume they must be competing but I don't know specifically what they are doing. Assuming they are competing in that way, how could such competition effect my legal education? Are they increasing class size or not hiring good professors? What? Or is it the possible distortion that Medis references in his post? I am totally confused by the question.
I was one of those quixotic 'borderline' candidates for the schools I really wanted to go to; my GPA was quite low, but my LSAT was good. Moreover I'd only been out of school for 1.5 yrs before applying, so my GPA was still considered a highly relevant piece of recent data. Had I not known that my LSAT was comparable to the median LSAT of some very good schools, I would have just assumed that applying to Top 20 schools was an exercise in futility and decided not to waste my money. As it is I DID apply to many of those schools, and ultimately was accepted at the University of Chicago. Therefore, in my case at least, the US News Median LSAT stat was absolutely critical in helping me reevaluate my chances. Without it, I feel I would be significantly worse off.
N.B. The behavior of the law journal editor is, as others have said, a good deal more than "moderately" disturbing. It's appallingly unprofessional conduct; more appalling, perhaps, is the possibility that such frankly anti-intellectual arrogance would be official policy for a journal.
I've done well (top 25%) at law school, but the skills that the LSAT tests are not the reason.