David's post on the "law porn" sent out by law schools to try to improve their US News rankings highlights a more general problem with the system. A substantial part (25%) of a school's ranking depends on ratings by randomly selected professors at other schools. Another 15% is based on a survey of randomly selected lawyers and judges.
Here's the problem: there are some 190 ABA-accredited law schools in the US. The average professor doesn't know much about what is going on at the vast majority of them. If I spent my time keeping up with the faculty publications, curricula, student quality, and so forth, at the other 190 law schools, I wouldn't have any time left over to do my own research and teaching. Realistically, I only know something about the top 30-40 schools (and even then with far from complete thoroughness), plus a handful of others that I am familiar with for some special reason (e.g. - I went there to do a presentation, and therefore know the faculty). I suspect that the same is true of the lawyers and judges. They too have their own work to do, and therefore can't spend their time keeping track of the doings at dozens of law schools.
This doesn't mean that the surveys are completely useless. Some valuable information can still be gleaned from them, especially if the errors of the ignorant US News voters somehow cancel each other out, leaving those knowledgeable about a given school to actually determine its ranking. However, I suspect that errors are not randomly distributed, and that there are some systematic biases. In particular, the voters are less likely to recognize the quality of schools that have recently improved their faculties and/or student bodies (this hurts George Mason, among others), less likely to give high rankings to schools outside major metro areas on the East and West coasts, and so on. I also suspect that the professors - and even more so the lawyers and judges - are likely to base their evaluations in part on what was true when they were in law school rather than regularly updating their evaluations of schools outside the top 20 or 30.
No doubt, there are also other biases that will affect survey responses in an environment where most of those surveyed are necessarily ignorant about the vast majority of the schools that they rate.
Related Posts (on one page):
- A weakness in the US News Ranking System for Law Schools:
- Effective and Ineffective "Law Porn":
Notre Dame and BYU feel your pain.
Chicago has, and has always had, a majority of left-wing professors. It's reputation as a "right-wing" school comes from the fact that nonliberals are a larger MINORITY there than at most other elite schools.
Fortunately, this information is easily available, even if USNews is not the one publishing it. You can just go to the website of your favorite law firms in the city of your choice and see who has been hired.
Say you want to work for an excellent entertainment law shop in Atlanta, and you want to know where to go to school. You're in luck! You can find pretty good answers using the information provided on pages like this one: http://www.gtlaw.com/people/
While Chicago, as home to several leading lights of legal conservatism (and just as many liberals), isn't typically left-wing, its balance is less contemptible to professors than ND and BYU's religious orientation or GMU's libertarianism.
Professors do, however, underrate Chicago –- it shouldn’t be tied with NYU and behind Stanford.
The biggest secret in law school rankings though is that if you want to work in BigLaw, you're much better off going to a second or third tier school located in a large legal market and graduating at the top of your class than going to a higher-ranked school that will never be visited by firms from the cities in which you'd like to work. For example, I worked at a BigLaw firm in Philly when I came out of school, and our incoming first year class had several people from Penn, a couple from Harvard, a Dukie, one UNC, one William &Mary, a couple from UVA, one G'Town, but also a handful from Villanova and a couple from Temple. Now, if you really want that kind of a job in Philly, surely you'd be better off going to Temple or Villanova over UC Davis (or Hastings or the Univ. of Arizona or any of the other "better" law schools that our firm didn't recruit from), no matter what the US News rankings say.
I would accept that Yale, Harvard, Stanford, NYU, Chicago, Boalt Hall etc. are all very fine law schools. But I think the differences are overblown. Are you that much better off having a Harvard grad representing you than a Boalt Hall grad? Probably, individual variations in quality and motivation would be much more important than difference in school.
I think grouping in tiers would be much smarter than trying to rank individual schools. What does it really mean if Harvard moves from 3 to 2, and Stanford moves from 2 to 3. That you should suddenly massively prefer Harvard grads to Stanford grads? I don't think rankings mean jack at this level of analysis.
GMU is not Harvard. But it is a pretty good school, and I would not be unduly worried about it if one of its graduates were representing me on some legal matter. I think crude rankings of this sort are more meaningful.
correct.....there shoudl be like 6 or 7 teirs instead of 4 and schools shouldnt be ranked beyond that
And, despite its location and middling ranking, I secured summer-job offers at national firms based in or with offices in NYC, Dallas, Miami, and Albuquerque -- and that was before I had a clerkship with a federal district judge (a resume bullet that opened doors in LA).
NYC firms pre-screened resumes and invited us to "the city" for interviews..
I got the job offers in other cities by writing letters, sent with my resume, grades, and writing sample, to the best firms in the cities I thought might be nice to work in, telling them I'd be in town on my own dime, and asking for an interview.
All 14 firms I mailed granted me full-blown interviews. Although a few more parochial partners said, "I didn't know Buffalo had a law school," these same individuals said they were impressed with my efforts.
And as a result of one of those interview, I got a federal clerkship.
Buffalo was a great school. I'd stack it up against any other school for the *learning experience*.
What the school lacks is a huge endowment -- which keeps it from hiring perceived superstars from other schools (the way Chicago did a long time ago), which keeps the school from being perceived as a "below market" bargain with up-and-coming faculty.
I have practiced litigation for 30 yrs &I can assure you where a lawyer went to school has no bearing how good a litigator is. My brother has practiced real estate law for longer than that and he will tell you it is the same in his nitch.
You sound like the exception rather than the rule when it comes to SUNY-Buffalo grads. I do not think that interviews for 14 out of 14 firms mailed is normal even for Yale or Harvard grads. Unless the firms suck.
In other words, you must have exceptional skill at selling yourself, which most of your colleagues are definitely going to lack.
This is a bullshit statement. Like it or not, students from higher ranked schools have better employment prospects. Maybe law firms are irrational (or are acting rationally based on the ignorant views of their clients), but they value people from name schools.
I very much doubt what you say is true. That the ability to litigate is unrelated to intelligence. The people who go to better schools are, on average, more intelligent than those who go to lesser schools. Now, intelligence does exist across multiple dimensions. (I am sure John Edward could kick the ass of most Yale grads when it comes to arguing before a jury. He has a very high emotional intelligence.) But on average, those from higher ranking schools have more intelligence, at least as it is conventionally measured by things like GPA and LSAT score.
The John Edwards of the world, who are better litigators than their Ivy League counterparts, are probably the exception more than the rule.
link to page with specialties
Very insightful. Clearly one of the fundamental problems with the USNews rankings that people have been complaining about since their inception hasn't slipped by you. Congrats.
Old but applicable. If you survey a group of people on how tall they are or how much they weigh you will get significantly different results than if you measure the heights and weights of the same group. With the survey method you are measuring how people view themselves, not objective charecteristics of people. Likewise, the survey is useful in determining how good the schools are percieved to be, but not so useful in determining how good they actually are.
That said, I received a first-rate legal education (I suppose it helped that my state's law school was a top-tier regional law school) and think my legal skills are as good as, or better than, most of my colleagues.
While I agree that different law schools have faculty of different caliber, I am still amazed how many people are aflutter that this Judge or that candidate went to a particular law school--as if the caliber of the education is a proxy for the caliber of the legal mind. Surprisingly, I even hear this from attorneys who went to other law schools and should know better.
I agree that several decades ago attending a highly regarded law school was only sometimes an accurate proxy of the person's legal mind, e.g., attending Harvard Law School in the 1950s shows more often that the person was from the Northeast and wealthy rather than that the person was brilliant.
However, because law school admissions are national and for the most part merit-driven, the law school hierarchy has become a useful proxy. Today, the smartest and most qualified students generally attend the best law schools.
Is Yale's smaller class size an advantage over Harvard? For some people it is, for others it isn't. It's probably a disadvantage for some as well. Is being in New Haven a disadvantage over being in Cambridge or New York -- and if so, how much? Again, there's no objective answer to this question.
Rankings gloss over these very important issues and give readers who haven't researched the schools in depth the false impression that someone else has evaluated schools using the same criteria they would have used. The school you would consider best for you might be thirtieth in the U.S. News rankings, and if you rely too heavily on the rankings you may never even know that school exists.
(Btw, these concerns apply to all college and university rankings, not just those of law schools or those published by U.S. News.)
You sure?
While every student takes pretty much the same LSAT, they do not all take the same undergraduate courses. Rankings are strongly correlated with GPA. Now, who is smarter: the kid who went to MIT with a double major in aerospace engineering and physics and got a 3.0, or the one who went to Slippery Rock and got a 3.7? US News would presume the latter to be smarter, every time.
Likewise, older students may have a lot more to offer, may be better law students (and lawyers), but often have lower LSAT scores (from not having taken tests in so long) and GPAs (from having gone to college in the era of the "gentleman's C"). Yet the straight-up GPA/LSAT system penalises schools that take such a student.
As for the issue of professors and judges - I've never seen a US News survey, but does it ask them to rank all of the schools, or does it try to account for lack of knowledge? Would it ask a professor to evaluate the three schools he is most familiar with?
The former.
Yes, I agree. Some other people have made similar points. Since this is a blog post, not an academic paper, it doesn't have to have a completely original thesis. However, to my knowledge previous commentators on this issue have not distinguished between random and systematic bias. If the impact of ignorance on the surveys was limited to the former, it would not be nearly as big a problem as if the latter were also present.
Locally, if you didn't graduate from an in-state school, you are regarded as an inferior creature. A fancy degree is nice, but you don't know the law of the state.
Nationally, I spent ten years in an agency in DC. One attorney was a Yalie. Good guy, but not considered better than the rest. The guy who got the boss's job, and deservedly so because he was the best attorney I've met in 30 years of experience, was from Samford.
Who gives a hoot about all this? An attorney is either good or he isn't. Where he or she went to school makes little difference. If you get into a dogfight with another pilot, do you care where he got his degree, or whether he got one at all? If he's a worse pilot than you, you have him. If he's a better one, he's gonna clean your clock. Trial and appellate work are about speed of reaction, insight, and tactical skills. Those didn't come from postgrad education.
It makes an enormous difference when you're fresh out of law school, have no track record as an attorney and perform in an interview pretty much like every other law student. It's very easy for a law firm to determine where you went to law school and practically impossible to judge your potential as an attorney from a 10 minute on-campus interview. Your law school counts for a lot in the hiring process for new grads, not because law firms think it's a guarantee of your future performance, but because your law school and your grades are pretty much all they have to go on.
If we had perfect information about the quality of every lawyer, of course this would be true. But in real life we often don't know for sure, especially with entry-level job candidates. The quality of the school they went to is one of several imperfect indicators of the lawyer's ability that employers can use in the absence of perfect information.
Also, law school applicants need information about the relative quality of the schools they are applying to.
That said, reputation and quality of education are generally related, but they are two very distinct concepts. We have multiple law schools in this area, and I always remind myself that the biggest difference between them is that the lowest ranking school actually fails people. You really have to work to fail at high-ranked schools. You really have to work to succeed at low-ranked schools.
Almost any school can point to one factor or another that US News should rank differently in order to make that school's ranking "more accurate." I would imagine that the strength of faculty bloggers is now increasing George Mason's reputation at the expense of schools where the professors concentrate on more traditional scholarship. Fair? I think so.
Professor Bernstein demonstrated the over-reliance on publishing when he wrote:
Now, the next sentence ackowledges that clinicians can make good clinicians, but it fails to acknowledge that someone who publishes brilliant law review articles and writes brilliant books can be a horrible teacher.
So another problem with the rankings is that too many professors (like Professor Bernstein) base their "reputation" views on factors with little relationship to actual teaching ability.
As someone who has taught at various stages of my career with varying degrees of success, I understand the difference between mastering a subject and teaching a subject. Those are two very different skills.
How does this differ from any other election? I doubt many people could name the members of their school board or city council. Even the name of their Congressman is a stretch for many. Wasn't there a survey published recently that found something like 30% of the population didn't know the name of the Vice-President?
You could even go a step further and allow each voter to only vote on 10 selected schools: say, the five closest to the mailing ZIP code plus 5 randomly chosen from the remaining 185. And it'd be a breeze to note, next to a school's overall scores, the number of respondents who marked "I don't know enough to rate this school." You could add other stats, too -- "# of grads from this school that I have hired" and so forth. As a plus, not having a list of over one hundred schools (as many as a third of them being ones the voter has no knowledge of) could improve the participation rate.
Every ranking system has problems, c.f. the BCS. Honestly, I prefer to write my own based on numbers most or all schools make available (tuition, LSAT at the median or 25th/75th percentiles, whether they have a competitive moot court team, whether you can write on to their law review, how often their law review gets cited...) That doesn't mean the ranking system we actually have can't be improved, though.
The former.
Not exactly. For each and every school, you can rank them from 1-5, but you can also choose "Don't Know." My guess is that "Don't Know" would be the more honest answer most of the time -- I don't know, however, how often it's chosen.
And what DJR did above should happen more often, so I say good for him.