pageok
pageok
pageok
Anomalies in the purported new US News rankings of law schools.--

There is something strange about the purported new US News rankings of law schools. The two schools with the highest placement rates at graduation (Berkeley 99% and Columbia 98.9%) are also the only two schools on the first page (top 59 schools) with identical placement rates for "employed at graduation" and "employed 9 months after graduation."

What a coincidence!

I work with enough data that when I see a coincidence like this, I recheck how the data or variables were constructed -- and I almost always find that the underlying data are wrong. Is the pdf of the officially unreleased new US News rankings an elaborate hoax? If the pdf is indeed real, then I think that there is a significant possibility that either Berkeley or Columbia -- or perhaps both -- unwittingly reported false data to US News.

While the odds of being employed at graduation are 99 to 1 at Berkeley (and 90 to 1 at Columbia), the odds at the other top 8 schools are much lower: 58 to 1 at Stanford, 27 to 1 at Harvard and Chicago, 26 to 1 at NYU, 25 to 1 at Yale, and 20 to 1 at Penn.

Usually, there is a big jump in employment between graduation and 9 months after graduation, but there is none in the data reported by Berkeley and Columbia to US News. For example, by 9 months after graduation, the relative odds of employment jump 911% at Yale (from 24.6 to 1 at graduation to 249 to 1 9 months later), 270% at Chicago, 169% at Penn, 152% at NYU, and 93% at Harvard, but only 13% at Stanford, 0% at Berkeley and 0% at Columbia.

Even if Berkeley weren't reporting suspiciously identical placement rates at graduation and 9 months later, I would still find it unlikely that Berkeley would have the highest placement rate at graduation (99%) among American schools because (unlike Columbia) it does not have the special advantage of being in one of the top 2 legal markets. And I also find it hard to believe that 99% of Berkeley graduates are employed 9 months after graduation when its bar passage rate in California is only 85%.

In the future, when a law school reports the same placement rates at graduation and 9 months after graduation, US News should treat such reports as probably erroneous data, leading to requests from the law school for more detailed information.

Dave N (mail):
Is the pdf of the officially unreleased new US News rankings an elaborate hoax?
If it is, it a very elaborate hoax.

Did you check to see how these rankings compared to other years?

Occam's Razor suggests that somebody at US News transposed data from one school to another and the proofreader did not catch it.
3.26.2008 1:39pm
Cheburashka (mail):
I went to Columbia. I think there were ~300 people in my graduating class.

I would be shocked -- shocked -- to learn that more than one of those people did not have a job as of graduation. The 99% number is too low. Other than political (or otherwise wealthy) children who go off after graduation to work on campaigns, etc., everyone except the guy last in the class got a job.
3.26.2008 1:49pm
Armen (mail) (www):
Prof. Lindgren,

The first number includes those who have job offers at graduation. Anecdotally, very high 90s sounds about right for Berkeley for all the reasons mentioned by Cheburashka. Obviously, after 9 months that number will not increase if it is already at the ceiling.

You imply that the second number should decrease given Berkeley's bar passage rate. But most firms that I know of let you retake the bar at least once. I can only assume that other legal employers have similar policies. But failing the bar twice takes you out of the 9 month window. So this means we wouldn't expect a drop in employment in the 9 month figure either based on bar passage alone. In fact, after a quick glance, I couldn't spot any drops between employed at graduation and employed after 9 months.

If we're at the ceiling and we have no reason to expect a drop, why are identical figures suspicious?
3.26.2008 2:05pm
A.C.:
At the larger schools, there are bound to be a few people who are pregnant at graduation or otherwise tied up temporarily with family matters. They don't always choose to work the first year out. And I'm not sure how you categorize people doing business start-ups, going on for additional degrees, or trying to write books.
3.26.2008 2:07pm
3L:
The employment numbers are fake; we all know that schools pump them up in various ways. At my school-- U Texas-- the actual employment at graduation rate is around 80%. By designating various students as "studying for the bar" or giving them temporary research assistant jobs, the school classifies them as "employed". Of course, the employment numbers are going to go up, even if a law grad has little choice but to work at a bookstore; that's on the stats as corporate employment.
3.26.2008 2:19pm
Ted Frank (www):
I think there was only a single student in my graduating class of ~190 that didn't have a job at graduation.

What makes no sense is penalizing U Chicago because Illinois has a high bar passage rate. That's what vaulted Berkeley (and, before it, NYU and Columbia and Stanford) over U Chicago.

Shame to see UC depreciate: it was #2 when I applied, #3 when I entered, #4 when I graduated, and has steadily dropped. The school had a huge comparative advantage that it was ashamed of. When it tried to be just like other top-ten law schools, confusing the importance of inter-school diversity with that of intra-school diversity, it only hurt itself when top conservative law students who used to turn down Harvard and Yale to go to Chicago started going to Harvard and Yale.
3.26.2008 2:37pm
bigchris1313 (mail):
How legit are these numbers? Source of the leak?
3.26.2008 3:13pm
OrinKerr:
BigChris,

I believe they are as legit as the U.S. News rankings get.
3.26.2008 3:25pm
at least i'm done with this law school thing (mail):
I'm curious about how many law schools hire graduates who don't have jobs, as research assistants, workers in non-legal departments (i.e. alumni affairs), etc. so that they can call them employed for statistical purposes.
3.26.2008 3:26pm
Pepperdine (mail):
Pepperdine Law School, which has managed to leap nearly 40 spots in the last two years, presents some questions - particularly, its class of 2006 Employment at 9 months figure of 95.0% (assuming the pdf is accurate).

Last year, law Professor Tom Bell broke the news about the change in how the 9 months Emp. numbers would be computed (to be consistent with the ABA survey), and the rankings effect that change would have had in a piece on his blog.

On July 17, 2007, I emailed Pepperdine career services to understand why, for the class of 2006, they had posted a nine-month number that struck me as unusually high (98% at one point) on their website, without giving a further break down of those numbers. I received the following class of 2006 data from Pepperdine in an email response to my inquiry (i am happy to forward this email directly to you so can verify its authenticity):

Hi Ken,

Here's a breakdown of our detailed numbers:

245 graduated in 2006
194 indicated that they had a job (79.2%)
2 were unemployed and seeking work (0.8%)
8 were unemployed and not seeking work (3.3%)
16 were enrolled in full-time degree programs (6.5%)
23 were studying for the bar full time (9.4%)
2 were of unknown status (0.8%)

If you have questions about why these numbers are different from the numbers on the website, I'm happy to explain (at length) the formula that was used in 2006.

XXXXXXX XXXX
Recruiting Coordinator
Pepperdine University School of Law
(310) XXX-XXXX
XXXXXX.XXXXXX@pepperdine.edu

Based on the above data, the 9 month number reported to U.S. News should have been approx. 85.7 (number of grads actually employed plus number pursuing grad degrees), correct?

So where did the 95.0% employed at 9 months appearing in this year's rankings come from??? As professor Bell's analysis from last year's ranking shows, the difference in the accuracy of this number can have a huge rankings impact, and would have dropped Pepperdine nearly 30 spots in last year's rankings.
3.26.2008 3:40pm
frankcross (mail):
How in the world do people know how many of their co-graduates have jobs? You must be terrific networkers.

I think the standard is utterly bogus for top schools, because probably all of the graduates of top schools could get some jobs, but a number of them choose not to do so immediately. Some may be going on to get PhDs, yet the stupid standard would punish the schools for having such students.
3.26.2008 3:45pm
Armen (mail) (www):
I also take exception to the use of "odds" instead of the simple percentage. the difference between 98.3% employed and 99% employed is not great at all. Given the class sizes, it's the difference between one or two people. So I'm just wondering why "odds" are a better measure than the percentage?
3.26.2008 3:47pm
Eric Muller (www):
File Orin's 2:25 p.m. above under "damning with faint praise."

:-)
3.26.2008 3:53pm
merevaudevillian:
Even more curiously, Berkeley's ABA data reported last year indicated that only 93.9% of graduates were employed at 9 months after graduation.
3.26.2008 4:13pm
hawkins:
I dont believe those numbers. I am currently a 3L at a school listed as having 90%+ of graduates employed at the time of graduation. I would guess over 70% of 3Ls do not currently have a job.
3.26.2008 4:20pm
CJColucci:
I don't have a view on whether the law schools are fudging their data or not. I would be surprised, however, if the actual data they had were sufficiently accurate for any kind of fine-grained comparison.
3.26.2008 4:48pm
merevaudevillian:
It should be noted that Columbia did the same thing last year: 99.2% at graduation, 99.2% at 9 months.

Penn's last year was 99.0%, 99.0%.
3.26.2008 5:11pm
3L:
How in the world do people know how many of their co-graduates have jobs? You must be terrific networkers.

Right! Because if there's one thing law students don't talk about, it's jobs and employment. Really.
3.26.2008 5:16pm
Nick Beat:
It is worth noting that (1) the U.S. News employment figures do not distinguish between law and non-law jobs, so that if you are working at all, you count; and (2) U.S. News assumes that 20% of the students the school can't track down are employed.

So an honest 99% employment figure is more a testament to good recordkeeping by the school than to the success of its graduates in getting law jobs.
3.26.2008 5:27pm
George Weiss (mail) (www):
nick beat:

yes but in the complete version (not yet out until Friday) you can also see what percentage of those who reported were in legal jobs jd preferred jobs etc...non jd preferred jobs etc..

and its 25% of those who don't report are unemployed.
3.26.2008 5:39pm
Nick Beat:
George:
I'm glad that that information will be available, but it is still significant that the actual rankings use only the overall numbers.
3.26.2008 5:47pm
crl8 (www):
The purportedly "real" U.S. News rankings list Wash.U's LSAT range as a 163-167, while Wash.U itself lists their range as a 162-167... anyone care to comment on this one? I would think/hope that Wash.U has their numbers right and this posting is a fake.

http://law.wustl.edu/ataglance/
3.26.2008 5:47pm
Hoosier:
Wait!

I can't find Cooley on the list.
3.26.2008 6:14pm
Blue:
Leiter has been making similar claims about the bogusity of the 9 month statistic for a little while: here are some of his posts about it. Of course other people on this thread have referred to some of what he claims-- schools pump up their ratings by hiring assistants, the bar passage rate is suspicious, ABA stats are different. The ranking for Berkeley surprises me a bit, but if it is due to gaming the employment rate, not that surprising.
3.26.2008 6:22pm
Edward A. Hoffman (mail):
Counting grads who are studying full-time for the bar but who don't have jobs lined up as "employed" is absurd. Even grads who have secured jobs at top firms get the summer off (usually with full pay) to study for the bar. In other words, the group of new grads studying full-time for the bar includes those who have jobs waiting for them and those who don't. Those who don't should be counted as unemployed. Otherwise, the only grads schools like Pepperdine actually call unemployed will be those who neither have a job lined up nor are preparing to take the bar exam -- a group which will be quite small at any decent law school.
3.26.2008 7:14pm
MPP (mail):
Jim,

Your point is well-taken, but for one misleading statement:

"And I also find it hard to believe that 99% of Berkeley graduates are employed 9 months after graduation when its bar passage rate in California is only 85%."

Most people take the bar in July and start their jobs in September/October. Bar results don't come out until the end of November. Even if you don't pass the bar this time, even the most ruthless of firms will give you at least one more chance to pass. All the while, you are employed.
3.26.2008 7:47pm
Anonymouse:
Second MPP's comment. I would add that employers might be especially willing to give you another chance in a place like CA, where the bar exam passage rate is low.
3.26.2008 8:00pm
Nick Beat:
It looks like two schools have moved out of the top 100 (Toledo, formerly #85; and U. San Francisco, formerly #100), and two schools have moved in (Hofstra, now #98; and Syracuse, now #100; both were very close last year according to the breakdowns of the data out there).

Not much change.
3.26.2008 9:11pm
Elliot Reed (mail):
Most people take the bar in July and start their jobs in September/October. Bar results don't come out until the end of November. Even if you don't pass the bar this time, even the most ruthless of firms will give you at least one more chance to pass. All the while, you are employed.
Especially here in California, where we have the hardest bar exam in the country. Only 85-90% of Stanford grads pass on the first try and the pass rate only goes down from there. I can't see a firm wanting to toss that many people just because they didn't pass the bar the first time.
3.26.2008 11:16pm
LM (mail):
What is it about Boalt's admissions process that skews the LSAT quartiles so much lower than comparable schools?
3.27.2008 2:44am
Cornellian (mail):
What is it about Boalt's admissions process that skews the LSAT quartiles so much lower than comparable schools?

I hear they're quite fixated on your GPA, so they tend to admit higher GPA's and lower LSAT's than comparable schools.
3.27.2008 3:12am
NYU Jew (mail):
Jim-
In paragraphs 4 and 5, I think you are referring to the odds of being <b>unemployed</b>, not the odds of being <b>employed</b>.
3.27.2008 4:56pm
theobromophile (www):
If 25% of those who do not report are considered unemployed, and 75% of those who do not report are considered employed (I have those numbers in the correct order, right?), wouldn't it make sense for a school to do everything possible to get those unemployed to not report? At least at my law school, you are encouraged to report a job as soon as you get one, and we get a zillion reminder emails about it, but there is very little incentive to report unemployment.

Now, for schools with the same numbers 9 months after graduation as at graduation: could it simply be that they are asking their graduates in such a way as to eliminate the possibility that someone who is unemployed will so report at graduation, and again, nine months later - wherein that person is counted as a non-reporter?
3.27.2008 6:02pm
Casual Peruser:
To be fair, California's relatively low bar passage rate is not simply a function of its difficulty. It is also--at least in part--a function of the fact that California allows graduates from non-ABA accredited law schools to sit for the exam, which skews the base on aggregate.
3.27.2008 6:23pm
theobromophile (www):
July 2006 pass rates, broken down by type of school (ABA accredited, Ca. accredited, correspondence, etc.), sex, race, and perhaps a few other things, are here.
3.27.2008 6:54pm
? (mail):
Pepperdine is already basking in its new found glory:



I've emailed Pepperdine, however, no answer back on how it managed a 95% employed at 9 mo. under the new aba survey (actually has a job + pursuing grad degrees), when, last year, its career services indicated in an email the following data for the class of 2006, and in 2005 according to the aba, only 81.7 were employed, 3.1 pursuing grad degrees:

Hi Ken,

Here's a breakdown of our detailed numbers:

245 graduated in 2006
194 indicated that they had a job (79.2%)
2 were unemployed and seeking work (0.8%)
8 were unemployed and not seeking work (3.3%)
16 were enrolled in full-time degree programs (6.5%)
23 were studying for the bar full time (9.4%)
2 were of unknown status (0.8%)
3.27.2008 7:01pm
Caspar the Friendly Guest:
Cornellian, that's only half the story.

Boalt's low LSAT range is because the school has to admit a certain % of California residents. But that also explains why the average GPA is so high -- the school has a lot of students with high GPAs from mediocre California colleges.

This leads to the best kept secret about the school formerly known as Boalt: if you are a non-California admit, you are virtually guaranteed to be in the top half of the class simply because so many of your classmates don't belong at a top-tier law school.
3.27.2008 10:57pm
Nick Beat:
Theobromophile, you have it backward--25% of the missing are considered employed, 75% unemployed.
3.27.2008 11:58pm
the natural:
"3L:

The employment numbers are fake; we all know that schools pump them up in various ways. At my school-- U Texas-- the actual employment at graduation rate is around 80%. By designating various students as "studying for the bar" or giving them temporary research assistant jobs, the school classifies them as "employed". Of course, the employment numbers are going to go up, even if a law grad has little choice but to work at a bookstore; that's on the stats as corporate employment."

------------------------

3L is right! Law schools lie and distort their employment rankings. My law school counted unpaid internships and non-legal jobs as "employment."

True story: I told career services that my job working for a small firm would end on graduation (in other words they could afford to employ me as a law clerk but not as an associate attorney). I was in the career services office some weeks after classes ended for an unrelated reason, and there was this book out with the names of my graduating class and where they were employed. Next to my name, they'd printed the name of my small firm and a check next to "employed!" Many names had no job at all.

How do law professors from non-elite schools sleep at night? There are NO jobs! Yet you continue to raise tuition, pack in larger 1L classes each year, and hide the truth about the awful job market.

No motivation other than pure greed explains this sad phenomenon. Stop accrediting new law schools, stop raising tuition, stop the lies.
3.28.2008 12:26pm
Marianna (mail):
Joan King at Brooklyn Law School is a liar.

"In September 2007 a Wall Street Journal article was published questioning the integrity of the law school's marketing campaign. It was reported that "a glossy admissions brochure for Brooklyn Law School, considered second-tier, reports a median salary for recent graduates at law firms of well above $100,000. But that figure doesn't reflect all incomes of graduates at firms; fewer than half of graduates at firms responded to the survey, the school reported to U.S. News. On its Web site, the school reports that 41% of last year's graduates work for firms of more than 100 lawyers, but it fails to mention that that percentage includes temporary attorneys, often working for hourly wages without benefits."

Wall Street Journal, Sept/2007
3.28.2008 12:45pm
Ted Frank (www):
frankcross: How in the world do people know how many of their co-graduates have jobs?

Well, in the case of UChicago '94, they were kind enough to send recent graduates an address book with the work addresses of all of my classmates, from which I could determine that they were all receiving mail at institutions that tended not to let people collect mail unless they were employees.
3.30.2008 8:22am