The University of Denver's Sturm College of Law recently concluded a study of the unacceptably high bar exam failure rate of many of its graduates. (Summary here.) The study found that students whose LSATs were in the bottom 20% of admitted students, and whose first-year grades were in the bottom 20%, were very unlikely to pass the bar. The administration has implemented a program to address the problem, but the professors who conducted the bar exam passage study are skeptical. Professor Sam Kamin writes:
The data that Professor Joyce Sterling and I have collected on more than 2,000 DU graduates indicate that curricular and extra-curricular choices that individual students make – whether to take bar classes, whether to do externships, whether to participate in the student law office, etc. – have little if any significant impact on their bar exam success. Thus, I am concerned that some of the proposed solutions – principally requiring more bar classes and in-class exams – will have no impact on bar passage and might mislead students into believing that this complex problem has a simple solution.
Professor Sterling and I found that the best predictors of poor bar exam performance are very low LSAT scores and low law school grades. Thus, the data indicate that instead of making broad curricular or pedagogical changes, the most likely path to improving our bar pass rate is to cease admitting students without a substantial likelihood of bar exam success, to identify at-risk students among those admitted, to help them develop the skills they need to succeed in law school, and to fail out those that we cannot find a way to help.
UPDATE: A commenter asks for DU's ranking. According to the latest U.S. News & World Report ranking, DU is in a tie for 95th place. Although it's not top tier, it is ahead of dozens of other law schools, many of which, I suspect, also have abysmal bar passage rates for the bottom 20% of their students.
I don't want to open a can of worms here, but it would also be useful to know if Sturm has an affimative action program and what percentage of students in the affirmative action program failed the bar.
Looking at that summary, it seems that the following are most important (and seemingly not dealt with in the proposed solutions).
-Grade inflation leading to students thinking they are better prepared for the bar exam than they really are.
-Failure to dismiss students who are not making satisfactory progress in law school.
-Statistical analyses indicating that the bottom 20% of the class after first year are unlikely to pass the bar regardless of the course of study.
-Lower LSAT students not being given enough extra assistance early and not being dismissed later when it is apparent they have not performed in a satisfactory manner and likely will not pass the bar.
It seems like raising the LSAT minimum and enforcing a mandatory curve in the first year along with dismissing anyone below a certain GPA level (it's C- where I am) would solve most of their problems. None of those were listed as solutions, though low achievers must "consult with the Academic Dean or Dean of Student Affairs to discuss their situation and make a plan for improvement." Gag me.
The upshot is, "we want the tuition from these people, but they are making us look bad so how can we look like we're doing something without losing the cash?"
1. News that folks who have trouble getting into, or passing out of, law school have trouble passing the bar is indeed not shocking.
2. Back when I was a young pup (cough, hack) in the 1970s, there were some antitrust suits against Bar assns. Theory was that the task of bar admissions was to restrict entry into the trade, not to set a minimum standard of skill. Proof (as best my sclerotic brain can recall) was that bar pass rates within a given state were uniform, year to year -- and inference was that they just passed the top 70%, 80%, whatever. I know that in my year there was a little problem with the ethics portion. They "hid" one of the five questions inside a seemingly pure evidence question. I saw it, but my informal poll indicated that about a third of people didn't. Since getting a zero on one out of five should have killed you, a full third should have flunked that ... yet the failure rate was no different than any other year.
If that's the case, you could take the bottom 20%, or 50%, or any other proportion, reject them, and bar pass rates would remain the same. A certain expected and customary proportion would be skimmed off and flunked no matter if they were composed of people picked off the streets, or of editorial boards of the top ten law reviews.
Being first on the list may or may not imply importance, but probably does imply that it is a significant aspect of failure to pass the bar.
This should be a rather easy one to fix.
It may reflect a difference in emphasis teaching at UCLA. I seem to recall that when I went to school there (early 80s), UC Davis consistently had the highest bar passage rate among state schools. As I recall, the standard "UCLA explanation" (at least among students) was that UC Davis made a point of teaching California law (the law school equivilent of "teaching to the test"), while UCLA, having aspirations of being a "national" law school, did not. That is, UCLA taught "national" common law (e.g., "the majority rule says," "the minority rule says," "the restatement / model [fill in] code says").
I wonder if there has been a change in emphasis. I really have no idea. Just speculating.
Secondly, they have been pinched for a number of years between the desire to admit many more minorities, and that of many higher tier universities to do the same. The result is that the quality (and, thus, bar pass rate) of the admitted minorities has plummetted much faster than that of the non-minority admittees.
One problem with anonymous grading is that it is hard to cover up the minority quality problem without grade inflation. And remember, they are trying to essentially make money, so flunking out too many students would be bad for the bottom line (as noted above, the University as a whole is non-profit, but the law school is run as a profit center to finance the rest of the school). Minority flunk out is apparently still a problem, even with grade inflation.
I may be wrong but I thought that California urrently tests for state law only on the Community Property setion, so I don't think the state vs. national issue has much relevance today.
It is commonly said that the lower the rank the more the curriculum is geared diretly toward bar passage.
There's hardly any California-specific law on that sucker, although they're adding Cal Civ. Proc. in 2007.
[Reposted because the "blockquote" didn't work the first time.]
Two amusing points on the multistate (again, dating to the 1970s):
In my time, in Arizona, the passing score on the MBE was 50%. Reason: they had just released an older exam with proper answers. The chief justice of the state supreme court took it, and checked against the answers ... he got right around 50% and would have flunked. The next time the salesman for the MBE came in, he got an earful. And thus for at least some years, the rule was that if you could do as well as the Chief Justice, you passed.
A classmate's wife, who was (as I recall) an elementary teacher, took sample, exam ... and passed with a rather nice score. That inspired her to enter law school, and she's a prosecutor today. Yes, an intelligent person, with the ability to "take a test," was able to pass without a day of law school, and no legal training save talking to her husband as he went through law school.
Information on the July 2005 bar exam pass rates, sorted by law school.
One interesting aspect that might distinguish Denver is that Colorado has a very "progressive" bar exam, requiring a unique problem/project. I don't remember all the details but I believe it is a packet with some interesting client in which one evalutes information given, write some petitions, etc. If memory serves, this project takes up 1/2 or even a whole day of the exam. How might this aspect relate to the problem? Might this type of project me more suspectible to in-school preparation?
Finally, if there is any more than a weak link between lawyering abilities and bar exam results, I've never seen evidence of it. General intelligence seems more predictive.
Many of the commentators here seem to uncritically accept the notion that law students who do better on exams that stress memorization and time pressures will do better in a profession that stresses research and due diligence. The assumption has no basis in reality.
I also think that the solicitude for the tax-payers of Colorado — who may just be throwing their money away to subsidize the educations of students who do not pass the Bar on their first try — is misplaced. National statistics show that over 90% of exam-takers pass the Bar by their third try, so those taxpayers will probably still get their money's worth.
Definitely the most intelligent post on here. For some reason, it seems that many commenters just don't get it. As you have mentioned, these silly tests are about memorization and regurgitation, not about diligent and careful lawyering. The commentators suggesting weeding out students or speculating about the supposedly harmful affects of affirmative action should NOT be taken seriously since they failed to address the issue of whether the bar exam itself is useful. If the bar exam tests mostly useless memorization and regurgitation ability, it doesn't say much about those who pass it other than that they are good monkeys. Why not have a bar exam that actually means something??? It is a pretty strange world when Harvard Law School and other elite schools have some students who fail the bar. Why waste so much time, energy, and money on a useless system?
I think the answer lies with some of these commentators. They just ASSUME that somehow passing the bar means something, and thus inertia and the irrational status quo are perpetuated. Sometimes, I think we live in a bizarre world when such obvious problems are not addressed. I guess that is the power of inertia.
Many of the commentators here seem to uncritically accept the notion that law students who do better on exams that stress memorization and time pressures will do better in a profession that stresses research and due diligence. The assumption has no basis in reality.
Not necessarily so. The Bar exam does reward diligence in preparation. People who fail it are usually those who did the least to prepare and put the least effort into it. To the extent they take that beyond the exam, it is an effective proxy for diligence.
In a perfect world, there would be no bar exam and the market would determine qualifications. That's not the world we have, and until we get there, students will have to pass the bar exam and law schools that admit and advance students whom the law schools know will not be able to pass the bar exam are not serving their students well.
I've noticed that on a lot of conservative/libertarian blawgs the authors are all for free trade, so long as it only costs the wages and jobs of lower skilled workers. I've not seen any of them get wroked up about the protectionist nature of te Bar.
Yes, yes, I know that they will say the bar serves a greater function than simply helping to keep low price competition from entering the market upon threat of imprisonment. But anyone who's practiced for any length of time knows enough schlubs who test well to know that the Bar is imperfect at this test, to put it charitably.
The Bar exam does reward diligence in preparation. People who fail it are usually those who did the least to prepare and put the least effort into it.
Anyway, just look at the ridiculous number of students, about 5-6 times what my school has in each class. I'll bet the students there couldn't get 5 minutes with a Prof for a sit down about res judicata, CE, and interpleader for 45 minutes like I had 3 days before finals. And, my first yeart Torts, Contracts, and Crim classed had about 35 people in them. I'll bet U of D had 350.
Bottom line: The bar is the great equalizer, and forces you to actually learn something in law school. Denver has only one option: lower their class size. Of course, with that type of $$$ at stake, you can forget that.
It isn't surprising that the DU students with lower grades and LSATS have lower bar passage rates--after all, it isn't likely that those students test well (side note: I think there is a huge difference between testing well and having the skills a good lawyer needs.) The question is: how do the DU students with low test scores and low grades compare to students at peer schools who came in with similar LSATs and have similar class rank. If the DU failure rate is much higher than those of its peers, it seems to me that it is safe to assume the school can do things to close that gap.
Mercer was always several points lower than the other three with pass rates in the low 80% range, but the others were always tightly packed in the mid 80%s. Georgia State always ended up a few tenths of a point ahead of the other two. I don't know what this says about the Bar exam, legal education, or what being a "top, mid or lower tier" law school means, but it must mean something.
Go team!
The Bar exam does reward diligence in preparation. People who fail it are usually those who did the least to prepare and put the least effort into it. To the extent they take that beyond the exam, it is an effective proxy for diligence.
</blockquote>
Ditto. You can criticize the bar's effectiveness at gauging ability to be a lawyer, or even its necessity, until the cows come home. It remains that a test, regardless of its substance, that requires takers to sit down and study <i>something</i> and exacts a significant amount of stress, <i>will</i> predict, most of the time, how good someone will be at a job that requires you to sit down and research and distinguish details, and exacts a significant amout of stress.
If it were up to me, I'd abolish the bar exam altogether. As SR pointed out in his/her excellent remark above, the bar exam does little but perpetuate the inadequate ideals (misplaced) of a long-ago generation where "minimal competence" is measured by 200 mult choice questions in 6 hours. Law schools who care about abysmal bar passage rates would be better served by focusing on initiating bar prep programs as early as 1L, instead of hiking up their LSAT requirements for admission. From 1997-1999, my Tier 2 law school in NYC accepted 5% of applicants with LSAT scores of 145-150. In 2004, the school accepted 0% of applicants with those LSAT scores - yet the school's bar passage rates have remained the SAME since 1999. Explain that one!
There was an article in the Houston Chronicle this last weekend about a man threatening to sue the state bar because you can only take the Texas bar exam five times and he had already flunked it four times. Would you want this man as your attorney?
Make no mistake, those 200 multiple choice questions are a bitch. The only thing we found interesting in Bar/Bri was the definitions of the different kinds of sodomy. Somehow I missed that in class.
Really? I found it rather markedly different. I didn't take an LSAT cram course -- I just used the little CD they sent along and did the practice questions on that -- but I suspect that if I had done, there would have been more similarity between my bar prep (Bar-Bri) and that cram course than my law school experience.
I think it really depends on how you define "knowledge." For the MBE, I know my knowledge-free multiple choice test-taking skills are quite good, but the MBE I actually felt I had to study for, much more so than the essays. In both cases, though, the kind of knowledge required was not really understanding or deep knowledge, so much as mere memorisation -- important, I think, and undervalued, but not particularly difficult. Just tedious.
Oh sure. But I think that's their fault, not so much because they're making standards more elastic to get more Blacks and Hispanics or anything, but because they just pass people on through. Shamefully enough, my experience at an elite law school was that you really had to do pretty badly on just about everything to get below a B on an exam.
There was a brief discussion of this issue in a post on Prof. Bainbridge's site a while back, in which I did argue that the Bar ought to be abolished and the market (and post-hoc state regulatory liability) would be adequate/superior protections in the legal field. But then, I'm not a law professor, and certainly do not run a blawg. On the other hand, I am conservative, if not libertarian, so that's almost there.
That said, though, I also have to admit that I'm sympathetic to a different model of law study, where you don't have to blow three years and a ton of cash on law school -- you just sit down, read the law, and take the exam. After you've passed, you apprentice yourself to some senior lawyer somewhere, and go from there. If the exam functioned in that context, I'd look at it more positively, as a kind of access/qualifications equaliser. As it is, though, you pretty much have to go to a law school and pay out the requisite cash.
I'm not sure I agree entirely with this regarding the predictive utility of the bar, but contra SR above, I do think that diligence is a major, and possibly the major factor in passing. The caveat would have to be that you can have bad luck, nerves, or a bad study plan, and all the diligence in the world won't save you then.
Isn't EVERYONE on the planet of some ethnic descent? Someone was just afraid to laundry-list the intended groups, and was afraid to write "minority." And of course, if one really breaks down "Euro-American" into Polish, Irish, etc., aren't we ALL minorities?
Three years after graduating from law school -- having served a judicial clerkship and worked abroad in the interim -- I took my first bar exam (in NY). I took it without so much as opening a book to study, because I had been working 18-20 hour days on an M&A deal for the better part of the two months leading up to it.
Despite being three years removed from the subject matter, not studying a bit and suffering from massive sleep deprivation, I passed the damned thing by a wide margin. As I should have, the exam isn't hard. Anyone with (a) minimal intelligence; (b) a semblance of a legal education; OR (c) the discipline to actually study can pass the damned thing. And anyone who can't manage to do that is too stupid and/or too slothful to be allowed to practice law.
Perhaps the bar exam doesn't test skills that one will use in 'real life'. Neither does an IQ test. But like and IQ test it does a pretty good job of identifying morons. And God knows, it's not like there aren't enough of them in the profession already.
Plus, quit your sniveling. In Japan, to become a lawyer (bengoshi) you have to pass the Japanese bar exam (the Shiho Shaken) which historically had a less than 3% passage rate. Those who do pass usually had to take the exam 5 or more times and most law school graduates never managed the feat.
Oh, and passing merely gives one the opportunity to enter an additional two-year long training program.
http://tinyurl.com/7zl5g
Having practiced in both Japan and the US, I must say the Japanese system results in a substantially less embarassing bar membership.
DU has the only night program in Colorado, and thus, attracts a lot of non-traditional students. However, at least in the past, their bar pass rate was not that much different from the much more traditional day students. Needless to say, CU seriously discourages any outside employment during at least your first year. I should also note that though CU grads would typically have a slightly higher bar pass rate, it wasn't nearly this noticable really until very recently. For most of its over 100 year existance, the DU law school was located near downtown Denver. This allowed easy access to the various state and federal courts, and resulted in relatively good hands-on training for aspiring lawyers. Also, it was convenient for practicing lawyers to drop by and teach an occasional class as adjuncts, again, resulting in a more practical legal education than was found in Boulder. (This last year, the DU law school finally joined the rest of the university south of downtown).
Finally, reiterating my point above. DU is a private, non-profit, university, offering a range of courses of study (Secretary of State Rice got her BA and PhD there). However, most of its schools lose money. The two exceptions are Business and Law. Both schools are "cash cows" for the university, and, essentially are run as profit centers to underwrite the rest of the university.
I think that this is missed by many. A law school can be quite profitable for a university once a critical mass is reached. That is because some of the more expensive costs of running a law school (like the required library) are not linear, but rather closer to a step function. The class rooms are a sunk cost, funded by the alums in fund drives. And in a private university like DU, with the resulting high tuition, the faculty and other salaries, along with other variable costs are much lower than the resulting tuition. And, thus, an incentive to push up the number of students to the highest possible levels.
Unfortunately for DU, I think that they have found one of the upper limits on this profit-making: a low bar pass rate is inevitably translates into a lower desirability as a law school. And hence the study, that presumably will rein this in a little bit. Also, another possible upper limits is the williness of alums (who just built all new law school facilities) to contribute - which is probably also positively related to bar pass rates.
By the way, it's highly likely that Prof. Sullivan failed the California bar only because she didn't study for it - and, as noted, the bar measures knowledge of minutae which most lawyers either never studied in law school or never use in practice.
I stand corrected. Thanks.
p.s. I took an online European Union Law course over the summer from DU. It was a great course and I learned alot.
I'd go further than Observer and some others, and say that a sufficiently bright high school student, given ten weeks and a personal instructor, could pass the bar. (I
didn't take Bar/Bri or any of those, so I can't really speak to them, but they struck me as wasteful for people with access to law books.)
My law school was a for-profit, non-ABA accredited night school. They let everyone in whose check cleared; that was 53 people for the first class. By the end of four years, there were nine of us, of which eight took the bar. All eight passed on the first try.
The real mystery to me isn't how some large percentage of Denver University's students don't pass, but how 12% of top-tier students don't pass. That strikes me as pretty amazing; there aren't any football scholarships to law school, right?
--JRM
Haha -- unsurprising, no, since American-style law schools were only introduced in what, 2004? I don't think they've graduated their first classes yet. Instead, people just graduate from undergraduate law study before they take the exam. It's the same system in Korea, and China too, I believe. Also, I think a more natural rendering would be shihou shiken, 司法試験.
Regarding whether the bar membership is more or less embarassing than here -- I have no real opinion on that.
My understanding, however, is that our ideas of what constitutes excellent lawyering are somewhat different here than there. In particular what one might call "creative lawyering" appears to be more highly regarded here than over there -- or so I have been led to believe both from my (extremely limited) study of Japanese law and from comments from practising Japanese lawyers. My own view of "creative lawyering" is that in the long run, it undermines the predictability and the fairness of the legal system, but I am, I think, in the minority over here.
If you were commenting only on their personal probity, their preparation, and their self-presentation, though, I'd say that's just a matter of Japanese society, not their diabolic bar -- everyone seems better turned out and prepared there than here, at least in business contexts. And there, again, I am not sure I agree -- I don't know enough Japanese lawyers (or American lawyers, frankly) to comment.
I worked for four years between my undergraduate degree and entering law school. My undergraduate degree was in Chemistry. I did very well on my LSAT, scoring in the 98th percentile. What really shocked me about law school was that although the work was hard in the sense that there was a lot of it and the teachers were demanding and some were condescending and demeaning, I didn't find it intellectually challenging. Certainly, it didn't require logical or rational thinking in the normal or classical meaning of those terms. I was especially shocked by the sloppy scholarship and just intellectual dishonesty of what were supposed to be the great minds of modern jurisprudence and legal thought on both the left and the right (I came to especially detest both Posner and Catherine McCinnon).
But what really baffled me was my fellow students. As an undergraduate I carried a solid 3.0 gpa. Chemistry is not an easy field of study and grade inflation was unheard of. I admit that there were some semesters when I was less than serious and my grades suffered as a result. But as I looked at my peers and my own performance, I always felt that I got the grades I deserved and that those who did better than me were either just smarter or were working harder (and that those who did worse were dumber or lazier). In law school, I ended up in the top 25% of my class, yet I could never figure out why some people did better than me or some people did worse. Some people who were at the top of the class I considered very bright people, others I thought were as dumb as a box of rocks. Likewise for some who struggled near the bottom of the class. Nor did it seem like the effort people put into it resulted in better or worse grades. One thing I did notice is that I seemed to get consistent grades from different professors eventhough our tests were all graded anonymously (we received a number before our exams and we put this on our tests rather than our names). In fact I had one professor who I took for three courses, where I got the exact same grade (an 84) on every single exam. And another Professor that I took for three courses and got three A's (obviously he was my favorite).
1. DU's first-time takers passage rate was 70 percent. I don't think this is a huge cause for concern. The passage rate for first-time failers was abysmal, but this is to be expected.
2. The study says a combination of low LSAT scores AND low first-year grades is a good predictor of bar exam failure. As a practical matter, it is impossible to "weed out" this class. You would have to plan on failing out a certain percent (20? 30?) of first-years, but only those who ALSO have low LSAT scores, and perhaps keep low performers who were admitted with higher LSAT scores.
3. So you can't do that. But you CAN fail out a very substantial percentage of low performers. That is exactly what my law school took to doing in the late '80s in an effort to increase their California bar passage rate. If I recall, about 30% were not "invited back" after their first year. The result? Bar passage rates went from around 50% to over 70%. Remarkably effective. Interestingly, it turned what should have been an easy-going So Cal law school into an anxiety-riddled pressure cooker. I remember being astounded by the lax approach of the Stanford students I worked with in the summer. They KNEW they were going to graduate, and the bar exam was something they could study for after graduation.
4. DU does indeed have a night program. It would be interesting to see a breakdown of the bar passage stats by day vs. night program. My guess is that the night program students have a pretty horrific passage rate. But I have no doubt that the night program is a "cash cow" for the University, and that abolishing the night program is not politically feasible.
5. On bar passage rates year-to-year: my experience is that California does a pretty good job with this. When becoming a lawyer was momentarily a sexy occupation (think "LA Law"), law school applications soared. So-called 3rd and 4th tier schools suddenly could choose much-better qualified applicants. Cal bar passage rates reflected this quality increase, going up substantially in the 1990s. They are back down again. In other words, the Cal bar exam DOES seem to strive for some absolute standard of competence rather than a rigid curve ("the top 70% pass") in any given year. I applied to become a grader for the exam at one point -- I didn't make the final cut, but I went through some training materials. I was impressed by the attempt to achieve consistency across graders. Indeed, the consistency of grades in what is considered a fairly subjective process (essay questions) was quite remarkable.
6. Having said that, the Cal bar exam certainly has its problems. I don't think it's "too hard" in general. Having some insight into the grading process, I have to say my #1 complaint is this: essay question/performance exam grading puts an extreme emphasis on issue spotting. There is almost no weight given to the ability to write clear English. And having practiced in California, I was continuously astounded by the almost complete lack of English writing ability among many of my colleagues. This lack of testing of writing skills has what some would consider a side benefit: it allows intelligent people for whom English is a second language to pass the exam. But that's also one of the problems: intelligent people who don't know English very well can readily pass the exam ...