I’ve blogged before about how law professors grade exams, and I wanted to blog some more about some of the difficult judgment calls professors have to make when grading. Exams that get everything right or do everything wrong are easy to grade. But they are also very rare. Most exams do a mix of things well and a mix of things poorly. Coming up with a grade is a difficult and nuanced job that requires applying a meaningful, consistent, and accurate judgment of how that particular mixture should measure up against all the other mixtures.
To see the problem, imagine a law school issue spotter. The professor sees two major issues in the question, issue one and issue two. The first issue requires working through complicated doctrine but has a reasonably clear answer, at least once you work through all the detailed parts. The second issue requires applying a vague standard to ambiguous facts. It’s not hard to see that the standard applies, but applying it well requires a mastery of the facts and the details of the legal standard; the outcome can be argued for either side, as it’s intentionally right down the middle.
Now imagine you have to grade the following 10 exams:
1) Exam 1 gets lost in the doctrine working its way through issue 1, but it ends up getting the correct answer by chance after making two mistakes that canceled each other out. The exam does a decent job applying the standard for issue 2.
2) Exam 2 provides very little analysis of issue 1 and simply announces the result (albeit correctly). The exam then does a very good job of applying the standard for issue 2. The exam then spends time discussing a third issue that is not actually raised by the facts; the discussion reveals that the student does not understand the law relating to that third issue.
3) On Exam 3, the student has a misunderstanding of one of the basic doctrine issues that leads the student’s analysis of both issues to start off on the wrong foot. However, once you get past that one error, the rest of the exam is excellent: The student is getting absolutely everything wrong, but it all seems to go back to one error.
4) Exam 4 goes through the basic issues reasonably well, but refuses to express any view of the strength of any argument or to reach any conclusion. Every argument, no matter how weak or how strong, is described as a position that “may apply.”
5) Exam 5 does the best job of anyone in the class on Issue 1. However, the exam completely misses Issue 2.
6) Exam 6 starts by discussing two issues not actually raised by the question; the answers reveal that the student has a basic misunderstanding of a topic not tested by the question. However, the student then addresses the two issues raised by the question and does a very good job with both of them.
7) Exam 7 starts with an outstanding answer to issue 1. It then starts to answer issue 2, but the student runs out of time after just a few sentences.
8) Exam 8 has misread the facts. However, it does a pretty good job with both issues 1 and 2 if you assume the facts that the student incorrectly thinks exist.
9) Exam 9 is hard to follow, as the student is a poor writer. It’s kind of hard to tell what the student was thinking with the first problem, as it’s hard to follow what the student was saying, but the student does end up with the right result. The analysis of issue 2 is easier to follow, and the student does a so-so job with it.
10) Exam 10 starts off terribly with an analysis that suggests that the student really has no idea what he is talking about. However, at the end of the answer, the student has a section called “alternate analysis” in which the student writes that he “may have misunderstood the problem” and that there is a different approach that is “arguably applicable.” The student then has a short answer to both issues that is exactly correct.
What makes grading so difficult and anxiety-inducing for professors — or at least for me – is that accurate grading requires a consistent and coherent method, rooted in a concept of the purpose of grading, for how high or low to mark each of these possible combinations of errors and correct answers.
Some professors grade with a checklist that gives “points” for certain things, but I have never found that works for me. My ultimate goal in grading exams is to gauge a student’s lawyerly ability. I try to imagine the exam author is another lawyer who I have the option of working with, and I gauge how enthusiastic I would be about working with that lawyer — and how much I would trust and rely on the lawyer’s judgement and analysis — based on the quality of the answer. I don’t think a checklist can really capture that. A checklist creates a kind of objectivity in grading, but the objective thing measured doesn’t seem to relate to anything a future employer would actually want to know. Instead, I make a judgment call, an overall assessment that factors in my judgments as to what each of these errors suggests about a student’s lawyerly ability. It requires a very careful read of the answer, and often a re-read, to get an accurate sense of where in the scale of answers that answer best fits.
Anyway, I mention these things not because I expect readers to have great sympathy for law professors. We have the single greatest job in the world, so no need to feel sorry for us. Rather, I thought readers might just be interested in the kinds of difficult judgment calls that go into grading law school exams.
Finally, I suspect some readers will say that most of the ten exams here made major errors and that most deserve bad grades. If you think that, you have an inflated sense of how well you took exams back when you were in law school. Your complaint is also addressed by the law school curve: In a curved class, it just doesn’t matter if all the answers are great or all the answers are bad. All that matters is the distribution in quality among the various exams that is then superimposed onto the pre-set curve to yield the actual grades.