I just handed in the grades for my Free Speech Law exam, and called and congratulated the A+ and A students, always my favorite part of the grading weeks. (The exam is blind-graded, but once I hand in the final grades, I can figure out who got what.)
I also decide to do some statistical analysis. This exam involved 9 doctrinal short-answer questions, and 1 policy question. Was there much of a correlation between the scores on the doctrinal side and the policy side, I wondered? Likewise, was there much of a correlation in past years between the doctrinal multiple choice and the half-doctrine half-policy essays that I used then?
It turns out that there is indeed a correlation (and, as Joe Doherty at our Empirical Research Group determined for me, a statistically significant one). The correlation coefficient, as calculated by Excel, is 0.40 for this exam, and it has ranged from 0.25 to 0.50 for past exams. A coefficient of 1 would mean that the two raw scores are perfectly correlated (if you did well on one half of the exam, you were guaranteed to do every bit as well on the other). A coefficient of 0 would mean that the two scores were completely uncorrelated — if you did great on the doctrinal side, that wouldn’t at all predict how well you did on the other side.
So what does all this mean? Well, you shouldn’t expect the correlation between doctrinal and policy scores to be perfect — the questions test different sorts of knowledge. Nor should you expect the correlation between multiple-choice (or short answer) and essay scores to be perfect; those also test somewhat different sorts of knowledge (since some material is hard to test using multiple choice and only appears on the essay), different exam-taking skills, and maybe even different problem-solving skills.
At the same time, it would be too bad if the correlation were zero: I would think that people who paid attention in class, who understood the big picture, and who are just plain smart would do better on all parts of the test. If there was no correlation, I might begin to fear what some students already claim to suspect — that law school exams (or at least my exam) are largely random.
So the results that I’ve gotten seem pretty encouraging to me, or at least not disheartening.
Comments are closed.