The Volokh Conspiracy - Does Scholarly Productivity Improve Teaching Quality in Legal Academia?

There is a longstanding argument over the question of whether scholarship improves teaching in academia or detracts from it. Some claim that productive scholars are better teachers because they have a greater command of the subject and more original insights to convey to their students. On the other hand, it's also possible that scholarly productivity detracts from teaching. The time academics spend writing articles and books could instead have been devoted to improving their teaching skills.

Ben Barton of the University of Tennessee Law School has an excellent new article testing the impact of scholarship on teaching. He shows that there is little or no correlation between scholarly productivity and teaching ability (as measured by student evaluations) in a sample of over 600 law professors from 19 schools. Thus, scholarship neither improves teaching ability nor detracts from it.

This study is an important advance over the existing literature. But I have two significant reservations about it. First, Barton was not able to find data controlling for other variables that might affect teaching quality. For example, the quality of teaching might be influenced by a professor's speaking style, organization, personality, and so on. Some of these omitted variables might well be inversely correlated with scholarship. If so, it's possible that if we controlled for them, we might find that better scholars are better teachers after all.

Second, student evaluations are a highly imperfect measure of teaching quality; some of their shortcomings could end up biasing Barton's results against the hypothesis that scholarship improves teaching. I'm not one of those professors who thinks that student evaluations are useless. To the contrary, they often contain important and valid criticisms of the professor. My own teaching, I believe, has improved as a result of attending to such critiques. Nonetheless, teaching quality is far from the only variable that influences student evaluation scores. For example, we know from previous research that scores go up if the professor makes the class easier. Evaluation results are also influenced by such irrelevant factors as the professor's physical attractiveness. Although I don't know of a study testing this, I'm willing to bet that student evaluation scores are heavily influenced by the quality of the professor's sense of humor.

Some of these factors may well be inversely correlated with scholarship. For example, it's possible that the most productive scholars make their courses harder than those of the less productive ones, thus taking a hit on their evaluations. It's even possible that more productive scholars are, on average, not as well-dressed as their less productive counterparts (perhaps because they spend more time doing research and less time paying attention to fashion trends). If so, this too would lead them to get lower evaluation scores.

In Barton's defense, it's very difficult to get data on many of the relevant control variables. And student evaluations are perhaps the only available quantitative data on teaching effectiveness at most schools. Overall, I think he did the best he could with the available evidence. However, I don't think his evidence is good enough to decisively reject the theory that scholarly productivity improves teaching.

	[Ilya Somin, September 10, 2008 at 11:52am] Trackbacks Does Scholarly Productivity Improve Teaching Quality in Legal Academia? There is a longstanding argument over the question of whether scholarship improves teaching in academia or detracts from it. Some claim that productive scholars are better teachers because they have a greater command of the subject and more original insights to convey to their students. On the other hand, it's also possible that scholarly productivity detracts from teaching. The time academics spend writing articles and books could instead have been devoted to improving their teaching skills. Ben Barton of the University of Tennessee Law School has an excellent new article testing the impact of scholarship on teaching. He shows that there is little or no correlation between scholarly productivity and teaching ability (as measured by student evaluations) in a sample of over 600 law professors from 19 schools. Thus, scholarship neither improves teaching ability nor detracts from it. This study is an important advance over the existing literature. But I have two significant reservations about it. First, Barton was not able to find data controlling for other variables that might affect teaching quality. For example, the quality of teaching might be influenced by a professor's speaking style, organization, personality, and so on. Some of these omitted variables might well be inversely correlated with scholarship. If so, it's possible that if we controlled for them, we might find that better scholars are better teachers after all. Second, student evaluations are a highly imperfect measure of teaching quality; some of their shortcomings could end up biasing Barton's results against the hypothesis that scholarship improves teaching. I'm not one of those professors who thinks that student evaluations are useless. To the contrary, they often contain important and valid criticisms of the professor. My own teaching, I believe, has improved as a result of attending to such critiques. Nonetheless, teaching quality is far from the only variable that influences student evaluation scores. For example, we know from previous research that scores go up if the professor makes the class easier. Evaluation results are also influenced by such irrelevant factors as the professor's physical attractiveness. Although I don't know of a study testing this, I'm willing to bet that student evaluation scores are heavily influenced by the quality of the professor's sense of humor. Some of these factors may well be inversely correlated with scholarship. For example, it's possible that the most productive scholars make their courses harder than those of the less productive ones, thus taking a hit on their evaluations. It's even possible that more productive scholars are, on average, not as well-dressed as their less productive counterparts (perhaps because they spend more time doing research and less time paying attention to fashion trends). If so, this too would lead them to get lower evaluation scores. In Barton's defense, it's very difficult to get data on many of the relevant control variables. And student evaluations are perhaps the only available quantitative data on teaching effectiveness at most schools. Overall, I think he did the best he could with the available evidence. However, I don't think his evidence is good enough to decisively reject the theory that scholarly productivity improves teaching.