pageok
pageok
pageok
"Last Lecture" a Lesson in Great Teaching:

Charles Lipson of the University of Chicago has an appreciation of Randy Pausch as to how the last lecture itself exemplifies the qualities of great teaching (not to mention why so many of us find teaching such a rewarding life's mission). One thing that always struck me as sort of unusual about Pausch's "last lecture" was the use of Powerpoint to give such personal reflections.

In a similar vein, there's a neat NBER paper by Scott E. Carrell and James E. West on what makes a good professor measured by the impact of professor quality on student learning. The results are somewhat surprising. Here's the abstract (unfortunately NBER papers are limited access):

It is difficult to measure teaching quality at the postsecondary level because students typically "self-select" their coursework and their professors. Despite this, student evaluations of professors are widely used in faculty promotion and tenure decisions. We exploit the random assignment of college students to professors in a large body of required coursework to examine how professor quality affects student achievement. Introductory course professors significantly affect student achievement in contemporaneous and follow-on related courses, but the effects are quite heterogeneous across subjects. Students of professors who as a group perform well in the initial mathematics course perform significantly worse in follow-on related math, science, and engineering courses. We find that the academic rank, teaching experience, and terminal degree status of mathematics and science professors are negatively correlated with contemporaneous student achievement, but positively related to follow-on course achievement. Across all subjects, student evaluations of professors are positive predictors of contemporaneous course achievement, but are poor predictors of follow-on course achievement.

taney71:
So what makes a good professor? Also does the books and course assignments he or she picks out come into play as aspects of good teaching?
7.29.2008 3:37pm
Curt Fischer:

We find that the academic rank, teaching experience, and terminal degree status of mathematics and science professors are negatively correlated with contemporaneous student achievement, but positively related to follow-on course achievement. Across all subjects, student evaluations of professors are positive predictors of contemporaneous course achievement, but are poor predictors of follow-on course achievement.


Wow, one conclusion that jumps out from this discussion is "the system works". The "best" professors as chosen by the university administration (as measured by academic rank) seem to be the "best" able to propel their students to success.

Another conclusion is that reliance on student evaluations to measure teacher performance is extremely dangerous. It incentivizes professors to simply hand out As to everyone.

Why don't universities start asking for student evaluations of a course a year or so after the course has ended? The current practice is to solicit student evaluations at the end of the course, but this post suggests it would be far better to let students experience how well (or not) the present course prepares them for their next before deciding how well they rate the present course's instructor.
7.29.2008 6:03pm
dave hoffman (mail) (www):
I've put up a copy of the Carrell/West paper, and written a bit about it, here.
7.29.2008 6:04pm
former science student:

One thing that always struck me as sort of unusual about Pausch's "last lecture" was the use of Powerpoint to give such personal reflections.

Using Powerpoint for all manner of presentations is the norm for those in the sciences (my background includes biology and computational science); research, class lectures, and "comedic" performances at conferences all get the same treatment of few words on screen, but plenty of visual aids for comedic or dramatic effect. In fact, I was surprised to get to law school and have professors who *didn't* use Powerpoint that way!
7.29.2008 6:42pm
Dan Simon (mail) (www):
If I've interpreted the results correctly, they suggest that more junior professors coddle their students, giving them good marks and few challenges, thus earning good evaluations while preparing their students poorly for future courses. More senior professors, on the other hand, make their courses harder and award lower marks, harming their own evaluations but imparting more knowledge and a better work ethic to their students.

Sounds at least plausible to me...
7.29.2008 7:41pm
JustMe:
+1 for what Dan Simon says.
7.29.2008 9:14pm
Zywicki (mail):
Dan Simon: Yes, I thought exactly the same thing. The word "surprising" isn't exactly right because it is one of those things that it seems surprising at first but is less so once you think about it a bit more.
7.29.2008 9:27pm
David Warner:
"We find that the academic rank, teaching experience, and terminal degree status of mathematics and science professors are negatively correlated with contemporaneous student achievement, but positively related to follow-on course achievement."

Sorry to repost yet again, but its just that good of a sentence. I don't know that I've ever read such a succinct yet comprehensive encapsulation of my experience as educated and educator.
7.30.2008 12:06am
neurodoc:
Supposedly (can't document it) at some medical school (UCLA?) years ago, an actor was rehearsed to give a lecture to unsuspecting medical students, and his "performance" was rated much higher than was a lecture on the same subject by a real professor who actually knew what he was talking about, so wasn't just mouthing words that had no real meaning to him. So, should actors be employed to lecture medical students based on scripts prepared by those who really know the subject matter? (Communication skills are important for leadership purposes, so should actors be preferred as presidents?)

I can believe that more accomplished faculty (higher "academic rank, teaching experience, and terminal degree status") are tougher graders and that tougher graders find less favor with students. (I can also believe that more junior faculty often try harder than senior faculty when teaching undergrads, and their students are appreciative of the extra effort and enthusiasm.) I am reluctant, however, to accept that less favorable student evaluations imply that the teacher is more likely to have done a better job of preparing his/her students than those who have received more favorable student evaluations. There is some logical error, isn't there, in concluding the latter probably follows from the former.
7.30.2008 1:22am
Curt Fischer:

I am reluctant, however, to accept that less favorable student evaluations imply that the teacher is more likely to have done a better job of preparing his/her students than those who have received more favorable student evaluations.


I agree that less favorable evaluations are unlikely to imply that a teacher is doing a better job. Fortunately we don't have to make such a dubious implication: as the text quoted by Prof. Zywicki makes clear:


We find that the academic rank, teaching experience, and terminal degree status of mathematics and science professors are [...] positively related to follow-on course achievement. Across all subjects, student evaluations of professors are [...] are poor predictors of follow-on course achievement.


In other words, the highly ranked professors teach students that go on to do better later. The student evaluations have nothing to do with later success. These conclusions do not rely on the assumption that evaluations correlate with anything.
7.30.2008 1:42am
David Warner:
Exactly. "Poor" here doesn't mean negative correlation, it means non (statistically significant) correlation.*

I'm wondering, does this imply that the overrating of puffball profs and underrating of tough/effective profs is prevalent enough to cancel out entirely the correct high scoring of outstanding teachers and low scoring of incompetent ones?

Sounds like there is an opportunity here to fairly dramatically improve students' grasp of their own learning processes somewhere along the line.


* - I suppose it could mean that the author thought it should be .6 and its only .4, but I doubt it.
7.30.2008 2:24am
Curt Fischer:


Sounds like there is an opportunity here to fairly dramatically improve students' grasp of their own learning processes somewhere along the line.



Maybe by having students wait a year to fill out course evaluations? Not only would it give some perspective about how a class sets them up (or doesn't) to succeed in a more advanced class, it would also force students into a deeper introspection about their own learning processes and educational trajectory too. Win win!
7.30.2008 9:23am