Via a list of links (Volokh to Prawfsblawg to Pub Sociology to Andrew Gelman), we are led to a study that addresses one of the crucial pressing issues of the contemporary academy: how (or why) do some people get good teaching evaluations? Answer: by giving good grades and by being sexy. Here’s the abstract of the paper, by Felton, Mitchell, and Stinson:
College students publicly rate their professors’ teaching at RateMyProfessors.com, a web page where students anonymously judge their professors on Quality, Easiness, and Sexiness. Using the data from this web site, we examine the relations between Quality, Easiness, and Sexiness for 3,190 professors at 25 universities. For faculty with at least 10 student posts, the correlation between Quality and Easiness is 0.61, and the correlation between Quality and Sexiness is 0.30. Using simple linear regression, we find that about half of the variation in Quality is a function of Easiness and Sexiness. Accordingly, these results suggest that about half of the variation in student opinion survey scores used by universities for promotion, tenure, and teaching award decisions may be due to the easiness of the course and the sexiness of the professor. When grouped into sexy and non-sexy professors, the data reveal that students give sexy-rated professors higher Quality and Easiness scores. Based on these findings, universities need to rethink the use of student opinion surveys as a valid measure of teaching effectiveness. High student opinion survey scores might well be viewed with suspicion rather than reverence, since they might indicate a lack of rigor, little student learning, and grade inflation.
Now, I admit I haven’t read the paper itself very carefully. And I’m always reluctant to criticize credentialed experts in fields outside my own (really, I am). But this abstract doesn’t fill me with a great deal of confidence.
First, the data come from RateMyProfessors.com. You’re kidding, right? Perhaps there might be some sort of selection bias in the students who take the time to fill in entries on the web site? More importantly, the authors seem to take for granted the existence of a priori categories of “easiness” and “sexiness,” and claim that their existence is distorting the sought-after measure of “quality.” It seems not to occur to them that, for example, sexiness and quality might correlate because they both are caused by some third external factor. Or that a class might be subjectively rated as relatively easy, not because the grade distribution was actually higher, but because the students came away with the feeling that they had really understood the material. Or, most likely of all, that students found certain professors to be sexy because they were good teachers. Effective pedagogy, you don’t need me to remind you, is hot.
(And no, I don’t have an entry at RateMyProfessors.com. But my teaching evaluations are pretty good. Draw your own conclusions.)