Comment on a Selective Review of the Validity of Student Ratings of Teaching
In a recent issue of this journal, Dowell and Neal [10] selectively reviewed empirical studies that attempted to validate student ratings by correlating them with student learning. By combining results from six studies, they calculated a rating/learning correlation of 0.20 and concluded that the research evidence showed unimpressive estimates of the validity of student ratings. In the authors' words, "Ratings have a long way to go to gain respectability" [10, p. 60]. In this article, I will review the same literature and seek to restore some respectability to student ratings. Dowell and Neal set out to review validity studies that included three elements of research design: (1) the use of section means as the appropriate unit of analysis, (2) the use of controls for initial student ability, and (3) the use of multidimensional criteria against which to validate student ratings. They found no studies that used section means, ability controls, or multidimensional criteria and consequently selected only six studies that used section means and ability controls. An overall estimate of the validity of ratings was obtained by averaging all rating/ learning correlations within a study, transforming the six averaged correlation coefficients into Z-scores, and then calculating a combined Z for all studies. Unfortunately, there are some major problems with Dowell and