Research increasingly questions the usefulness of course evaluations. It is true that the surveys administered across colleges and universities provide feedback to the educator. However, evaluations are also used to measure learning and teaching effectiveness, something that statistician Philip Stark and teaching consultant Richard Freischtat from UC-Berkeley claim evaluations do not do.

One of the reasons for that is that non-instructional factors influence the ways students perceive classroom experiences. For example, Basow and Martin demonstrate that the identity of the instructor consciously or unconsciously influences the final ratings (see additional examples below).

And yet, the results from those surveys are still used to judge and compare educators' performance. If the perception is that evaluations offer bad data that lead to the wrong answers, as Stuart Rojstaczer argues, ethically questionable solutions should not be surprising. The coping strategies described by The Chronicle's Stacey Patton illustrate that idea.

The solution, some argue is in focusing on other mesures of teaching effectiveness, e.g. peer evaluations. A solution like that makes a lot of sense when the survey results are used to promote or punish educators. Still, in the age of big data and analytics, discarding the survey data seems like a waste. Given sufficient numerical data, a combination of statistical factor and cluster analysis can offer a highly customizable yet standardized algorithm for mining evaluations.

Select sources on the various biases of course evaluations:

Gender

MacNell, Lillian, Adam Driscoll, and Andrea N. Hunt. "What’s in a name: exposing gender bias in student ratings of teaching." Innovative Higher Education (2014): 1-13.

Race, ethnicity

Smith, Bettye P. "Student ratings of teaching effectiveness: An analysis of end-of-course faculty evaluations." College Student Journal 41.4 (2007): 788

Native language, nationality

Amin, Martin E. "Students' Sociocultural Background as a Discriminating Factor in the Evaluation of Teaching in a Bilingual University in Central Africa." Teaching in Higher Education 5.4 (2000): 435-445

Personality

Clayson, Dennis E., and Mary Jane Sheffet. "Personality and the student evaluation of teaching." Journal of Marketing Education 28.2 (2006): 149-160

Controversial course material

Nast, Heidi J. "'Sex','Race' and Multiculturalism: Critical consumption and the politics of course evaluations." Journal of Geography in Higher Education 23.1 (1999): 102-115

Gateway courses

Silbey, David. "Courses That Get Bad Evaluations." TheChronicle.com. 15 Jan. 2009. Web. 22 Sept. 2015.