I have read the recent contributions of my colleagues Michelle Maroto and Carolyn Sale on the debate around the use of teaching evaluations at the University with considerable interest and appreciation. They take up a question that has generated wide interest and commentary in higher education: why is it that university administrators are so attached to measures of teaching quality that are demonstrably misleading? It is an important puzzle. Luckily, I’m an anthropologist, and I’m here to help.
This is a problem with which anthropologists have long grappled in their research. It is, essentially, the problem of magic. Why have so many people in so many places and at so many times seemed not to have realized that magical methods are not effective? Why don’t they apprehend that astrology is not predictive, that oracles are unreliable, that chicken entrails are pretty much undecipherable? All of these might have appeared to offer promising leads in a few trials, but with repeated use shouldn’t people have *noticed* that they don’t really produce the results claimed for them?
The answer is that these practices do work, and beautifully: just not on the tasks to which they are ostensibly put. The difficulties they tackle and the solutions they produce are political, not technical or epistemological. They absolutely do produce effects in the world. People who turn to such means again and again are not fools about causality. Not in the least!
Any anthropologist worth his or her salt has seen this happen in the field (or at least read about it in graduate school): social alliances are made and broken via the deployment and interpretation of magical phenomena. What does it mean that the crops are failing, people are getting sick, and the weather is behaving oddly? Whose fault is it and what auguries can different factions invoke to build their coalitions? Magical undertakings are modes of persuasion and — to use an idiom apropos to the case at hand — team-building exercises.
My non-anthropological colleagues are understandably perplexed and frustrated that reiterating rigorous empirical results about bias in teaching evaluations leaves the University of Alberta administration totally unmoved. Do faculty skeptics really have to explain about the fallibility of chicken entrails *again*? Shouldn’t it be obvious? Administrators say they want good information about teaching quality. USRIs cannot be relied upon to provide it.
Dear colleagues: look behind another door. Yes, you are right, teaching evaluations don’t do what it says on the box, so to speak. So what is it that they do accomplish that makes them so valuable to administration, so resilient in the face of failure, such perennial objects of ritual obeisance? They are instruments of political alliance-making. That’s the voodoo that they do so well.
Witness the last occasion on which they were challenged in a ceremonial clash from which they emerged, as they always must, victorious. What tournament of symbols was enacted there? Caped and costumed as the heel, the proffies (BOOOOOOOOO!): a role taken at the most recent GFC, for the extra delectation of the crowd, by a lady proffie (double BOOOOOOOOO!). Lady profs are the Ric Flairs of the academic game, everybody knows that. Come on.
But wait, guys, wait! It’s gonna be okay!
HULK UNIVERSITY ADMIN HOGAN!!!!! THAT DUDE IS AWESOME!
My metaphors have crept a bit (blame Roland Barthes!), but look: the utility of teaching evaluations is in figuring faculty as the heel for the purposes of shared governance. That’s why it doesn’t matter very much if teaching evaluations are any good at evaluating teaching. That’s not what they are for. The contemporary administratively-dominated university hasn’t come about because university administrators are fools who don’t understand basic statistics. It’s come about because they are, among other things, masterful politicians.
Conflict over USRIs creates a dramatic narrative that can be framed as one in which professors wish to escape accountability. Students, who know what it is to be judged by professors, are concerned to have means to judge professors in their turn. Administration appears to step in as a powerful ally, assisting students in holding the responsible line.
Students are quite right to want the quality of the teaching they receive rigorously evaluated and they are quite right to insist their input and oversight be prioritized in that evaluation. Faculty were themselves students and know the various horrors of incompetent, tyrannical, or unjust teaching. We are on students’ side about this. Students and faculty working together could design brilliant systems of evaluation that would be superior in fairness and ameliorative actionability to the ones that are utilized at present.
But by splitting students from faculty on this highly visible issue — an issue that is understandably a top priority for students — university administrations win tremendous powers. They are thereby able to frame faculty as the “heels” for the purposes of many other governance conflicts to which administration, students, and faculty are party. By showily seeming to ally themselves with students on the “goodness” (fuzzily defined) of USRIs, they distract from the many, many other teaching issues that affect students and about which students are never consulted at all.
A critically important example, on which I will end, is the constantly increased reliance upon contract academic staff in undergraduate teaching. This has dramatic consequences for students, and yet is an issue upon which they are hardly invited to think, let alone vote. Students often do not know — and are not encouraged to understand — which of their professors are tenured and tenurable and which are contingent. The quality of the classroom performance could be stellar or abysmal from either, but when students need things like recommendation letters they may not realize that a letter from a full-time faculty member may be weighted differently by evaluation committees than one from contract faculty, or that if they need a letter after the lapse of a few years a temporary academic might be impossible to track down where a permanent one is usually attached to the same old familiar department (or, in case they have moved on to another tenured position at another university, easily trackable by that department). Students, who in my experience are earnestly concerned about fairness, may not realize that a harsh evaluation of an instructor can lead to a collegial discussion about teaching methods for a full-time faculty member but to the loss of livelihood for a contingent one. By the same token, effusive praise can lead to a raise for a full-time faculty member while producing nothing at all for a contingent instructor who is the first to be cut when cuts come.
Faculty would love to talk about these things with students. When it comes to university governance, our interests are much more often allied than opposed. But although faculty collectively spend vastly more time with students than do administrators, we rarely use the main context in which we come to know and care about students to reach out to them about how the university is run. It would be awkward within the teaching relationship, which we certainly take to be as precious as students do. Where we do meet and where we could and should build alliances, in forums for shared governance, a bit of black magic is performed to divide us. It’s an old trick, and an effective one.