How to read student evaluations constructively
Plus admiration for Netflix’s “The Chair” and a pre-Y2K Sandra Oh film shout out.
Hello and welcome to Academia Made Easier. I am so glad that you are here.
Like many people in academia, I was disappointed by the announcement this January that Netflix is not moving forward with a second season of The Chair. I greatly enjoyed watching it. One particularly memorable storyline was that of Professor Joan Hambling and her student evaluations. After admitting to her department chair that she had not read any student evaluations since the late 1980s, Hambling reads a handful of the evaluations and then decides to take action – by burning them.
Image of The Chair’s Professor Joan Hambling burning her student evaluations, via Buzzfeed
I understand her feelings. Here is an example of a student evaluation comment I received that I would like to burn: “She is not very friendly before class starts but I guess she isn’t paid to be.” Ouch. I received this comment for a class I taught as a grad student and my reaction is “Oh, fuck off” whenever I think about it.
And yes, decades later, I still think about it.
I have a lot of thoughts about student evaluations. I do think it is important for students to be able to provide feedback about their instruction. Evaluations also provide an opportunity for feedback to help instructors understand how students perceive the course and their teaching style, which can be used to inform future teaching. I have personally been able to use evaluations this way and I know from experience they can be a useful information source.
However, it is hard to overlook the many problems associated with student evaluations. The response rates for student evaluations are often pretty terrible, making their representativeness highly suspect. Evaluations can correlate with students’ expected grades, creating problematic incentives for grade inflation. And there is considerable evidence that student evaluations are a dumpster fire of gender, racial, and other systemic biases. For individual faculty, particularly those in less-represented groups within their discipline or department, student evaluations can be personally hurtful, containing abusive comments.
While it may be tempting to channel Professor Joan Hambling and simply burn the evaluations, most instructors lack the privilege to do so. Universities often use student evaluations to inform career decisions, such as including evaluations in tenure and promotion files and/or considering evaluations when selecting contract instructors. For instructors who have any degree of contingency in their role, the career relevance of student evaluations makes them risky to ignore.
So, how should you approach reading your own student evaluations? Well, that’s what today’s small thing to try immediately is about.
One Small Thing to Try Immediately: Decide Your Personal Student Evaluation Strategy
For a long time, my approach to reading student evaluations was as follows:
Before reading the evaluations, assume at least one super shitty comment. Feel dread.
Read evaluations as soon as they are available to me.
Gloss over all of the positive comments and hone in on any negative comments. If there are no negative comments, find the more tepid positive comments (e.g., “I actually enjoyed the class”) and read some negativity into those (“actually”?!?!?!).
Eat chocolate.
This is not a strategy I recommend and thankfully it is not one I currently use. Instead, I recommend (and use) the approach below.
1. Decide beforehand why you are reading the student evaluations. “Start with why” is generally a good approach, so begin here. What are your reasons for reading your student evaluations? Here are some possible responses to prompt your thinking:
I am required to include a summary of student evaluations in my tenure/promotion file. I need to read the student evaluations to do so.
I am curious to know how students (or at least those who completed the evaluation) experienced my course. Reading the student evaluations will serve as a source of information.
I want to improve the student experience of my course. The student evaluations may provide me with ideas for how to do so.
I am concerned that the student evaluations will include negative comments that will affect my ability to secure future teaching contracts. I need to read the student evaluations to determine if I need to speak with the department chair to discuss and address any such feedback.
I hope to be considered for a future teaching award that includes student evaluations as a metric. I need to read the student evaluations to determine if I am a viable award nominee.
Some students took the time to complete the evaluation. I feel this obligates me to read their feedback.
I am a curious person and want to know the evaluation results, period. If I don’t read them, I will think about them incessantly.
Be really clear with yourself about your “why” before you move to the next step. If you decide you have no reason to read the evaluations and that is truly an option for you, feel free to exercise that privilege.
2. Given your “why”, consider your best approach to reading the evaluations. You can choose to read the evaluations immediately, while the course is fresh in your mind, or after a set period of time, such as when you are preparing to teach the course again. You can choose to read the evaluations yourself or you can have a trusted person read them and summarize them for you.
By having both a reason and a plan, you can increase the utility and the safety of the experience.
3. When reading the evaluations, be mindful of the inherent biases in the process. As noted above, student evaluations are prone to all sorts of biases. It is possible that the only students who completed the evaluations are those who either loved you or hated you. It is possible that your evaluations reflect students’ unconscious biases with respect to race, gender, age, or other factors. Positive evaluations might in part reflect the fact that you are white and/or male and/or give higher-than-average grades. Or, positive evaluations may accurately indicate that you delivered a great course. Negative evaluations might in part reflect the fact that you are female and/or not white and/or teaching an unpopular required course. Or, negative evaluations may be a legitimate signal that the course wasn’t very good. Who knows?
By keeping the inherent biases in mind, you can aim to neutralize both the glow and the sting of evaluations and read them with an eye to fulfilling your “why” for reading them in the first place. For example, if your goal is to improve the student experience in your class, you can take the comments and consider them in a more neutral manner. A positive comment can be translated into “consider doing more of X” and a negative comment can become “consider if there are more effective ways to approach Y”.
To return to my own example, at the time I received that comment about not being friendly before class, I was an introverted graduate student with very little teaching experience. The comment stung because I was frankly too scared to chit-chat with students before class. Over time, I reframed this comment as feedback that students often desire connection and relationship with their instructors. This framing was helpful for me to see the time before class as an opportunity for connection with students and was actually helpful to me. (But still not terribly kind!)
4. Consider participating in university discussions about the appropriate use of student evaluations. As universities strive to move forward with equity, diversity, and inclusion, there is an opportunity for evidence-informed collegial discussions about the appropriate design and use of student evaluations. You may wish to be part of such discussions on your own campus.
If you are, I encourage you to start such discussions by considering why student evaluations are used in the first place. What problem are they intended to solve? Is it to provide students with an opportunity to share feedback with instructors and/or department chairs on the student experience? To provide instructors with information about student perceptions of their courses? To provide department chairs and/or tenure and promotion committees with information to protect “teaching quality” in programs?
Knowing what teaching evaluations are intended to accomplish can help focus discussions by pointing to follow-up questions. If the goal is to allow for student feedback, is it appropriate to use them for other purposes in light of the known biases? If the goal is ongoing teaching improvement, do the questions used in the evaluation support this? If the goal is assessing teaching quality, how can student evaluations be combined with or replaced by other forms of instructional assessment? In light of the issues of bias, I feel these questions and conversations are important.
What are your own thoughts about and experiences with student evaluations? Please use the comment button at the bottom of this email/post to let me know!
Chipping Away: What I Have Been Up To
A quick update on some of my own activities since my last newsletter, since I have your attention:
I wrote something! In my April Skills Agenda column, “Teaching students how to work with and understand the limits of data”, I talk about the idea of data literacy and the need for students to learn how to use quantitative and qualitative data responsibly. I argue that this instruction needs to go beyond research methods classes and suggest ways for instructors to build it into their classes. Give it a read and let me know if you agree!
I appeared on a podcast! I once again had the pleasure of speaking with Heather Ross on her Better Me podcast. In our latest discussion, we explore modern “busy culture” (spoiler: neither of us is a fan of it!). Please download it to your podcast queue so I can join you on your next walk/drive/cleaning binge.
I rearranged my home office! My new setup allows Storm to supervise my work more effectively. She is not always impressed with me:
Image: my cat Storm, sitting on top of my bookshelf and looking down at me, unimpressed with my efforts.
Until next time…
As we approach the end of April, I am feeling re-invigorated. My book project is moving into its next stage (more on this soon), spring is raising my spirits, and I am celebrating my birthday this week, which means birthday cake is in my future! I hope you have things you are looking forward to as well. Please feel free to let me know how you are doing in the comments. I always love hearing from you!
Stay well, my colleagues.
P.S. Back to The Chair: did you know that Sandra Oh is Canadian? The first thing I remember seeing her in was the pre-Y2K Canadian film Last Night. She won the Genie Award for Best Actress for the role, demonstrating her long-standing awesomeness.
Want to help support my chocolate habit? You are very sweet. Buy me a coffee is a site that allows readers to show their appreciation for the unpaid labour of writers, artists, and other creatives. Please check it out!
I love SophieGrace clothes: perfect for work, comfortable machine-washable fabrics, and a Canadian company. If they suit your own style, use the affiliate code AcademiaMadeEasier to get 15% off.
I haven’t read mine, because we’ve had so many strikes in the UK and students are really grumpy. A few very strong students have said positive things about the course and I asked them for a paragraph with feedback that I can include in probation packets.
I reviewed a male colleague’s probation packet and his scores were a full point above mine, but he only teaches MA students, versus me, with a mandatory, first year, first term course…with incredibly low response rates.
On evaluations... Like Joshua did, I would sometimes ask them specific questions. For example--if they felt inclined--to tell me which book/reading I really should keep and which one they would dump. Ditto graded assignments. Also, I'd ask if they expected something to be covered that wasn't. As he said, it is possible that it is incorrect, but I was out of the room when they did it in class. l retired about the time all written contact with students was pushed online.