I’ve written before on student evaluations, with a bit of a giggle about some of the answers I get.
As I have said, my worry has always been that the kinds of questions asked are too vague, and the responses don’t really reflect, well…anything:
There is also a “multiple choice” part of the questionnaire that students must fill out. It contains a really silly question which says something like “I felt part of a group of students and staff who were committed to learning”. What does that mean exactly? There’s too many variables. I suspect that the real question is whether the teacher actually cares about his or her subject and the students. One always enjoys a subject more if the teacher is enthusiastic. I like the written comments on the questionnaires the best. They are the bits I take to heart, both positive and negative.
It seems I’m not alone in questioning these evaluations.
Via Cearta.ie, I came across some posts on Concurring Opinions on the topic. First, Sarah Lawsky poses a series of questions about about student evaluations and their uses, audience and so forth. Secondly, Dan Hoffman finds an interesting paper which suggests (as he summarises it):
That is, well-regarded, young, inexperienced teachers provide better short-term results (hypothesis: enthusiasm), but over the longer term unpopular, older, experienced teachers add the most value.
And “good teaching” in introductory couses does not necessarily lead to good results in later follow-on courses. He wonders how much teaching surveys should matter in the hiring process of universities as a result of this information.
I think I tend to agree with Eoin’s point at Cearta.ie that some lecturers can be both experienced and enthusiastic. Certainly, I had a number of lecturers who fitted that description when I was at university. Experienced does not necessarily mean unpopular or boring.
I do wonder about the way in which questions are framed and the audience for whom they are intended. The generic questionnaires at my university are supposed to be “one size fits all” and consequently, some of the questions don’t really fit law subjects very well.
For example, there is a question saying, “The teacher made appropriate use of technology”, but in some subjects in which I have taught, a considered decision has been made not to use any technology whatsoever. Students always get confused about what to answer in response to that question. Surely an appropriate use of technology may be “none at all”? But obviously if one is doing an IT degree, one would expect a very high use of technology.
As I’ve said in my previous post, I always get some feedback saying that students love it when I don’t have Powerpoint, and an equal amount saying that they hate the lack of Powerpoint. My personal opinion is that Powerpoint is a tool of the devil. Its usefulness is restricted to typing case citations and headings onto slides so that I don’t have to write them on the board.
21 Comments
I’ve always wondered about the student who called me ’straightforward and frank’, which struck me as ever so slightly backhanded. To this day I can’t imagine which particular aspect of my teaching she was referring to.
Powerpoint is a tool of the devil when used as THE POINT of the presentation, rather than as a tool of the presentation.
Worst ever training session - trainer printed the slides, had the slides up on the board, and proceeded to read them.
I wanted to say “you know what? Everyone in this room had a degree to get this job. There is a CHANCE we might be able to read the slides without help”
PC, I choose to think “straightforward and frank” is a compliment. Personally, it’s exactly what I like - no waffle, I know where I stand.
But I know what you mean…
MsLaurie, I COULD NOT AGREE MORE! See my previous posts about Powerpoint here and here. The first post outlines a very similar situation to the one you describe (seems to be a problem with corporate presentations), but the second outlines how Powerpoint can actually lead to very bad accidents…
On Powerpoint, don’t get me started on the two or three day conference where 90% of the presentations consisted of people reading word for word from Powerpoint. Two or three days going on a life sentence.
On uni teaching, the Oxbridge tutorial system is hardly sustainable en masse but I despair of genuine education when there is virtually zero one on one time with staff so you can get something approaching a meeting of minds.
At least where I work, the questions are basically ripped off the CEQ and then run for individual subjects — so the individual questions shouldn’t matter too much if you know what the factors are (very few people do, and my university doesn’t provide this info, probably because the people in charge don’t know what a factor is, let alone how to check them).
I did worry about this, and got the data for all the subjects. As it turns out, the questions don’t work for individual subjects in the way they work for the CEQ (in fact, they’re basically a waste of time in the form which they are using). I pointed this out to our statistically rather ignorant staff who run this (who can’t even calculate SDs properly…), and they simply didn’t want to know and told us we should instead focus on the qualitative stuff (these guys obviously didn’t do or slept through stats 101)
This of course is great for big subjects, since you basically get every reponse for anything possible, which gives good leverage to make changes that other people might not want, so you can use arguments like this: “we had to change it based on qualitative feedback!”
PC - You’ve always been straightforward and frank with me. My psychiatrist is delighted. It pads her account.
.
Say it loud. I’m blunt and I’m proud.
L’eagle -
.
I’ve designed a fair few of these questionaire thingies for various people who like throwing their money away on bollocks. There’s a bureaucrat’s tool’ It enables them to make some report saying 54% of such and such think so and so - it’s nonsense. The data retrieved is subject to the variation that comes of highly subjective assessments of the very vague.
.
And like most such control freakery they lead straight to the Rule of the Sycophant.
Unfortunately explaining why that’s bad pencil pushers (of the private or public sectors) is almost impossible for the same reason as selling word peace to an arms dealer is fruitless.
That’s why I’m in to decentralized curricula. This will allow good teachers to throw off the pencil-necked geeks and simply be creative. There must be teachers out there who know that the 19th century style of education we employ isn’t working anymore, that realize that the various shreds of po-mo jargonia isn’t the answer and that have ideas.
Let ‘em rip.
No rip on powerpoint, no matter how tangential, is complete without once again referring readers to this gem from Peter Norvig, a professor of considerable reknown in the better circles of computer science; current Director of Research for some small Californian outfit.
And even more tangentially, and from the same august AI specialist and spare-time comedian, there is Einstein’s Performance Review.
In Trinity, the Law School stopped using the centralised assessment forms, and generated our own instead. They are marginally preferable, but the real gain is the space for additional comments.
As for powerpoint, consider A slide into mediocrity by Tara Brabazon in the Times Higher.
Eoin.
“In Trinity, the Law School stopped using the centralised assessment forms, and generated our own instead”
We used to do this where I worked — which was far better in case you happened to know about psychometrics (you could find out which of tutors etc. was doing well). Unfortunately, it doesn’t work well with Adrien’s points, which is that the data is really for strange management delusions, not to improve the quality of things in a meaningful way (in fact it’s often negative, since people often go for good ratings over useful learning)
Love all the spoof powerpoints - I’m so glad that I’m not the only one who hates ‘em.
Yes, I have always suspected the true purpose of student evaluations is to keep the pencil pushers happy and give them a bunch of statistics to work with…
As a MAS, I was appalled when we were begged to fill in these forms. Most of the undergrads are ignorant, conservative, bovine fools. The idea they have the capacity to evaluate academics is a sad reflection on the low-wattage drivel they must be being shovelled. This whole university as group therapy gig is a scandal.
Give me a tyrannical teacher who disdains undergraduates, but has a total clue than one of these fembots who is “one of the girls.”
Of course the ultimate student evaluation would be the quality of responses in final examinations and major essays.
JG at #14, the problem with exams and essays as an evaluation is that you can be the best teacher in the world, but if students don’t study or even bother to turn up to class then you shouldn’t be judged by the quality of their exams or essays.
As I said in my post on challenging marks, one of the difficulties in being a teacher is that there is only so much you can do. You can lead a horse to water, but you can’t make it drink, no matter how much you might coax it or bully it. The final mark is up to the student. I think some of the outrage at KRudd’s idea of assessing high school teachers by students’ marks reflects this.
I am interested in how students perceive my classes, and if there are any valid criticisms I take them on board, but personally, I don’t believe my job is to spoon-feed students. As I say, in the end, it’s up to them how much work they want to put into it.
I’ve learnt to stop worrying and love Powerpoints as a law teacher in Property Law and Contract Law. The thing is the slides must be complementary to what your presenting, not trying to replicate it. So if you’re orally presenting lots of words/text (which is what law teachers usually do; I’ve yet to see interpretive dance take hold as a method of explicating legal doctrine) then the thing to avoid is slides which also present lots of text. Students can’t effectviely take in text aurally and visually.
So I use the slides for more visual representations. If I amteaching in a small group with a whiteboard, I usually fill the whiteboard with diagrams and timelines and flow charts etc, so when lecturing I use the powerpoints for this purpose, as well as for setting out the structure of a lecture or an area of law. Visual aids are a great help in law: stylised maps when explaining an easements problem; timelines for adverse possession; diagrams to illustrate how a joint tenancy can be severed by mutual transfers etc.
As for relying on what students think of a course, I’m reminded of the following exchange from David Mamet’s Tin Men, a great 1980s movie, where Danny De Vito was going to buy a lovely new cadillac:
SALESMAN
Now, how much are you willing to
pay?
BB
There ya go… there ya go…
you’re doing it… you’re doing
one of those hustle numbers.
SALESMAN
I’m just trying to get an idea
how much you’re willing to pay.
BB
Four dollars… I want to pay four
dollars a month.
SALESMAN
That’s not an honest answer.
BB
What do ya want to hear? That I’d
love to pay three hundred and fifty
a month… is that what you want to
hear? Tell me how much you want me
to pay and I’ll tell you how much
I’ll pay, but don’t do a hustle on
me… I don’t like that. How much
do I want to pay? I’d like to pay
nothing!
In short, ask students what type of assessment they’d like, and there’s the danger they’ll say ‘Nothing! We’d like no assessment!’. How would you like to be taught? ‘Spoon fed, without any requirement to do any self-directed learning’ etc etc
Anthony, I love diagrams! If I do have Powerpoint slides, that’s the other thing I put on them (particularly useful for sorting out equitable proprietary priority disputes). I also love flowcharts, but I tend to put them on handouts so that people can write their own notes on them.
The few times I’ve used powerpoint, it’s been for case citations and visuals, nothing more. As a general rule, however, I provide a detailed course guide at the beginning of the term - that seems to do the trick. Very few people are really gifted with powerpoint. I’ve had one teacher who was brilliant (don’t mind mentioning his name - Prof Vernon Nase in tort at UQ. I think he’s at UWA now) and I’ve seen one comedian use them to devastating effect (Laurence Clark).
I’m a big fan of detailed course guides myself. Although it is depressing when students don’t read them. Every semester, I get at least one student who complains that they don’t know what cases will be in the next lesson, and I say, “READ THE COURSE GUIDE!”
JG - Give me a tyrannical teacher who disdains undergraduates,
.
Yep. A really snooty patronizing superior absolute bastard is one of the prime motivators for excellence.