Student evaluations again

By Legal Eagle

I’ve written before on student evaluations, with a bit of a giggle about some of the answers I get.

As I have said, my worry has always been that the kinds of questions asked are too vague, and the responses don’t really reflect, well…anything:

There is also a “multiple choice” part of the questionnaire that students must fill out. It contains a really silly question which says something like “I felt part of a group of students and staff who were committed to learning”. What does that mean exactly? There’s too many variables. I suspect that the real question is whether the teacher actually cares about his or her subject and the students. One always enjoys a subject more if the teacher is enthusiastic. I like the written comments on the questionnaires the best. They are the bits I take to heart, both positive and negative.

It seems I’m not alone in questioning these evaluations.

Via Cearta.ie, I came across some posts on Concurring Opinions on the topic. First, Sarah Lawsky poses a series of questions about about student evaluations and their uses, audience and so forth. Secondly, Dan Hoffman finds an interesting paper which suggests (as he summarises it):

That is, well-regarded, young, inexperienced teachers provide better short-term results (hypothesis: enthusiasm), but over the longer term unpopular, older, experienced teachers add the most value.

And “good teaching” in introductory couses does not necessarily lead to good results in later follow-on courses. He wonders how much teaching surveys should matter in the hiring process of universities as a result of this information.

I think I tend to agree with Eoin’s point at Cearta.ie that some lecturers can be both experienced and enthusiastic. Certainly, I had a number of lecturers who fitted that description when I was at university. Experienced does not necessarily mean unpopular or boring.

I do wonder about the way in which questions are framed and the audience for whom they are intended. The generic questionnaires at my university are supposed to be “one size fits all” and consequently, some of the questions don’t really fit law subjects very well.

For example, there is a question saying, “The teacher made appropriate use of technology”, but in some subjects in which I have taught, a considered decision has been made not to use any technology whatsoever. Students always get confused about what to answer in response to that question. Surely an appropriate use of technology may be “none at all”? But obviously if one is doing an IT degree, one would expect a very high use of technology.

As I’ve said in my previous post, I always get some feedback saying that students love it when I don’t have Powerpoint, and an equal amount saying that they hate the lack of Powerpoint. My personal opinion is that Powerpoint is a tool of the devil. Its usefulness is restricted to typing case citations and headings onto slides so that I don’t have to write them on the board.

21 Comments

  1. Posted September 17, 2008 at 1:04 pm | Permalink

    I’ve always wondered about the student who called me ‘straightforward and frank’, which struck me as ever so slightly backhanded. To this day I can’t imagine which particular aspect of my teaching she was referring to.

  2. MsLaurie
    Posted September 17, 2008 at 3:00 pm | Permalink

    Powerpoint is a tool of the devil when used as THE POINT of the presentation, rather than as a tool of the presentation.

    Worst ever training session – trainer printed the slides, had the slides up on the board, and proceeded to read them.

    I wanted to say “you know what? Everyone in this room had a degree to get this job. There is a CHANCE we might be able to read the slides without help”

  3. Posted September 17, 2008 at 3:39 pm | Permalink

    On Powerpoint, don’t get me started on the two or three day conference where 90% of the presentations consisted of people reading word for word from Powerpoint. Two or three days going on a life sentence.

    On uni teaching, the Oxbridge tutorial system is hardly sustainable en masse but I despair of genuine education when there is virtually zero one on one time with staff so you can get something approaching a meeting of minds.

  4. conrad
    Posted September 17, 2008 at 5:17 pm | Permalink

    At least where I work, the questions are basically ripped off the CEQ and then run for individual subjects — so the individual questions shouldn’t matter too much if you know what the factors are (very few people do, and my university doesn’t provide this info, probably because the people in charge don’t know what a factor is, let alone how to check them).

    I did worry about this, and got the data for all the subjects. As it turns out, the questions don’t work for individual subjects in the way they work for the CEQ (in fact, they’re basically a waste of time in the form which they are using). I pointed this out to our statistically rather ignorant staff who run this (who can’t even calculate SDs properly…), and they simply didn’t want to know and told us we should instead focus on the qualitative stuff (these guys obviously didn’t do or slept through stats 101)

    This of course is great for big subjects, since you basically get every reponse for anything possible, which gives good leverage to make changes that other people might not want, so you can use arguments like this: “we had to change it based on qualitative feedback!” 🙂

  5. Posted September 17, 2008 at 6:23 pm | Permalink

    PC – You’ve always been straightforward and frank with me. My psychiatrist is delighted. It pads her account. 🙂 .

    Say it loud. I’m blunt and I’m proud.

  6. Posted September 17, 2008 at 6:34 pm | Permalink

    L’eagle –
    .
    I’ve designed a fair few of these questionaire thingies for various people who like throwing their money away on bollocks. There’s a bureaucrat’s tool’ It enables them to make some report saying 54% of such and such think so and so – it’s nonsense. The data retrieved is subject to the variation that comes of highly subjective assessments of the very vague.
    .
    And like most such control freakery they lead straight to the Rule of the Sycophant.

    Unfortunately explaining why that’s bad pencil pushers (of the private or public sectors) is almost impossible for the same reason as selling word peace to an arms dealer is fruitless.

    That’s why I’m in to decentralized curricula. This will allow good teachers to throw off the pencil-necked geeks and simply be creative. There must be teachers out there who know that the 19th century style of education we employ isn’t working anymore, that realize that the various shreds of po-mo jargonia isn’t the answer and that have ideas.

    Let ’em rip.

  7. Jacques Chester
    Posted September 17, 2008 at 9:44 pm | Permalink

    No rip on powerpoint, no matter how tangential, is complete without once again referring readers to this gem from Peter Norvig, a professor of considerable reknown in the better circles of computer science; current Director of Research for some small Californian outfit.

  8. Jacques Chester
    Posted September 17, 2008 at 9:49 pm | Permalink

    And even more tangentially, and from the same august AI specialist and spare-time comedian, there is Einstein’s Performance Review.

  9. Posted September 18, 2008 at 12:26 am | Permalink

    In Trinity, the Law School stopped using the centralised assessment forms, and generated our own instead. They are marginally preferable, but the real gain is the space for additional comments.

    As for powerpoint, consider A slide into mediocrity by Tara Brabazon in the Times Higher.

    Eoin.

  10. conrad
    Posted September 18, 2008 at 5:41 am | Permalink

    “In Trinity, the Law School stopped using the centralised assessment forms, and generated our own instead”

    We used to do this where I worked — which was far better in case you happened to know about psychometrics (you could find out which of tutors etc. was doing well). Unfortunately, it doesn’t work well with Adrien’s points, which is that the data is really for strange management delusions, not to improve the quality of things in a meaningful way (in fact it’s often negative, since people often go for good ratings over useful learning)

  11. John Greenfield
    Posted September 18, 2008 at 8:52 am | Permalink

    As a MAS, I was appalled when we were begged to fill in these forms. Most of the undergrads are ignorant, conservative, bovine fools. The idea they have the capacity to evaluate academics is a sad reflection on the low-wattage drivel they must be being shovelled. This whole university as group therapy gig is a scandal.

    Give me a tyrannical teacher who disdains undergraduates, but has a total clue than one of these fembots who is “one of the girls.”

  12. John Greenfield
    Posted September 18, 2008 at 8:53 am | Permalink

    Of course the ultimate student evaluation would be the quality of responses in final examinations and major essays.

  13. Anthony
    Posted September 18, 2008 at 11:59 am | Permalink

    I’ve learnt to stop worrying and love Powerpoints as a law teacher in Property Law and Contract Law. The thing is the slides must be complementary to what your presenting, not trying to replicate it. So if you’re orally presenting lots of words/text (which is what law teachers usually do; I’ve yet to see interpretive dance take hold as a method of explicating legal doctrine) then the thing to avoid is slides which also present lots of text. Students can’t effectviely take in text aurally and visually.

    So I use the slides for more visual representations. If I amteaching in a small group with a whiteboard, I usually fill the whiteboard with diagrams and timelines and flow charts etc, so when lecturing I use the powerpoints for this purpose, as well as for setting out the structure of a lecture or an area of law. Visual aids are a great help in law: stylised maps when explaining an easements problem; timelines for adverse possession; diagrams to illustrate how a joint tenancy can be severed by mutual transfers etc.

  14. Anthony
    Posted September 18, 2008 at 12:07 pm | Permalink

    As for relying on what students think of a course, I’m reminded of the following exchange from David Mamet’s Tin Men, a great 1980s movie, where Danny De Vito was going to buy a lovely new cadillac:
    SALESMAN
    Now, how much are you willing to
    pay?
    BB
    There ya go… there ya go…
    you’re doing it… you’re doing
    one of those hustle numbers.

    SALESMAN
    I’m just trying to get an idea
    how much you’re willing to pay.
    BB
    Four dollars… I want to pay four
    dollars a month.
    SALESMAN
    That’s not an honest answer.
    BB
    What do ya want to hear? That I’d
    love to pay three hundred and fifty
    a month… is that what you want to
    hear? Tell me how much you want me
    to pay and I’ll tell you how much
    I’ll pay, but don’t do a hustle on
    me… I don’t like that. How much
    do I want to pay? I’d like to pay
    nothing!

    In short, ask students what type of assessment they’d like, and there’s the danger they’ll say ‘Nothing! We’d like no assessment!’. How would you like to be taught? ‘Spoon fed, without any requirement to do any self-directed learning’ etc etc

  15. Posted September 18, 2008 at 5:41 pm | Permalink

    The few times I’ve used powerpoint, it’s been for case citations and visuals, nothing more. As a general rule, however, I provide a detailed course guide at the beginning of the term – that seems to do the trick. Very few people are really gifted with powerpoint. I’ve had one teacher who was brilliant (don’t mind mentioning his name – Prof Vernon Nase in tort at UQ. I think he’s at UWA now) and I’ve seen one comedian use them to devastating effect (Laurence Clark).

  16. Posted September 18, 2008 at 6:33 pm | Permalink

    JG – Give me a tyrannical teacher who disdains undergraduates,
    .
    Yep. A really snooty patronizing superior absolute bastard is one of the prime motivators for excellence. 🙂

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*