Skepticlawyer

Dave sent me this interesting post from Prawfsblog about law students grading their own answers. The author of the post was speaking to another law lecturer, who proposed the following idea for an exam:

…[H]e might give three questions on an exam and ask students to answer all of them. But he also asks them to choose one of the questions they think they did the best on so that will be worth twice as much. E.g., three questions worth 25 points each, but the one question chosen by the student to double up on will in fact be worth 50 points, while the others remain at 25 points each, for a total of 100.

update: I neglected to mention that the student also gets the choice to diversify, such that each of the n questions count for 1/n of the exam points, so students have the choice to avoid making a decision wrt their confidence levels.

I thought this was pretty interesting and hadn’t heard of its use in the annals of law teaching. As far as I can tell, the skill underlying the choice of which question to select is figuring out one’s own confidence levels in an answer.

I’m a little bit dubious about the use of such a method, partly because of my own experience as an undergraduate. Sometimes I was really confident about a subject, and I turned out to do not so well. Other times, I thought I’d barely passed and I turned out to have blitzed the exam. I think there was only one time where I made an accurate prediction of whether I’d done well or not.

There are a number of reasons for this.

First, I always did better in difficult exams. Difficult exams gave me a chance to show off my knowledge, whereas it’s much harder to differentiate one’s self if the exam is easy. But if the exam was difficult, I tended to think I’d done worse than I had.

Secondly, I tended to work harder for subjects where I thought I was going to bomb (Property, Corporations Law, Taxation and other “commercial subjects”)…paradoxically, the upshot of this was that I did really well at those subjects, and not so well in the ones where I just cruised on through.

Thirdly, I had no idea about what made an answer good or not. It is only now, after a few years of marking other people’s exams, that I have an idea of what examiners are looking for. I try to give my students an idea of this (well structured, mentions all relevant points, refers to relevant case law/legislation, emphasises the right issues, has an argument or at least a coherent thread if it is an essay question). It’s the emphasis which is the killer. Sometimes one makes a misjudgment and spends half a question on something irrelevant, or of marginal relevance. This means that the mark will naturally be lower because there’s less time to discuss the really important stuff. Or you stuff up the timing and end up with not enough time to deal with the really important issue. But you might not realise this at the time.

I always try to give students an opportunity to get feedback about their exams. I have discovered that often they are quite surprised about which questions they did badly on, and which questions they did well on. Sometimes, confidence is a bad thing, because it means that the student has blithely skipped over various issues, whereas if they are more cautious about a question, they are more likely to go through the issue thoroughly. Once I show the student why they received the mark, they are quite willing to accept it.

So, I think that students will often misjudge which questions they do well. There’s also a link back to my post on imposter syndrome, which tends to suggest that some people have great difficulty in judging whether they are really good at academic pursuits. Therefore, a marking policy such as the one above could be disasterous.

Furthermore, I do not think there is any real parallel between practice and exams. When one is in practice in a certain area, you live that area of the law every day, and have no doubt about your competence. And if you need to go and confirm your knowledge or check up on something, you can do that before you get back to the client. Exams call for a very different skill-set.

Dave did mention the idea that this would be good for a quiz show, however, and I think he is right. It would be interesting to see how well people do at judging their competency.