Yesterday, there was an interesting discussion at a faculty brown bag lunch I attended. The topic of discussion was how law professors grade exams. Assuming that you give an essay exam with more than one question, you have two options: (1) You can read student A’s exam in its entirety, assign a score, move on to student B’s exam, do the same, move on to student C’s exam, etc.; or (2) You can read student A’s answer to question 1, assign a score, read student B’s answer to question 1, do the same, move on to student C’s answer to question 1, etc. I call option 1 “Exam by Exam” grading and option 2 “Question by Question” grading (or exam hopping).
It seems to me that both approaches have pros and cons. With the exam by exam approach, you get a better sense of each student’s entire exam. Was this an exam that was extremely well written and organized throughout? Did it seem like this student was doing gangbusters on the first several questions and simply ran out of time on the last question(s)? These questions can easily be answered when a professor takes it one exam at a time. The main con seems to me to be selective information processing, the unconscious inclination to search out and recall information that tends to confirm one’s existing belief and devalue disconfirming evidence. When I used to grade exams using this approach, I found myself fighting this bias. Basically, when a student had a great first answer or answers, I tended to be more forgiving of later errors, figuring that these must have been silly mistakes because obviously the student knew the material well. And if a student started poorly, my assumption was that later great answers were more the product of luck than skill. Now, I think that I was able to combat this bias for the most part, and I think that most professors can as well, but it is definitely something to take into account.
With the question by question approach, the professor has the advantage of comparing apples to apples. The professor can read each student’s answer to question 1 back-to-back-to-back and more easily apply the same standard to each individual question. The main con is that a professor applying this approach sacrifices some of the big picture analysis mentioned above (Personally, I follow the question by question approach, and, after scoring the last question on each exam, read the prior questions again so that I can (a) correct any prior mistakes, and (b) see the exam as a whole).
So, which approach do you use and why? You can respond by answering the following poll and/or leaving a comment.
-Colin Miller
Posted by Evidence ProfBlogger on September 16, 2010 at 09:26 AM
Comments
I guess, but wouldn’t it make sense to give or subtract the “style” points based on each question too?
Posted by: anon | Sep 16, 2010 3:53:04 PM
Bruce, I am mainly thinking of the professor who grades an exam using a grading key but then likes to add or subtract “style points” at the end. If an exam has 4 questions, and the professor exam hops, when he comes back to question 4 on Student A’s exam, he might give it a “20.” If the professor gave the student a “20” on the other 3 questions, the final score would be an 80. Unless that professor marks down more than “20” after each question, he can only rely upon the answer to question 4 to determine the quality of the writing in the exam apart from substance. If the professor, however, read the exam in its entirety, it would be easier to judge the quality of the writing on the whole exam. That’s the reason I exam hop but then, in the example given above, would re-read the answers to questions 1-3 again to get a better sense of the overall exam.
Posted by: Colin Miller | Sep 16, 2010 2:12:07 PM
I’m not sure I understand the “pros” of the exam-by-exam approach, since they sound exactly like the negatives: that it allows positive or negative information about exam performance to bleed over from one question to the next. It seems to me that the whole point of having separate questions is to isolate performance on each question. (That’s certainly what I tell my students in warning them about time management.) Is there some other positive that doesn’t involve cross-contamination?
Posted by: Bruce Boyden | Sep 16, 2010 11:36:51 AM