A comparison of scoring algorithms for multiple answer MC-exams

 Martin Fischer ¹
Daniel Bauer
Veronika Kopp

¹ Medizinische Klinik Innenstadt der Universit?t M?nchen, Schwerpunkt Medizindidaktik, M?nchen, Deutschland

---

Workshop

Objectives: To compare different scoring algorithms usually employed in determining students scores in multiple correct answer multiple-choice (MC) exams regarding performance, reliability, selectivity, and item difficulty.

Methods: Data from 420 3rd year medical students' end of term exam in internal medicine in February 2005 at Munich University were analysed (30 MC questions; up to 15 possible answers, up to 6 correct answers per question, at least as many distractors as true answers).

Scoring Algorithms: Each question scored a maximum of one point. No negative scores were applied. We compared:

- "Dichotomous" (D): One point if all true and no wrong answers were chosen.

- "Partial 1" (P1): One point for 100% true answers; 0.5 points for 50% or more true answers; zero points for less than 50% true answers.

- "Partial 2" (P2): A fraction of one point depending on the total number of possible answers was given for each correct decision (picking a right or ignoring a wrong answer); for each wrong decision one such fraction was subtracted.

Results: The P1-algorithm showed best results concerning item selectivity, item difficulties, and internal consistency (Cronbach's alpha), respectively.

Conclusions: The P1-algorithm seems to be the preferable method for the scoring of multiple answer MC-exams.