[Use of elaborate feedback and an audience-response-system in dental education]
Alexander Rahman 1Silke Jacker-Guhr 1
Ingmar Staufenbiel 1
Karen Meyer 1
Michaela Zupanic 2
Merle Hahnemann 1
Anne-Katrin Lührs 1
Jörg Eberhard 3
1 Medizinische Hochschule Hannover (MHH), Klinik für Zahnerhaltung, Parodontologie und Präventive Zahnheilkunde, Hannover, Deutschland
2 Universität Witten/Herdecke, Fakultät für Gesundheit, Department für Medizin, Institut für Gesundheitssystemforschung, Witten, Deutschland
3 Medizinische Hochschule Hannover (MHH), Klinik für Zahnärztliche Prothetik und Biomedizinische Werkstoffkunde, Hannover, Deutschland
Abstract
Introduction: The aim of the study was to examine the effect of an elaborate feedback and an audience response system (ARS) on learning success.
Methods: Students of the 1
Result: The results of the final exams showed no significant difference between the study and the control group regarding the learning success.
Conclusion: Although no significant effect on learning success was found, the ARS creates a more interactive, positive learning environment.
Keywords
feedback, audience-response-system, learning environment
Introduction
Interactivity and tests as measurement of learning success are critical aspects of high-grade teaching and of great importance not only to students, but also to teachers. Posing in-class test questions at the end of curricular units presents an effective tool not only for increasing student interactivity and attentiveness, but also for confirming successful learning [9]. Preszler et al. documented that in practice often only a small percentage of students participate in dialogues with the teacher during class [24]. Despite this, student feedback is an essential factor for the students themselves, the teachers, and overall for the quality of teaching [11], [14], [15]. Through effective feedback, students and teachers can recognize if their learning behaviors and teaching methods are successful or not and where improvements in performance are needed [4]. For about 20 years now, it has been known that feedback in medical education has a positive effect on the learning process and is a main factor for improved performance in exams [5], [13]. Van Houten and van Houten [29] demonstrated in their study that in the conventional lecture setting immediate feedback during class is superior to delayed response by corrected papers. If feedback is given following an incorrect answer, care should be taken that any resulting discussion does not address why a mistake was made, but rather focus on the relevant information that leads to the correct answer, so-called “targeted feedback” [18]. Musch [20] also points out that in controlled, empirical studies, an elaborate feedback leads to better performance than basic information about the accuracy of the given answer. By detailed discussion of the issues surrounding the questions asked, the student receives renewed information about the correct response which promotes the learning process. Thus, the elaborative, informative feedback is primarily suitable following incorrect responses [5], [16], [18].
The advantage of anonymous computerized quizzes lies in the opportunity to “learn without embarrassment”. Even weaker students can evaluate themselves and realize what they have learned in the preceding class session or where learning deficiencies exist. The audience response system (ARS) allows to ask multiple choice questions directly to the auditorium using a projector (see Figure 1 [Fig. 1]). The projection of the results indicates for the instructor not only which aspects need to be addressed more detailed, but also shows weaknesses in the content, along with any deficiencies concerning prior knowledge of the students. Rubio et al. reported [25], that a study group which used ARS during lectures on Radiology scored substantially higher in a test three months afterwards than the control group which did not participate in interactive lectures with ARS. In dental medicine, students who very frequently experienced ARS in lectures, performed significantly better than students who used ARS less often or not at all [8], [22], [23], [24], [25]. Based on the fact that there are still too few empirical findings on this topic, the following research question was addressed.
Issue under investigation
Does a course which uses an audience response system (ARS) with elaborate feedback result in a higher level of education among students than a lecture without the elaborate feedback?
Methods
The following study was approved by the ethical commission of Hannover Medical School (study no.: 1006-2011). The students (n=74) during their first clinical semester (summer semester 2011) were randomly assigned to a study or a control group. The randomization was done by the Institute of Biometry at Hannover Medical School considering the factors age, sex, and performance during the preliminary medical examination (“Physikum”). In terms of the content, the lectures for the study and control group had the same structure and used the same presentation slides. In the course on conservative dentistry, both groups received the same lecture content on various sub-disciplines (cariology, periodontology, endodontology, pediatric dentistry) from the same instructor. During each lecture, students answered five multiple-choice questions (see Figure 1 [Fig. 1]) on the stated learning objectives using an ARS. This number of questions was chosen because the recommended number of questions within a 50-minute class session is between two to four [3] and two to five [2]. To assure quality, all multiple-choice questions were subjected to a pre- and post review process [26]. Department employees conducted a test for comprehension, logic, and hints about the correct answer. Parallel to the lecture, five times per week in the afternoon, the practical preclinical course in conservative dentistry was held, during which students acquired the necessary practical skills in the field of conservative dentistry for the subsequent clinical courses (see Table 1 [Tab. 1]).
In a seminar, the participating instructors attended an introduction to the use of feedback and feedback rules by the principal investigator. In this study, elaborate, informative feedback that is given directly in spoken form was applied. After announcing the correct answer, the students in the study group received a detailed feedback from the instructors, regardless of whether the majority had responded correctly or incorrectly. The elaborate feedback covered a brief summary of the reasons why a certain answer was correct and, if needed, why the other possible answers were not applicable. For the control group, only the correct answer was displayed and the lecture was then resumed without discussing the question or answer in further depth. For both groups, the particular lecture sessions lasted 45 minutes, with or without feedback. At the completion of the lecture series, an evaluation survey with 18 questions based on a 6-point Likert scale and a computerized multiple-choice final exam (20 items) was performed. Data were analyzed in cooperation with the Institute for Biometry using SAS 9.2. The results of the final exam were evaluated using a primary and secondary analysis. For the primary analysis, a covariance analysis (ANCOVA) was calculated to compare the results in both groups taking into account the stratification defined by performance on the preliminary medical examination. To confirm the primary analysis, a two-sample T-test was used as a secondary analysis in order to compare the performance results in both groups.
Results
The evaluation of the exams showed no significant difference between the results of the study and the control group. The 37 participants in the study group earned an average of 14.49 points, and the 36 participants in the control group scored only slightly lower with an average of 14.42 points (-0.07 points). The results of the final exam showed no significant difference between the study and control group (p=0,362) even with the application of a covariance analysis and taking the co-variable “score on the preliminary medical examination” into consideration. Likewise, when verifying the primary analysis with the T-test, no significant difference (p=0,873) was detected between the two groups.
73 of 74 student surveys could be analyzed. Eighty-five percent of the students rated the use of ARS to be useful. Ninety percent of the surveyed students agreed on the statement that a class with ARS is more motivating (see Figure 2 [Fig. 2]). 81% of the students indicated that they participated more actively in the lecture due to the ARS. The evaluation showed that the students rated the use of ARS very positive and would like the system to be integrated into other lectures.
Discussion
The aim of the present study was to bring more interaction into the student lectures by using a specifically applied elaborate feedback and an audience response system and by facilitating successful studying among students. The results of the computerized final exam demonstrated no significant difference between the study and the control group. Thus in this study, the elaborate feedback is not a factor that, together with an ARS, positively influenced learning.
A criterion which could have influenced these results was the fact that the practical preclinial course in conservative dentistry took place parallel to the lectures. It was taught by the same instructors who also gave the lectures. During the practical course, all students had the opportunity to talk to the teachers and ask questions, inorder to clarify misunderstandings, and learn more about a specific topic. In addition, there were multiple practical demonstrations for both groups on performance of the various techniques, during which, at least from time to time, the theoretical context of the practical work was addressed. As a result, the students in the control group could have received a feedback outside of the lectures. This might have been especially important when they addressed the topics more intensively while preparing for the final exam. If they felt the need for clarification, they were able to ask questions about the requested topic during the practical course. Not allowing the control group to ask questions during the lecture or the practical course would have been one possibility for limiting the influence of external factors, but this is ethically not justifiable in practice.
Another factor which could have influenced the students is the challenge of answering multiple-choice questions during the lectures. The participants in both groups automatically were dealing with the topics in more depth than they would have done during the more traditional lecture setting. As a result of these multiple-choice exercises, the effect of the elaborate feedback may have been obscured, as already described by Gauci et al. [12].
For many students, multiple-choice questions pose a challenge if they are not trained in processing them. Since in each of the ten lectures, five multiple-choice questions were asked, a certain amount of practice may have been developed over the time and made it easier for the students to respond to the multiple-choice questions during the final exam. Stoddard and Piquette assumed in their study that the intensified focus on the curricular content makes additional feedback superfluous [27].
A further limitation concerning this study could have been the low number of test subjects (73 students). For reasons directly related to capacity, the participants of our study were limited to the students in their first clinical semester. A multi-center study with a much higher number of test subjects (>11.000) might demonstrate significant findings, but poses questions regarding the practicability.
Nevertheless, an ARS creates a positive learning atmosphere. This is confirmed not only by the evaluations and observations of other studies [1], [7], [10], [17], [19], [25], [28], but also by the results of the present investigation. The majority of students in this study affirmed that a course with the ARS approach is more motivating than a purely traditional lecture setting because it leads to active participation and better imparting of the curricular content.
Also in previous studies, it was not the ARS itself which was considered of being responsible for the increased levels of attention and participation, in some cases even for better exam results, but rather the interactivity that results from the application of an ARS [12], [21], [23].
Conclusion
Though no significant difference in the final exams of the study and the control group were found, the positive evaluation results of this study show that an audience response system helps to create a positive and interactive learning atmosphere between instructors and students. Therefore, the use of the ARS can be strongly recommended during lectures.
Acknowledgements
Our special thanks are due to the Deutsche Gesellschaft für Mund-, Kiefer und Zahnheilkunde (DGZMK) for the financial support of this study.
Competing interests
The authors declare that they have no competing interests.
References
[1] Alexander CJ, Crescini WM, Juskewitch JE, Lachman N, Pawlina W. Assessing the integration of audience response system technology in teaching of anatomical sciences. Anat Sci Educ. 2009;2(4):160-166. DOI: 10.1002/ase.99[2] Allan D, Tanner K. Infusing active learning into the large-enrollment biology class: seven strategies, from the simple to complex. Cell Biol Edu. 2005;4(4):262-268. DOI: 10.1187/cbe.05-08-0113
[3] Beatty I. Transforming students learning with classroom communication system. EDUCASE Center Appl Res (ECAR) Res Bull. 2004;3:1-13.
[4] Bienstock JL, Katz NT, Cox SM, Hueppchen N, Erickson S, Puscheck EE; Association of Professors of Gynecology and Obstetrics Undergraduate Medical Education Committee. To the point: medical education reviews--providing feedback. Am J Obstet Gynecol. 2007;196(6):508-513. DOI: 10.1016/j.ajog.2006.08.021
[5] Boehler ML, Rogers DA, Schwind CJ, Mayforth R, Quin J, Williams RG, et al. An investigation of medical student reactions to feedback: a randomised controlled trial. Med Educ. 2006;40(8):746-749. DOI: 10.1111/j.1365-2929.2006.02503.x
[6] Cain J, Black EP, Rohr J. An audience response system strategy to improve student motivation, attention, and feedback. Am J Pharm Educ. 2009;73(2):21. DOI: 10.5688/aj730221
[7] Cain J, Robinson E. A primer on audience response systems: current applications and future considerations. Am J Pharm Educ. 2008;72(4):77. DOI: 10.5688/aj720477
[8] Elashvili A, Denehy GE, Dawson DV, Cunningham MA. Evaluation of an audience response system in a preclinical operative dentistry course. J Dent Edu.2008;72(11):1296-1303.
[9] Fabry G. Unterrichtmethoden: Welches Mittel zu welchem Zweck? In: Fabry G. Medizindidaktik. Bern: Verlag Huber; 2008. S.119-129.
[10] Fies C, Marshall J. Classroom response systems: A review of the literature. J Sci Educ Technol. 2006;1:101-109. DOI: 10.1007/s10956-006-0360-1
[11] Fischer PM, Mandel H. Improvement of acquisition of knowledge by information feedback. In: Mandl H, Lesgoold A (Hrsg). Learning issues for intelligent tutoring systems. New York: Springer; 1988. S. 187-241. DOI: 10.1007/978-1-4684-6350-7_9
[12] Gauci SA, Dantas AM, Williams DA, Kemm RE. Promoting student-centered active learning in lectures with a personal response system. Adv Physiol Educ. 2009;33(1):60-71. DOI: 10.1152/advan.00109.2007
[13] Hattie J, Timperley H. The power of feedback. Review. Educ Res. 2007;1:81-112. DOI: 10.3102/003465430298487
[14] Huang C. Changing learning with new interactive and media-rich instruction environments: virtual labs case study report. Comput Med Imaging Graph. 2003;27(2-3):157-164. DOI: 10.1016/S0895-6111(02)00089-7
[15] Kamin C, Deterding R, Lowery M. Students perceptions of a virtual PBL experience. Acad Med. 2002;77(11):1161-1162. DOI: 10.1097/00001888-200211000-00028
[16] Kulhavy RW, Stock WA. Feedback in written instruction: The place of response certitude. Educ Psychol Rev. 1989;4:279-308. DOI: 10.1007/BF01320096
[17] MacGeorge EL, Homan SR, Dunning Jr. JB, Elmore D, Bodie GD, Evans E, Khichadia S, Lichti SM, Feng B, Geddes B. Student evaluation of audience response technology in large lecture classes. Educ Technol Res Develop. 2008;56(2):125-145. DOI: 10.1007/s11423-007-9053-6
[18] McKendree J. Effective feedback content for tutoring complex skills. Human-Comp Interact. 1990;5(4):381-413. DOI: 10.1207/s15327051hci0504_2
[19] Miller RG, Ashar BH, Getz KJ. Evaluation of an audience response system for the continuing education of health professionals. J Contin Educ Health Prof. 2003;Spring(2):109-115.
[20] Musch J. The Design of Feedback in Computer Based Trainings: Models and Findings. Z Padagog Psychol. 1999;13(3):148-160. DOI: 10.1024//1010-0652.13.3.148
[21] Patry M. Clickers in Large Classes: From Student Perceptions Towards an Understanding of Best Practices. Int J Scholar Teach Learn. 2009;3(2).
[22] Pileggi R, O'Neil PN. Team-based learning using an audience response system: an innovative method of teaching diagnosis to undergraduate dental students. J Dent Educ. 2008;72(10):1182-1188.
[23] Pradhan A, Sparano D, Ananth CV. The influence of an audience response system on knowledge retention: an application to resident education. Am J Obstet Gynecol. 2005;(5):1827-1830. DOI: 10.1016/j.ajog.2005.07.075
[24] Preszler RW, Dawe A, Shuster CB, Shuster M. Assessment of the effects of student response systems on student learning and attitudes over a broad range of biology courses. CBE Life Sci Educ. 2007;(1):29-41. DOI: 10.1187/cbe.06-09-0190
[25] Rubio EI, Bassignani MJ, White MA, Brant WE. Effect of an audience response system on resident learning and retention of lecture material. AJR Am J Roentgenol. 2008;(6):319-322. DOI: 10.2214/AJR.07.3038
[26] Smolle J. Klinische MC-Fragen rasch und einfach erstellen: ein Praxisleitfaden für Lehrende. 2., [veränd.] Aufl. Berlin, New York, NY: De Gruyter; 2010. S. I-IV
[27] Stoddard HA, Piquette CA. A controlled study of improvements in student exam performance with the use of an audience response system during medical school lectures. Acad Med. 2010;85(10 Suppl):37-40. DOI: 10.1097/ACM.0b013e3181ed3b40
[28] Van Dijk LA, van den Ber GC, van Keulen H. Interactive Lectures in Engineering Education. Eur J Eng Educ. 2001;26(1):15-28. DOI: 10.1080/03043790123124
[29] Van Houten R, Van Houten J. The performance feedback system in the special education classroom: An analysis of public posting and peer comments. Behav Ther. 1977;8(3):366-376. DOI: 10.1016/S0005-7894(77)80071-3
