KLAUS-DIETRICH KRÖNCKE, PhD, is professor at the Institute of Biochemistry and Molecular Biology I at the Medical Faculty of the Heinrich-Heine-University of Düsseldorf. The work described here is part of his master thesis of the german postgraduate course ‘Master of Medical Education’.
Medical Department of the Heinrich-Heine-University of Düsseldorf, Düsseldorf, Germany
| ABSTRACT | Section: |  |  |
|---|
Background: Computer-based learning is a tool that, when designed appropriately, can be used to effectively meet worthy educational goals.
Aims: This study aimed to compare acceptance and effectiveness of a tutored computer-based practical course (CPC) with a traditional biochemical laboratory experiment in pre-clinical medical education.
Method: Under quasi-randomised conditions two cohorts of second-year medical students performed either a CPC or a laboratory experiment. Students’ perceptions were obtained by assessment questionnaires, knowledge retention was investigated by post test.
Results: The students evaluated the CPC highly significantly better than the laboratory experiment. Students performing the CPC demonstrated a statistically significant greater knowledge retention compared to students who performed the laboratory experiment.
Conclusions: These findings show that learning objectives concerning basic biochemical knowledge can be acquired in a CPC and that medical students accept such a CPC as a substitute for a real laboratory experiment.
| Introduction | Section: | |  |
|---|
Electronic learning tools offer an exciting potential for improving student learning, either as an aid to or as a replacement for traditional teaching methods. Particularly in medical education there is a rapid expansion of e-learning tools (Ellaway & Masters 2008; Masters & Ellaway 2008), and computer-based applications are now part of modern medical curricula all over the world (Ruiz et al. 2006). Most of these e-learning applications are web-based learning or teaching programs (e.g. Colsman et al. 2006 and websites cited therein; see also the webside ‘The Multimedia Educational Resource for Learning and Online Teaching’: http://www.merlot.org). To date implementation or evaluation of only a few e-learning practicals for medical students have been published (Dwyer et al. 1997; Hughes 2001; Levine 2002; Hoa et al. 2006; Dantas & Kemm 2008).
We recently have implemented a tutor-supported computer-based practical biochemistry course ‘Polymerase Chain Reaction (PCR)’ in our curriculum. Second-year medical students learn essential features of the PCR with the aid of embedded text and audiovisual animations, respectively, search for specific PCR-relevant data in PubMed, and finally solve medical-related PCR tasks. The students have evaluated this computer-based practical course (CPC) very positively (Kröncke & Becher 2008). We therefore decided to devise further tutor-supported CPCs.
In one of our compulsory biochemical laboratory experiments pre-clinical medical students measure the oxygen consumption of isolated rat liver mitochondria under various conditions. Learning objective is to get a deeper insight into the biochemical mechanism of respiration and energy generation in the human body. Here the implementation of a tutored CPC containing an interactive computer software virtually simulating the oxygen consumption of mitochondria is described. This new devised learning unit also contains textual information, embedded images and visual animations about ‘energy generation in humans and poisoning with cyanide, carbon monoxide, and uncouplers of the respiratory chain’ for self-studies as well as formative self-tests. This study aimed to investigate student acceptance of the CPC and to investigate whether students performing CPC demonstrate greater knowledge retention compared to students performing the ‘wet’ laboratory experiment.
|
| Methods | Section: | | |
|---|
The study was designed as an experimental field study (Figure 1). Ethical approval to the study was given by the ethics committee of the Heinrich-Heine-University of Düsseldorf.
|
The grouping of all second-year medical students into 21 groups (maximum: 16 students) in the winter semester 2007/08 was adopted from the previous semester. Neither students nor lecturers of the Institute of Biochemistry and Molecular Biology I had any influence on that.
In Düsseldorf medical students perform a biochemical practical training in the third semester. A written description of the laboratory experiments including workflows and theoretical background information is always available for the students in the intranet of the university. In 30-minute introductory seminars prior to each laboratory experiment lecturers discuss the topic with the student groups. Student knowledge about the theoretical background of the respective experiment is audit tested by the lecturers, and ill preparation leads to exclusion from the respective practical course. Thus, all students have a similar theoretical knowledge prior to performing the laboratory experiments.
Groups with odd numbers carried out the traditional laboratory experiment and groups with even numbers the CPC, but this was not announced. To avoid any lecturer bias the same 2 lecturers A and B contemporaneously performed the introductory seminars in turns with respect to groups with odd or even numbers using the same PowerPoint slides. To evaluate the students’ motivation to perform the subsequent learning unit the students were asked to respond to an anonymous questionnaire containing 5 rating-type questions with a 6-point Likert scale (1 = ‘strongly agree’ or ‘very high’ to 6 = ‘strongly disagree’ or ‘very low’). After completion of the motivation questionnaire the groups were told to perform the laboratory experiment or the CPC.
Students in groups of 4 measured the oxygen consumption of isolated rat liver mitochondria under various conditions, e.g. after addition of dinitrophenole (DNP; uncoupler of the respiratory chain) and cyanide, respectively. Results were recorded and printed, and the students calculated the respiratory control index Q as well as the P:O ratio. Two educational assistants supervised the students. No formative feedback test was performed. Including the calculations, the laboratory course lasted about 2 hours.
Students in twos in their own time and at their own pace measured with an interactive computer software the simulated oxygen consumption of mitochondria under various conditions. Resulting graphs were printed and the students calculated Q and P:O similar as the students performing the laboratory experiment. Including the calculations the students needed about 1 h for the virtual experiment. Three educational assistants supervised and helped the students with the calculations. After completing the calculations the student teams left the room for a 10–15 minute break. The students in twos then performed self-studies on the generation of ATP in humans under aerobic, anaerobic and fasting conditions as well as on symptoms and therapies of poisonings with cyanide, carbon monoxide, and DNP. The self-study consisted of a learning path containing text (e.g. a real-life intoxication case with DNP) with keywords linked to images and flash animations. Finally the students in twos performed 2 formative self-tests on the computer. After filling out 15 cloze deletion test questions they automatically got their own answers presented together with a predefined ‘optimal’ answer. They then answered 2 × 10 multiple-choice (MC) test questions by selecting one correct out of 5 items. At the end the results were automatically shown to them. During the self-study and the self-tests only 1 educational assistant was present to answer questions. The whole computer-based learning unit including the break lasted about 2–2.5 hours.
Immediately after the laboratory courses or the CPCs the students were asked to respond to an anonymous contentment questionnaire containing 12 rating-type questions with a 6-point Likert scale (1 = ‘strongly agree’ or ‘very good/school grade 1’ to 6 = ‘strongly disagree’ or ‘very bad/school grade 6’) and 3 free text questions (‘what was very good?’, ‘what was very bad?’, ‘what should be improved?’). The questionnaires were electronically scanned and statistically analysed using SPSS. Free text comments were transcribed and thematically analysed to determine the number of students who made similar comments.
The compulsory biochemical MC test written about 2 weeks after the teaching sessions was used to verify the randomisation of the study participants. The test contained 40 questions but no specific questions from the self-studies or self-tests of the CPC.
At the beginning of seminars in biochemistry or physiology about 3 months after the teaching sessions the students were unannounced asked to voluntarily fill out a knowledge questionnaire. Students were assured that the test results were treated confidentially. To improve motivation, 3 book coupons (100.- EURO each) were advertised to be drawn by lot under all students participating under their name. To make it more difficult to copy answers from a neighbour, 3 versions of the questionnaire were distributed which differed only in the sequences of the 5 free text and the 5 cloze deletion test questions. The students needed about 15–20 min for the test with no imposed time limit. The questionnaires were numbered and copied twice without the students’ names. Two lecturers of the Institute of Biochemistry and Molecular Biology I, which otherwise were not involved in the study, independently assessed the copied and blinded questionnaires. With an assessment template they both scored the answers to the free text questions (0–4/5 points) and to the cloze deletion test questions (0–1 points). Assignment of 0.5 points was permissible. In case of significant score differences (>1.5 points for a free text answer or >0.5 points for a cloze test answer), the 2 referees were asked to discuss their different scores with each other and to eventually modify them. Subsequently the scores were averaged and unblinded.
Analysis was performed using Students’ t test (two tailed for independent samples) with p < 0.05 considered as significant.
| Results | Section: | | |
|---|
In the winter semester 2007/08, within 2 weeks 162 students (64.8% female, 35.2% male) in 11 groups went through the computer-based practical e-learning course (E) and 133 students (60.9% female, 39.1% male) in 10 groups through the laboratory course (L). To verify the motivation of the students to do these learning units, an anonymous motivation evaluation was carried out directly after the introductory seminars and prior to the announcement, which group will perform E and which will perform L. The questionnaires were completed by 162 E-students (100%) and by 132 L-students (99.3%). With the exception of question 1, no statistically significant differences were found between the motivations and expectations of the students who subsequently performed E or L (Table 1
|
Directly after completion of the laboratory courses or the CPCs the students were asked to fill out an anonymous contentment questionnaire. Of the E-students, 156 (96.3%) completed the questionnaire, of the L-students only 117 students (88.0%), because unfortunately in one L-group the evaluation questionnaires were not distributed. Figure 2 shows the ratings on selected questions from the students’ survey. All questions shown were highly significantly better rated by the E-students showing that the students liked the CPC better.
|
In the E-groups 81 students (51.9%) and in the L-groups 53 students (45.3%) answered at least one free text question. In both E- and L-groups the introductory seminar (8.3% of all E-students vs. 14.5% of all L-students) and the helpful educational assistants (6.4% vs. 9.4%) were most frequently mentioned positively. In the E-groups, 10 students (6.4%) appreciated that the experiment was performed via computer, 9 students (5.8%) liked the cloze and the MC self-tests, respectively, and 6 students (3.8%) appreciated, that the experiment could be repeated by pressing a ‘reset’ button. Only 2 students (1.3%) mentioned that they did not like the working on a computer. All other clustered commentaries were given by less than 3.8% of the students. Particularly noticeable, no clustered improvement proposals were given by more than 4 students.
About 2 weeks after the teaching sessions, 139 E-students (85.5% of those who had performed the CPC) and 109 L-students (82.0% of those who had performed the laboratory experiment), respectively, participated in the compulsory biochemical MC test. Mean test scores for the E-students (63.6% female, 36.4 male) were 45.6 ± 14.9 and for the L-students (60.7% female, 39.3% male) 48.6 ± 13.5 (maximum = 80). The difference of 3 scores was found to be not statistically significant (p = 0.099). This shows that the quasi-randomisation of the students into E- or L-groups had been successful.
About 3 months after the practical courses all students who participated in seminars of biochemistry or physiology were unannounced asked to voluntarily fill out a knowledge questionnaire. Of the 292 seminar participants 246 (84.2%) completed the knowledge questionnaires. Marked with names were 214 questionnaires (71.2%), while 32 (13.0%) were completed anonymously.
Students were divided into E (e-learning group; n = 87; 63.2% female, 36.8% male), L (laboratory experiment group; n = 87; 59.8% female, 40.2% male), and C (control group = students which neither had participated in E or L; n = 40; 57.5% female, 42.5% male). E-students were 22.8 ± 2.4, L-students 23.7 ± 4.1, and C-students 24.8 ± 5.4 years old with only the difference between E and C being statistically significant (p = 0.032). In Figure 3 total test scores (maximum = 28) as well as the sum of the scores of the free text and of the cloze test questions are shown including minimum and maximum scores achieved. Students of the E-groups scored better than students of the L- and of the C-groups. The differences were small but statistically significant, with the exception of the difference between E and L concerning the cloze test results.
|
| Discussion | Section: | | |
|---|
Very few medical students will ever need biochemical laboratory skills, and undergraduate biochemical laboratory experiments are primarily performed to illustrate and to reinforce theoretical biochemical knowledge. A tutor-supported lesson combining a virtual and interactive simulation of a biochemical experiment with text, images, animations, and a learner control via questions and immediate feedback was devised and implemented. Data handling and interpretation was similar as in the parallel performed laboratory experiment. It was evaluated whether the students accepted this computer-based learning unit as a substitute for a real laboratory experiment and whether graduation from this computer-based learning resulted in greater knowledge retention compared to graduation from the ‘wet’ laboratory practical.
The study participants were not randomised due to the rigid schedule of lessons in the third semester, but the results of the compulsory biochemical MC test showed that the study was performed under quasi-randomised conditions. The students evaluated the CPC significantly better than the laboratory experiment, although a positive amplification effect due to cross-communication cannot be excluded. In an unannounced knowledge test concerning the topic of the teaching sessions 3 months later the E-students scored better than the L-students, although the 2 cohorts had a similar theoretical knowledge before the learning units and despite the fact that the students had learned for the compulsory biochemistry test which was performed between the teaching sessions and the post test. This may explain why the scoring differences were small, although they were statistically significant.
It is thought that active exchanges with others enhances student performance and satisfaction (e.g. Michael 2006; Braeckman et al. 2008). With regard to activities, engagement, guidance, feedback, and repeated and individualised practice the implemented CPC offers a range of advantages. Students doing the CPC benefit from being more active, as they perform the virtual experiment in twos compared to 4 students performing the laboratory experiment, and as they additionally perform self-studies and self-tests about the medical relevance and the theoretical background of the biochemical experiment. In addition, the E-students spent more time in the e-learning than in the laboratory course (2–2.5 h versus 2 h), and their attention was directed towards the specific learning objectives without distraction of laboratory manipulations required in the real experiment. Moreover, the more positive educational climate in a team of 2 compared to a team of 4, and the more flexible options of computers compared to the quite rigid scheduled environment in a laboratory may have also positively influenced the learning outcome. Furthermore we noticed that the E-students discussed a lot during the virtual experiment, the self-studies and the self-tests, either in their teams but also between the teams. This is in line with findings that students like to do e-learning with a partner (Rawson & Quinlan 2002). In the laboratory, where 4 students work together, individual students can hide and not reveal their weakness. In contrast, in the CPC the students are under higher social pressure to collaborate and to understand the content of the learning unit. It is therefore not surprising that in the unannounced knowledge test the E-students scored significantly better than the L- or the C-students. This indicates that the computer-based learning unit is more effective than the laboratory experiment with respect to knowledge retention, although duration of the teaching sessions as well as supervision expenses all in all are comparable.
With 8.0 ± 2.8 and 6.7 ± 2.8 (maximum score: 28), respectively, the knowledge test results appear to be not very satisfactory. So the question is allowed whether we really can talk about knowledge retention. However, the students were totally unprepared to perform the knowledge test, and it was a formative test with a low motivation to score well. In addition, free text questions (maximum score: 23) are always rather challenging for students. The results of the cloze deletion test questions (maximum score: 5) were much better, because these kind of questions are quite easy to answer. Furthermore, the E-students scored nearly twice as high as the C-students (4.1 ± 2.8 scores) which had performed neither E nor L and thus served as a control group. This indicates that really knowledge retention and not just recall was measured.
A recent meta-analysis showed that internet-based learning is associated with large positive effects compared with no intervention, but that effects compared with non-internet instructional methods are heterogeneous and generally small, thus suggesting effectiveness similar to traditional methods (Cook et al. 2008). David Cook in this journal stated that e-learning research to date has done little to inform educational practice and that further no-intervention-controlled studies or comparisons with traditional instructional methods are not needed (Cook 2009). This may indeed be true for web-based, but not for other computer-based learning units, e.g. tutor-supported computer-based practicals. To date only few studies have been published which describe the implementation of e-learning practicals in medical education, and only some of these papers contain measurements of knowledge retention compared to traditional laboratory practicals (Hughes 2001; Sancho 2006; Dantas & Kemm 2008). However, summative tests in form of MC test questions were used, and no active knowledge in form of unannounced free text questions was inspected. Also differences in the study design and the quality of the e-learning courses make it difficult to compare the here described results with results from the above cited studies.
| Conclusions | Section: | | |
|---|
This study shows that a computer-based practical course can provide a learning tool that is more effective than a ‘wet’ laboratory experiment for developing knowledge retention and understanding that does not involve practical details of the experimental methods used, and that medical students accept such a CPC as a substitute for a real laboratory experiment.
| Acknowledgements | Section: | | |
|---|
I thank the students who responded to the questionnaires, Thomas Becher for installing the software and helping to analyse data, Dr. Judith de Bruin for generating and analysing the questionnaires, Luka Peters for creating the flash animations used in the CPC, Dr. Gunter Wagner and Dr. Sibylle Soboll for rating the knowledge test, Dr. Wim Gijselaers, Dr. Martin Fischer, Dr. Stefan Herzig, and Dr. Christoph Stosch for their advices with the study design, and Dr. Wilhelm Stahl for supporting this study in word and deed. I also thank the Educational Dean of the Medical Faculty of the Heinrich-Heine-University of Düsseldorf for funding the 3 book coupons.
Declaration of interest: The author reports no conflict of interest. The author alone is responsible for the content and writing of the paper.
| References | Section: | | |
|---|
- Braeckman LA, Fieuw A, van Bogaert HJ. A web- and case-based learning program for postgraduate students in occupational medicine. Int J Occup Env Health 2008; 14: 51–56
- Colsman A, Sticherling M, Stopel C, Emmrich F. Computer-assisted learning in medicine. How to create a novel software for immunology. Arch Dermatol Res 2006; 298: 1–6
- Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: A meta-analysis. J Am Med Assoc 2008; 300: 1181–1196
- Cook DA. The failure of e-learning research to inform educational practice, and what we can do about it. Med Teach 2009; 31: 158–162
- Dantas AM, Kemm RE. A blended approach to active learning in a physiology laboratory-based subject facilitated by an e-learning component. Adv Physiol Edu 2008; 32: 65–75
- Dwyer TM, Fleming J, Randall JE, Coleman TG. Teaching physiology and the world wide web: Electrochemistry and electrophysiology on the internet. Adv Physiol Educ 1997; 18: S2–13
- Ellaway R, Masters K. AMEE Guide 32: e-Learning in medical education. Part 1: Learning, teaching and assessment. Med Teach 2008; 30: 455–473
- Hoa D, Micheau A, Gahide G. Creating an interactive web-based e-learning course: A practical introduction for radiologists. RadioGraphics 2006; 26: e25
- Hughes IE. Do computer simulations of laboratory practicals meet learning needs?. Trends Pharmacol Sci 2001; 22: 71–74
- Kröncke KD, Becher T. Implementation and evaluation of a tutor-supported computer-based practical biochemistry course ‘polymerase chain reaction (PCR)’ in preclinical education. GMS Zeitschrift für Medizinische Ausbildung 2008; 25: Doc90
- Levine AE, 2002. Evaluation of world wide web-based lessons for a first year dental biochemistry course. Medical Education Online 7:13 (available from http://www.med-ed-online.org)
- Masters K, Ellaway R. AMEE Guide 32: e-Learning in medical education. Part 2: Technology, management and design. Med Teach 2008; 30: 474–489
- Michael J. Where's the evidence that active learning works?. Adv Physiol Educ 2006; 30: 159–167
- Qayumi AK, Kurihara Y, Imai M, Pachev G, Seo H, Hoshino Y, Cheifetz R, Matsuura K, Momoi M, Saleem M, et al. Comparison of computer-assisted instruction (CAI) versus traditional textbook methods for training in abdominal examination (Japanese experience). Med Educ 2004; 38: 1080–1088
- Rawson RE, Quinlan KM. Evaluation of a computer-based approach to teaching acid/base physiology. Adv Physiol Educ 2002; 26: 85–97
- Ruiz JG, Mintzer MJ, Leipzig RM. The impact of e-learning in medical education. Acad Med 2006; 81: 207–212
- Sancho P, Corral R, Rivas T, Gonzalez MJ, Chordi A, Tejedor C. A blended learning experience for teaching microbiology. Am J Pharm Educ 2006; 70, Article 120
- Smolle J, Staber R, Neges H, Reibnegger G, Kerl H. Computer-based training in dermatooncology – a preliminary report comparing electronic learning programs with face-to-face teaching. J der Deutschen Dermatologischen Gesellschaft 2005; 3: 883–888
