An Online, Problem-Oriented Course of Health Information Systems for ...

An Online, Problem-Oriented Course of Health Information Systems for Undergraduate Health Students: design, implementation and assessment

By Paulo Marcondes Carvalho Jr. M.D., University of Taubaté, Brazil M.S., State University of Campinas, Brazil Ph.D., State University of Campinas, Brazil

Project

Submitted as partial fulfillment of the requirements for the degree of Master in Health Professions Education in the Department of Medical Education College of Medicine University of Illinois at Chicago, 2002 Chicago, Illinois


          

   

           

Paulo Marcondes Carvalho Junior

Advisor: Alan Schwartz, PhD Committee Members: Arthur Elstein, PhD Ara Tekian, PhD First Advisor (licensed): Annete Yonke, PhD

Marília, Brazil 2002

iii


       

                 

iv

ACKNOWLEDGEMENT

I would like to thank the W.K. Kellogg Foundation for sponsoring me in this Master of Health Professions Education Program. I appreciate the generosity of the Marília Medical School to include me in this project. I thank so much my project committee, my first advisor, Annette Yonke, and the final advisor, Alan Schwartz, the members, Arthur Elstein and Ara Tekian who provided the assistance and guidance, giving me the freedom and autonomy to work in this project and allowing my intellectual growth. I am sure that all other MHPE teacher and staff played a main role in this process, and I am very touched by all. Grateful acknowledgments are extended to the students enrolled in this project pilot course, who played a main role in this process.

v

TABLE OF CONTENTS

INTRODUCTION.........................................................................................................1 Distance Education ....................................................................................................1 Health Professions Development: the FAMEMA project ...........................................4 Health Information Systems Training ........................................................................6 Background - The Project Context .............................................................................9 METHODOLOGY......................................................................................................11 Justification .............................................................................................................11 Objectives................................................................................................................12 Hypothesis...............................................................................................................12 Population ...............................................................................................................12 Data Collection........................................................................................................12 COURSE IMPLEMENTATION .................................................................................13 a) Course tools.........................................................................................................13 b) Software..............................................................................................................14 c) General Structure of the Course ...........................................................................17 ASSESSMENT OF RESULTS....................................................................................19 RESULTS AND DISCUSSION ..................................................................................21 Student's Previous Knowledge.................................................................................21 Student's Pre and Post-Test ......................................................................................24 Web Server Log Analysis ........................................................................................29 Author's Perception - "Diary" ..................................................................................31 CONCLUSION ...........................................................................................................34 APPENDICES ............................................................................................................36 Appendix A .............................................................................................................36 Appendix B .............................................................................................................39 CITED LITERATURE................................................................................................40 VITA...........................................................................................................................45

vi

LIST OF TABLES TABLE I - Student Distribution by Course and Class, FAMEMA, 2001.................................... 21 TABLE II - Students Answer for Informatics Previous Knowledge, FAMEMA, 2001............... 22 TABLE III - Percentage of Positive Answer of System Use, FAMEMA, 2001 .......................... 22 TABLE IV - Simple Item Analysis for the Pre-Test of the Online Course, FAMEMA, 2001 ..... 25 TABLE V - Content Pairing Between Pre and Post-Test of the Online Course, FAMEMA, 2001 .......................................................................................................................................... 26 TABLE VI - Simple Item Analysis for the Post-Test of the Online Course, FAMEMA, 2001.... 26 TABLE VII - Comparison Between Pre and Post-Test of the Online Course, FAMEMA, 2001 . 27 TABLE VIII - Result of McNemar's Test Between Pre and Post-Test of the Online Course, FAMEMA, 2001................................................................................................................ 28

vii

LIST OF FIGURES

FIGURE 1 - Number of citations for internet search engines, FAMEMA, 2001 ......................... 24 FIGURE 2 - Monthly page requests for the Online Course web server, FAMEMA, 2001 .......... 30 FIGURE 3 - Day of the week page requests for the Online Course web server, FAMEMA, 2001 .......................................................................................................................................... 31 FIGURE 4 - Sequence of activities ............................................................................................ 39

viii

SUMMARY Context: The internet permits the implementation of distance education courses. The called 'physician for the 21st century' presupposes the ability to manage and use health information systems. This content is not effectively taught in Brazilian health sciences programs. Setting: Marilia Medical School, Brazil. Objectives: a) to implement a non-face-to-face course using teaching-learning innovative methodologies and free software through the internet; b) verify if students acquire knowledge in the online course; c) analyze the advantages and disadvantages for creating and developing the online course. Design: creating and implementing an online course about 'health information systems', using free software from the internet. Before the course the enrolled students do a pre-test. After the course the students perform another MCQ test as a post-test. Subjects: 43 undergraduate medical and nursing students, all volunteers. Results: Before the training, the students have computer literacy and use the internet and search engines almost always. All questions showed an increase in right answers. A total of 61.0% and 88.2% of correct answers were done, respectively, in the pre and post-test. There were 46.5% student drop-out since enrollment. In an interview the students refer lack of time to study for this course. The web page logbook showed a decrease in page views during the students' vacation and weekends. Only 18.1% of page views were from outside the institutional network. Conclusions: It is feasible to implement an online course using free software. The undergraduate students learn with this kind of course, besides the large amount of student drop-out.

ix

INTRODUCTION

Distance Education

The use of computers and communication technologies in learning has a history going back at least 30 years. In that time it has been called by many names, including computer-mediated communication (CMC), computer conferencing, online learning, internet-based learning, and telematics (McCormack and Jones, 1997). Kearsley (1998) defines online education as "any form of learning/teaching that takes place via a computer network." The internet consists of several services, that when combined, can recreate and enhance the interaction possible within classroom walls. The World Wide Web (WWW or just web) can make these interactions simpler than in the past because all of the services can be accessed through one software package, a web browser. Two-way computer mediated communication insert the interactivity that is necessary in distance courses delivered to teachers. The asynchronous technologies do not require the learners to be online at the same time. The web can also be used by the self-motivated teacher to find materials for lessons plans, student activities and personal enrichment (Box, 1999). The advent of the World Wide Web and related technologies has encouraged medical schools to generate a wealth of online curricular materials (Candler and Andrews, 1999). Administrators at universities see distance education as a viable alternative because it increases enrollment with less utilization of physical space (op. cit.). Although the importance of Information Technology (IT) in education is realized, literature reviews shows that there are some disagreement on effectiveness of IT as a main educational tool, requiring further empirical studies. As an example, in dentistry, the theoretical advantages of

1

2 web-based continuing education are numerous, but the currently available online resources are mostly of unacceptable quality (Schleyer, Johnson and Pham, 1999). Online continuing dental education (CDE) courses may become an important tool to help practitioners keep current. Several issues, however, need to be addressed before online CDE can reach its full potential (Schleyer, 1999). A study at California State University at Northridge, (Schutte, 1997; Black, 1997; Schutte, 1998), reported that students in a virtual classroom scored 20% higher than students in a traditional classroom (p