Nurse Education Today 34 (2014) 836–841
Contents lists available at ScienceDirect
Nurse Education Today journal homepage: www.elsevier.com/nedt
Evaluation of an online continuing education program from the perspective of new graduate nurses Selcuk Karaman a, Sevda Kucuk b,⁎, Melike Aydemir a a b
Department of Computer Education & Instructional Technology, Ataturk University, 25240 Erzurum, Turkey Department of Computer Education & Instructional Technology, Istanbul University, 34452 Istanbul, Turkey
a r t i c l e
i n f o
Article history: Accepted 7 September 2013
Keywords: Nurse education Distance education Online education Continuing education Program evaluation
s u m m a r y The aim of this study is to evaluate the online continuing education program from the perspectives of new graduate nurses. An evaluation framework includes five factors (program and course structure, course materials, technology, support services and assessment). In this study, descriptive research methods were used. Participants of the study included 2.365 registered nurses enrolled in the first online nursing bachelor completion degree program in the country. Data were collected by survey. The findings indicated that students were mostly satisfied with this program. The results of this study suggest that well designed asynchronous online education methods can be effective and appropriate for registered nurses. However, the provision of effective support and technological infrastructure is as vital as the quality of teaching for online learners. © 2013 Elsevier Ltd. All rights reserved.
Introduction Online education has become a convenient learning option for adult learners. Attendance at campus classes is often difficult for them (Maxfield, 2009) because of their work schedules, families, and other responsibilities. Given their working conditions, it is hard for nurses to continue their studies or develop a new profession via traditional classroom based education. Therefore, online education provides increased flexibility, access, and cost-effectiveness in nursing education (Holly, 2009). The number of nurses that joined online continuing education programs has increased dramatically in the last decade. Researchers have claimed that these nursing training programs have positive outputs at in-service training and graduate programs (Seiler and Billings, 2004). However, the quality of these programs has become a concern as the number of online continuing education programs grows (Martinez et al., 2006). Evaluation of Online Learning One of the key issues to be considered for the continuing development of online education is program evaluation. Evaluation plays an important role in identifying what is effective and what is not. Evaluation can also enable educators to think about what they are doing and achieving during the implementation (Rovai, 2003). The evaluation of online learning requires an evaluation approach or a model. There are ⁎ Corresponding author. Tel.: +90 442 231 4036; fax: +90 442 236 0445. E-mail addresses:
[email protected] (S. Karaman),
[email protected] (S. Kucuk),
[email protected] (M. Aydemir). 0260-6917/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.nedt.2013.09.006
many models for evaluation in the literature. Implementation, interpretation and analysis of evaluations are difficult with detailed and complex models. It is important to determine sub-factors of evaluation models to develop a more suitable and understandable framework. A summary of the literature relevant to evaluation factors of online education is presented below in Table 1 (Shelton, 2011). In order to build a clearer and more comprehensive evaluation framework, we reviewed a range of studies and examined evaluation variables. Five major clusters of factors have emerged; program and course structure, course materials, technology, support services and assessment. The scope and definitions of these factors are given below. Program and Course Structure The quality indicator that appeared frequently in the online education literature involved the overall program and course structure. Course structure includes the design of the program, organization of the courses in the curriculum, consistency between course objectives and the aims of the program (Chaney et al., 2007). Goals and structure of the program should be appropriate to students' conditions, needs and expectations. Course Materials Online education based on learning materials has various formats. Online learning materials should be designed based on the principles of learning theories (Rahman, 2006) in such a way as to facilitate effective learning of modular content and to guide the students' self learning. Course materials should be clear and meet program standards. Students
S. Karaman et al. / Nurse Education Today 34 (2014) 836–841 Table 1 The factors used for evaluation of online education. Author(s)
Factors
Khan (2001)
Institutional, management, technological, pedagogical, ethical, interface design, resource support, and evaluation. Lee and Dziuban (2002) Administrative leadership and support, ongoing program concerns, web course development, student concerns, and faculty support. Lockhart and Lacy Institutional readiness/administration, faculty services, (2002) instructional design/course usability, student readiness, student services, learning outcomes, retention. Osika (2004) Faculty support, student support, content support, course management system support, technology support, program support, and community support. Moore and Kearsley The number and quality of applications and enrollments; (2005) student achievement, student satisfaction, faculty satisfaction, program or institutional reputation, and the quality of course materials. Haroff and Valentine Quality of instruction, quality of administrative recognition, (2006) quality of advisement, quality of technical support, quality of advance information, and quality of course evaluation. Chaney et al. (2007) Teaching and learning effectiveness, student support, technology, course development/instructional design, faculty support, evaluation and assessment, and organizational/institutional impact. Jung (2011) Interaction, staff support, institutional quality assurance mechanism, credibility, learner support, information and publicity, learning tasks
should be provided information about what materials are included, how to use them and how they are related to each other (Horton, 2000). Technology Technology trends at the organizational level are important because they have both a direct and an indirect impact on training and development (Driscoll, 2002). The technological dimension of online learning includes infrastructure planning, hardware, and software (Shelton, 2011). Technology should be reliable, user-friendly, and accessible at all times (Billings, 2000; Billings et al., 2001). Infrastructure should facilitate different types of online learning activities. In addition it should promote mechanisms to improve stability and robustness of hardware and software systems to minimize disruption to online learning (Horton, 2000). Support Services One of the most important issues in online education is the support service in terms of providing guidance, preventing frustration and feelings of loneliness. Regular feedback mechanisms should be designed and implemented to assess the success and failure of the various support systems created for the online education system (Ragan, 1999). Support services should aim at reducing students' individual deficiencies and contributing to their learning (Usun, 2004). Providing support services to the online learning population is an important part of creating the feeling of belonging (Chaney et al., 2007). Assessment Assessment of the students for awarding of certificates, diplomas or degrees is the most difficult phase in the online learning process. Assessment activities (assignments, online exams) and final examinations should be appropriately designed for the online students and reflect the objectives of the course. Questions in the final paper should measure the learning outcomes at different levels of cognitive ability. The ratio of the marks of online and paper based assessments should be appropriate (Iqbal et al., 2011).
837
The evaluation framework which includes these five factors (program and course structure, course materials, technology, support services and assessment) was used to evaluate an online nursing education programs in this study. The program is the online bachelor completion degree in nursing (HELITAM), which has about 20,000 registered nurses. This is the first online nursing program in Turkey. Students are required to have two-year college degrees and be working as nurses in professional settings (e.g., clinics or hospitals) to be admitted to the program. There are ten courses in this two-year program such as healthcare management, communication in nursing, and biostatistics. The courses were delivered through interactive learning packages, lecture notes and videos on the Learning Management System (LMS). The students participated in two online exam activities and a paper based exam in each semester. Paper based exams were carried out in exam centers over the countries. Proctoring multiple choice final exam assessments were applied for reliability. About 13,000 students graduated from this program in 2011. Online nursing courses have increased rapidly in recent years and a number of studies have been conducted that examine these programs. However, little research has been published regarding the quality of online graduate nursing courses (Carter, 2010). Evaluation is an essential component of program improvement and renewal and long-term success. Moreover, online learning program evaluation results can be extrapolated to improve other programs (Rovai, 2003). In this study, evaluation of the HELITAM program is most closely aligned with the purposes of participant-oriented evaluation approaches. Using this approach, evaluators work to portray the multiple needs, values, and perspectives of the program stakeholders in order to make judgments about the value or worth of the program (Martinez et al., 2006). Evaluation of this program will contribute to the creation of online education policy for adult learners. It will be particularly helpful for directing online nursing education programs in the future. This study can also be considered as a case study of teaching nurses online. The aim of this study is to evaluate the online nursing bachelor completion degree program from the perspectives of new graduates. Consequently, the research question addressed in this study is: 1. How do new graduate nurses from an online continuing education program evaluate the program using the dimensions of program and course structure, course materials, technology, support services and assessment? Methods This study was designed as a case study research. Case study is an ideal methodology when a holistic, in-depth investigation is needed. Case study investigates a contemporary phenomenon within its reallife context (Yin, 2009) and analyses of persons, events, decisions, periods, projects, policies, institutions, or other systems that are studied holistically by one or more methods (Thomas, 2011). It has been used in varied investigations, particularly in sociological studies, but increasingly, in instruction. Case study is designed to bring out the details from the viewpoint of the participants by using multiple sources of data (Feagin et al., 1991). The researchers used both quantitative and qualitative approaches during data collection and analysis. The quantitative data were collected by survey and the qualitative data were collected through open-ended items in the survey. Participants Participants in the study are registered nurse students who completed all requisites of online nursing bachelor completion degree program and qualified for graduation. For this reason the participants who were about to graduate are called as students in the study. Since all students are working as RNs there are no major differences among students in terms of socioeconomic status and availability of technical resources
838
S. Karaman et al. / Nurse Education Today 34 (2014) 836–841
such as computer and internet access. Data were collected from the students at the end of the last semester. Students voluntarily participated in the survey among 13,000 students (N = 2.365; age range: 25–54 years; 95% female). The electronic survey was only available to new graduates of the program on the Learning Management System (LMS). Data Collection
dimensions. The descriptive statistics are shown in tables and interpreted with the help of qualitative data. General means of evaluation dimensions are illustrated in Table 2. The highest ranking dimensions were program and course structure and course materials. The lowest ranking dimensions were support services and technology.
Program and Course Structure
Participants were surveyed using the “Program Evaluation Survey of Online Continuing Education” instrument at the end of the last semester. The instrument had six sections (demographic information; program and course structure (12 items); course materials (six items); technology (four items); support services (five items) and assessment (three items)). Students were asked to rate the items on a five point Likert scale ranging from 1 to 5 where 1 = strongly disagree and 5 = strongly agree. There were also open-ended questions in the survey for each evaluation dimension. The absolute approach used to draw conclusions about satisfaction for participants. The survey included 5-point agree–disagree response choices, we can assume that agreement with positively-worded items and disagreement with negativelyworded items would represent satisfaction, whereas disagreement with positive-worded items, and agreement with negative-worded items represent dissatisfaction. In this survey positive statements for satisfaction were scored as 5–4–3–2–1. The score ranges were determined as follows: strongly agree: 4.21–5.00, agree: 3.41–4.20, partially agree: 2.61–3.40, disagree: 1.81–2.60, and strongly disagree: 1.00–1.80. The survey was developed by the researchers based on a review of related literature. After peer review by two graduate students, two experts examined the survey, and based on their feedback, the instrument was revised. It was then checked by a Turkish language expert for clarity of the language. After the revision, a pilot test was conducted before the main study using the same sample. Exploratory factor analyses were done for construct validity. The exploratory factor analysis was started with 42 items in total. The 12 items were subsequently excluded either for bearing low factor loading values or high factor loading values in multiple factors and etc. Exploratory factor analysis showed that there were items in the scales loaded under five factors. The total amount of variance revealed by this structure of the scale, which was summed up in 30 items and five factors as a result of exploratory analysis is 55.36%. After gathering data from 2.365 students, the Cronbach alpha coefficient was calculated as 0.92, indicating a satisfactory reliability. Data Analysis In this study, the quantitative responses were analyzed through descriptive statistics. Means and standard deviations of questionnaire items were calculated. The qualitative data were gathered through open-ended items in the survey and analyzed via content analysis.
Survey questions related to program and course structure are illustrated in Table 3. Generally students thought the program was quite useful and satisfactory in terms of program and course structure. A majority of students stated that the HELITAM program was a good opportunity for them (M = 4.70). As shown in Table 3, the lowest ranking item related to program and course structure was that my reputation has increased at work because of participation in this program (M = 3.32). Participants appreciated the convenience and flexibility of the program. Students' responses to open ended questions supported the quantitative findings that students were mostly pleased with the overall design of the program. The basic idea behind the satisfaction of program was the desire to get a new education and professional development. Flexibility and time boundlessness of online learning were also important. Feelings of being up to date in knowledge of nursing were also related to perceptions about program and course structure. Sample statements from open-ended questions are given below: “…the program was quite useful for me and also the information was up to date. I think that I developed myself. I succeeded with distance education without detriment to my work.” “The best part of distance education is the ability to study whenever we wanted. I listened to the lectures repeatedly. Also we studied regularly thanks to getting courses weekly.”
Course Materials Survey questions related to course materials are illustrated in Table 4. Generally students stated that they liked the course materials and the materials supported their learning. A majority of students stated that the reading text materials were sufficiently informative (M = 3.86). Participants expressed in open responses that course materials were quite informative and audio-visual components provided permanent records of learning. Many students found that the quality and quantity of the resources were valuable and appropriate. However, some students complained about long and detailed materials and preferred to study from books instead of online materials. Sample statements from the open-ended questions are given as follows: Table 3 Means and standard deviations of program and course structure dimension.
Results The main findings obtained from the survey are presented in this section according to the five main dimensions of the evaluation framework and responses to the survey items are presented under related
Table 2 Mean scores of dimensions in evaluation framework. Evaluation dimensions
M
Program and course structure Course materials Assessment Technology Support services
3.84 3.80 3.68 2.93 2.70
Items
Mean SD
Availability of HELITAM program was a big opportunity for me. I think this program is useful for professional development. This program enabled me to undertake education without losing time. This program provided me with the information I needed. Lectures were suitable for the aim of the program. The program quality prepared me for master's education. Course contents were suitable for the aim of the program. The placement of courses in the semester was appropriate. Course content in different formats (reading, e-material, video etc.) was sufficient. The program was well designed. All of the courses in this program were necessary. My reputation has increased at work thanks to participating in this program.
4.70 4.29 4.00 3.93 3.83 3.83 3.77 3.70 3.69
.689 .984 1.094 1.074 1.088 1.100 1.114 1.130 1.195
3.54 3.50 3.32
1.168 1.293 1.234
S. Karaman et al. / Nurse Education Today 34 (2014) 836–841 Table 4 Means and standard deviations of course material dimension.
839
Table 6 Means and standard deviations of support service dimension.
Items
M
Reading texts were sufficiently informative. Topics of lecture notes were consistent with each other. Course resources included up to date information. Objectives of contents which included knowledge, skills and behaviors were clearly stated in course materials. E-lesson packages supported my learning. Course resources were appropriate to my level of study.
3.86 1.054 3.85 .973 3.84 1.043 3.81 1.000
SD
Items
M
SD
I got enough support related to student affairs. I could send some requests and suggestions about lectures. I could contact instructors when it was necessary. I could get support when I had problems with courses. I could get support when I had technical problems.
3.14 2.91 2.54 2.50 2.42
1.304 1.286 1.288 1.235 1.226
3.80 1.051 3.65 1.076
“Course materials I received were up to date, suitable for the nursing curriculum and easily understandable.” “Every course subject was supported individually with documents and videos. Thus I have a permanent record of the resources.” “Some of the course materials were long and too detailed.” “Course materials could be provided as a book, it was difficult to file and print every week.”
interaction with instructors. Sample statements from open-ended responses are given below: “My calls were not answered and I couldn't interact with the instructor.” “More than half of my calls were not answered, but when I did reach the call center, I was helped.” “I think support services are one of the deficient aspects of the program. It was hard to find someone to talk to when there was an emergency or a need to consult about an issue.”
Technology
Assessment
Survey questions related to technology are illustrated in Table 5. Students' views about the technology dimension were average. Some of students stated that they did not have any problem accessing course materials (M = 3.39). The lowest ranking item shows that students had some problems in accessing the system (M = 2.32) and these problems reduced their willingness to learn (M = 2.86). The open ended responses relating to technology indicated that generally students had some problems in accessing the system especially during online exams and technical issues often prevented their learning. Nevertheless, they were pleased with using online education technology and quite satisfied with their improved computer literacy skills. Sample statements from open-ended questions are given below:
Survey questions related to assessment are illustrated in Table 7. Generally students found that exams were reliable (M = 3.88). Students stated that exams evaluated their knowledge level accurately (M = 3.63). Also students stated that exams were consistent with course contents (M = 3.55). Participants expressed their satisfaction with exams. According to students while online exams provide flexibility, paper based exams ensured reliability of overall assessments. In particular, they commented that online exams in the middle of the term were convenient for them. Sample statements from open-ended questions are given below: “The questions were consistent with lecture contents.”
“We sometimes couldn't attend courses. We experienced some difficulties during online exams.”
“…I can say that paper based exams are quite good.”
“It was great in terms of improving my computer literacy.”
“Questions were definitely appropriately prepared. They were measuring our ability to reason rather than just memorizing.”
“Problems of accessing the system negatively affected my willingness to study.”
“Having the opportunity to take online exams whenever I want, I had no problems despite my working conditions.”
Support Services
Conclusions
Survey questions related to the support services are illustrated in Table 6. The mean of this dimension is lower than the others. Student ratings regarding the support services related to student affairs were average (M = 3.14). A majority of students stated that they did not seek enough help with their academic (M = 2.50) and technical (M =2.42) problems. Participants also expressed a lack of professional support in their open ended responses. Students stated that they faced some problems on reaching the call center. Also, students complained about insufficient
The aim of this study was to evaluate the online continuing education program from the perspective of new graduate nurses. An evaluation framework which includes five factors (program and course structure, course materials, technology, support services and assessment) was used to evaluate the online nursing education program. The findings indicated that students were mostly satisfied with this program. The findings are discussed below by evaluation dimension. Students were mostly pleased with the overall design in terms of convenience and flexibility of the program. These findings are consistent with the literature, which reports that nurses are satisfied with an
Table 5 Means and standard deviations of technology dimension.
Table 7 Means and standard deviations of assessment dimension.
Items
M
SD
I accessed course contents easily via the system. I had no problems which reduced my willingness to learn. I had no problem in accessing the system. I hadn't to spend time on system problems.
3.39 2.86 2.32 1.82
1.239 1.384 1.280 1.353
Items
M
SD
I think exams in the program were reliable. Final exams were suitable for assessing my knowledge level. Exam questions were consistent with course contents.
3.88 3.63 3.55
1.085 1.283 1.209
840
S. Karaman et al. / Nurse Education Today 34 (2014) 836–841
online format particularly because it provides convenience and flexibility (Fearing and Riley, 2005; Phillips, 2006; Gerkin et al., 2009). Students stated that this program was a big opportunity for them as they could gain their bachelors degree thanks to the program which is the first online continuing education program in Turkey for nurses. The use of distance learning models is a major factor in motivating an adult student to return to school (Tsai, and Chuang, 2005). The asynchronous delivery format of this program might increase the convenience for working adults (Maxfield, 2009). Students stated that the program was useful in terms of professional development. This finding is partly related to their perceived need for education. “Adult learners that volunteer to learn have intrinsic motivation and usually take courses to upgrade or develop skills needed for employment” (Moore and Kearsley, 2005, p. 162). Positive approach to the program can be based on the match between the course objectives and their needs (Karaman, 2011a). Also, students are pleased with the scope and quality of the program. The findings are consistent with previous online education research that suggests the primary predictor of student satisfaction is quality of instruction (Moore, and Kearsley, 1996; DeBorurgh, 2003; Sun et al., 2008). Many students found that the quality and quantity of the resources, which included modules, audio-visual materials and texts were valuable and appropriate for them. Satisfaction with materials related to diversity and interactivity of contents because effective course materials should be interactive, conversational, self-pacing and pedagogically sound (Rahman, 2006). In terms of material structure, students were satisfied with having learning objectives in course materials. Making explicit the connection between course activities and learning outcomes facilitates metacognition, in other words, enables the students to monitor their own learning and may improve motivation (Avery et al., 2008). Some of the students complained about long and more detailed materials. These may be caused by the frustration of information overload (Maxfield, 2009). Furthermore, some of the students preferred to study from books rather than online materials. This finding may be related to the reading habits of adult learners (Billings et al., 2001). Quality of technology access is one of the most important factors influencing student satisfaction (Bolliger and Martindale, 2004; Bower and Kamata, 2000). Students expressed that they had some technical problems and these kinds of problems reduce willingness to learn. It means that frustration with technology decreased learner satisfaction as stated in many researches (Carter, 2010; DeBorurgh, 2003; Hillman et al., 1994). Having technological problems may be related to insufficient computer literacy and technical infrastructure. Online learners must be familiar with the technology used in the program in order to be successful (Belanger and Jordan, 2000). Students complained about insufficient support for them to resolve their problems. This finding is in parallel with other studies which stated that graduate students complained about technology problems and the lack of technical support (Muilenburg and Berge, 2005). Some students complained about the lack of interaction with the instructor. Access to the instructor and personal contact are important to students and impact their satisfaction with the learning experience (Kooker et al., 1994). Students also expected immediate help from instructors and support staff. However, this was an asynchronous program, based mostly on learning materials with a huge number of students; these are major constraints on the program. Therefore, low satisfaction from support services of this program was inevitable. Nevertheless, lack of interaction and personal contact with the instructor was repeatedly cited as the major weakness of online education (DeBorurgh, 2003; Kooker et al., 1994). Assessment is a critical dimension in assuring the quality of e-learning (Frydenberg, 2002). Students were pleased with the format of exams (online and paper based exam) in the program and they thought that exams evaluated their knowledge level accurately. Students found that online exams were beneficial in terms of engagement
with courses and were convenient. Time flexibility for online exams reduces exam anxiety (Karaman 2011b; Olt, 2002; Ozden et al., 2004) and gives more freedom. Students also thought that the paper based exam was suitable for online education in terms of objective testing (Holcomb et al., 2004). This finding runs counter to another study which appears the need not include assessment dimension in judging e-learning quality (Jung, 2011). In designing, developing, and delivering online education courses, students' needs and perceptions should be central (Sahin, and Shelley, 2008). The sample in this study was huge and respondents were new graduates from the program. We obtained a deeper understanding of respondents' evaluation of the program by using open ended questions in the survey. Opinions and recommendations of online learners who completed the program can be used to improve the course or program (Rovai, 2003). However, it should be regarded that perspectives of students can easily be affected by regional and cultural factors of Turkey as well as context of the program. Contextual properties are important for those who are considering incorporating online instruction in existing or emerging programs of a similar nature. Another constraint of this study is that it did not consider learner variables such as ages, gender, technology skills and experiences with e-learning in examining the learner perception on e-learning. The results of this study suggest that well designed asynchronous online education methods can be effective and appropriate for registered nurses. A large number of nurses may attend this kind of online continuing education program. However, the provision of effective support and technological infrastructure is as vital as the quality of teaching for online learners since a lack of technical and academic support decreases learning motivation. The design should enhance learner support, interaction and technologies. As a result of this study, the following recommendations are specifically directed to academic administrators and researchers. The design of online continuing education programs needs to provide: • • • • • •
courses which can contribute to professional development certification or degree interactive and self-learning materials accessibility and durability of technical services continuous academic and technical support both online and under observation exams.
Different formats (synchronous, asynchronous) and scope (undergraduate, graduate, certification) of programs could be examined as future studies. Also researching the effects of this kind of program on students' performance at work could be valuable. References Avery, M., Cohen, B., Walker, J.D., 2008. Evaluation of an online graduate nursing curriculum: examining standards of quality. Int. J. Nurs. Educ. Scholarsh. 5 (1), 1–17. Belanger, F., Jordan, D.H., 2000. Evaluation and Implementation of Distance Learning: Technologies, Tools and Techniques. Idea Publishing Group, Hershey, PA. Billings, D.M., 2000. A framework for assessing outcomes and practices in Web-based courses in nursing. J. Nurs. Educ. 39 (2), 60–67. Billings, D.M., Connors, H.R., Skiba, D.J., 2001. Benchmarking best practices in Web-based nursing courses. Advances in Nursing Science 23 (3), 41–52. Bolliger, D.U., Martindale, T., 2004. Key factors for determining student satisfaction in online courses. Int. J. E-Learn. 3 (1), 61–67. Bower, B.L., Kamata, A., 2000. Factors influencing student satisfaction with online courses. Acad. Exch. Q. 4 (3), 52–56. Carter, M.W., 2010. An Evaluation of Online Graduate Nursing Programs: A Comparison of Three Private, Religious-affiliated Universities. Phd Union University, Tennessee, USA. Chaney, B.H., Eddy, J.M., Dorman, S.M., Glessner, L.L., Green, B.L., Lara-Alecio, R., 2007. A primer on quality indicators of distance education. Soc. Public Health Educ. 10 (2), 222–231. DeBorurgh, A.G., 2003. Predictors of student satisfaction in distance-delivered graduate nursing courses: what matters most? J. Prof. Nurs. 19 (3), 149–163. Driscoll, M., 2002. Web Based Training: Creating E-learning Experiences. Jossey-Bass/ Pfeiffer, San Francisco. Feagin, J., Orum, A., Sjoberg, G. (Eds.), 1991. A case for case study. University of North Carolina Press, Chapel Hill, NC. Fearing, A., Riley, M., 2005. Graduate students' perceptions of online teaching and relationships to preferred learning styles. Medsurg Nurs. 14 (6), 383–389.
S. Karaman et al. / Nurse Education Today 34 (2014) 836–841 Frydenberg, J., 2002. Quality standards in e-learning: a matrix of analysis. Int. Rev. Res. Open Distance Learn. 3 (2). Gerkin, K.L., Taylor, T.H., Weatherby, M.F., 2009. The perception of learning and satisfaction of nurses in the online environment. J. Nurses Staff Dev. 25 (1), 8–13. Haroff, P.A., Valentine, T., 2006. Dimensions of program quality in web-based adult education. Am. J. Distance Educ. 20 (1), 7–22. Hillman, D., Willis, D.J., Gunawardena, C.N., 1994. Learner–interface interaction in distance education: an extension of contemporary models and strategies for practitioners. Am. J. Distance Educ. 8, 30–42. Holcomb, L.B., King, F.B., Brown, S.W., 2004. Student traits and attributes contributing to success in online courses: evaluation of university online courses. J. Interact. Online Learn. 2 (3), 1–17. Holly, C., 2009. The case for distance education in nursing. J. Online Learn. Teach. 5 (3) (Retrieved from http://jolt.merlot.org/vol5no3/holly_0909.htm). Horton, W., 2000. Designing Web-based Training: How to Teach Anyone Anything Anywhere Anytime. John Wiley and Sons, New York. Iqbal, M.J., Ahmed, M.U., Rauf, A., 2011. Evaluation of examination system at Allama Iqbal Open University (AIOU). Int. J. Acad. Res. 3 (3). Jung, I., 2011. The dimensions of e-learning quality: from the learner's perspective. Educ. Technol. Res. Dev. 59 (4), 445–464. Karaman, S., 2011a. Nurses' perceptions of online continuing education. BMC Med. Educ. 11 (86) (Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3215975/). Karaman, S., 2011b. Examining the effects of flexible online exams on students' engagement in e-learning. Educ. Res. Rev. 6 (3), 259–264. Khan, B., 2001. A framework for web-based learning. In: Khan, B. (Ed.), Web-based training. Educational Technology, Englewood Cliffs, NJ, pp. 75–98. Kooker, B.M., Itano, J., Efinger, J., Dungan, J., Major, M., 1994. Interactive television: delivering quality graduate nursing education to remote sites. J. Nurs. Educ. 33, 188–190. Lee, J., Dziuban, C., 2002. Using quality assurance strategies for online programs. Educ. Technol. Rev. 10 (2), 69–78. Lockhart, M., Lacy, K., 2002. As assessment model and methods for evaluating distance education programs. Perspectives 6 (4), 98–104. Martinez, R., Liu, S., Watson, W., Bichelmeyer, B., 2006. Evaluation of a Web-based master's degree program: lessons learned from an online instructional design and technology program. Q. Rev. Distance Educ. 7 (3), 267–283. Maxfield, R.J., 2009. Asynchronous online learning: perceptions and experiences of nontraditional adult emergency services students. Int. Fire Serv. J. Leadersh. Manag. 3 (2), 23–30. Moore, M.G., Kearsley, G., 1996. Distance Education: A Systems View. Wadsworth Publishing Company, San Francisco. Moore, M., Kearsley, G., 2005. Distance Education: A Systems View, 2nd ed. Thomson Wadsworth, Belmont, CA. Muilenburg, L.Y., Berge, Z.L., 2005. Student barriers to online learning: a factor analytic study. Distance Educ. 26 (1), 29–48. Olt, M., 2002. Ethics and distance education: strategies for minimizing academic dishonesty in online assessment. Online J. Distance Learn. Adm. 5 (3) (Retrieved from http:// www.westga.edu/~distance/ojdla/fall53/olt53.html). Osika, E.R., 2004. The Concentric Support Model: A Model for the Planning and Evaluation of Distance Learning Program. Unpublished doctoral dissertation Chicago State University, USA (Retrieved from ProQuest Dissertations and Theses. (UMI No. 3150815)). Ozden, Y., Erturk, I., Sanli, R., 2004. Students' perceptions of online assessment: a case study. Am. J. Distance Educ. 19 (2), 77–92. Phillips, J.M., 2006. Preparing preceptors through online education. J. Nurses Staff Dev. 22 (3), 150–156.
841
Ragan, L.C., 1999. Good teaching is good teaching: an emerging set of guiding principles and practices for the design and development of distance education. Cause/Eff. 22 (1), 20–24. Rahman, M.H., 2006. Developing course materials for open and distance learning: BOU perspective. Turk. Online J. Distance Educ. 7 (4). Rovai, A.P., 2003. A practical framework for evaluating online distance education programs. Internet High. Educ. 6, 109–124. Sahin, I., Shelley, M., 2008. Considering students' perceptions: the distance education student satisfaction model. Educ. Technol. Soc. 11 (3), 216–223. Seiler, K., Billings, D.M., 2004. Student experiences in Web-based nursing courses: benchmarking best practices. Int. J. Nurs. Educ. Scholarsh. 1 (1), 1–12. Shelton, K., 2011. A review of paradigms for evaluating the quality of online education programs. Online J. Distance Learn. Adm. 4 (1) (Retrieved from http://www.westga. edu/distance/ojdla/spring141/shelton141.html). Sun, P., Tsai, R.J., Finger, G., Chen, Y., Yeh, D., 2008. What drives a successful e-learning? An empirical investigation of the critical factors influencing learner satisfaction. Comput. Educ. 50, 1183–1202. Thomas, G., 2011. A typology for the case study in social science following a review of definition, discourse and structure. Qual. Inq. 17 (6), 511–521. Tsai, C.C., Chuang, S.C., 2005. Colloquium: the correlation between epistemological beliefs and preferences toward Internet-based learning environments. Br. J. Educ. Technol. 36 (1), 97–100. Usun, S., 2004. Learner support services in distance education system (A case study of Turkey). Turk. Online J. Distance Educ. 5 (4). Yin, R., 2009. Case Study Research: Design and Methods, 4th ed. Sage Publishing, Thousand Oaks, CA.
Selçuk Karaman is an associate professor in the Department of Computer Education & Instructional Technology and vice dean of the Open Education Faculty at Ataturk University. His research focuses on learning objects, distance education, Internet applications in education, management in instructional system, instructional and performance technology and mental models. Author contributions: conception and design, writing the manuscript, and statistical expertise. Sevda Kucuk is a Ph.D. student and research assistant at the Department of Computer Education & Instructional Technology at Ataturk University. Her research interests are in the computer-based instruction, teacher education, distance education, instructional design, instructional strategies, and research methods. Author contributions: conception and design, data collection, and writing the manuscript. Melike Aydemir is a Ph.D. student and research assistant at the Department of Computer Education & Instructional Technology and online program coordinator of the Distance Education Center at the Ataturk University. Her research interests are in the distance education, interaction among learners to aid learning during distance education, instructional technology and instructional design. Author contributions: data collection, analysis and interpretation, and writing the manuscript.
