creating class discussions; and (5) creating computer-based instruction. ... learning tracks, and managing learning activities in an online environment (Falvo ...
Web-based Learning Management System Considerations for Higher Education
Chih-Hung Chung, University of North Texas Laura A. Pasquini, University of North Texas Chang E. Koh, University of North Texas
Abstract Coinciding with the development and growth of the Internet, there has been a dramatic increase in the application of the Learning Management Systems (LMS) in higher education. University and college campuses should consider evaluating each LMS to ensure that the system meets the requirements and demands of the institution. Therefore, the purpose of this study is to present a model which incorporates the concepts and findings from research on LMS application in higher education. The alternative model was modified based on the Technology Acceptance Model (TAM). In addition, five categories of LMS features for higher education are discussed including: (1) transmitting course content; (2) evaluating students; (3) evaluating courses and instructors; (4) creating class discussions; and (5) creating computer-based instruction. This study reviews and discusses prior research and provides several recommendations including a model of development and design of an LMS for future implementation in higher educational environments. Keywords: Learning Management System, Technology Acceptance Model, Human Effect, Human Computer Interface.
Learning and Performance Quarterly, 1(4), 2013
24
Web-based Learning Management System Considerations for Higher Education Introduction With the development of learning technologies, learning management systems (LMS) have become an important component in the education field. Most universities and colleges in the United States (U.S.) have adopted an LMS to support instructor teaching activities and student learning processes. One of the most important features of an LMS is to provide an environment for learning and teaching without the restrictions of time or distance (Epping, 2010). Since the increased development of the Internet, the LMS concept has been broadly applied at various higher education institutions around the world. LMSs helps instructors and learners discuss the course content by posting and responding to each other, maintaining student learning tracks, and managing learning activities in an online environment (Falvo & Johnson, 2007). Currently, LMSs improve instructor teaching and student performance across various fields of study (Boggs, Shore, & Shore, 2004). Since Blackboard Corporation introduced Blackboard in 1997 (Falvo & Johnson, 2007); over 60 countries have adopted the Blackboard LMS. As technology continues to develop, Blackboard faces increasing competition from LMS rivals, such as Desire2learn and Moodle. These learning management systems present convenient functions and innovative course management approaches. For instance, Moodle, an open-source course management system, offers a flexible management environment for users, and it can quickly add or modify available extension models (Kumar, Gankotiya, & Dutta, 2011). In addition, Unal and Unal (2011) suggested that Moodle would be an effective alternative LMS to Blackboard and other LMSs because of its flexibility and open-source resource. These instructor functions and course management approaches urge users to choose a flexible LMS system. As a result of increased competition and advanced technology, higher education institutions now have a wide variety of options to manage their learning curriculum. Each LMS offers specific functions and management approaches, so choosing the appropriate system becomes an important concern for educational institutions. In addition, an LMS does not offer enough finalized functions to satisfy the demands of the institutions. As a result, institutions must spend valuable time and effort comparing each LMS system individually to ensure that the one chosen meets their demands. Although LMSs have become increasingly popular, several drawbacks and limitations exist. There is a lack of social interaction. Social interaction encourages high learner motivation, which has the potential to improve users’ teaching and learning performance (Chou & Chou, 2011). In addition, Huffman and Huffman (2012) suggested that students utilizing the appropriate technological tool improved learning performance. This paper addresses the question, “What types of designs should LMSs utilize for higher education?” The purpose of this manuscript is to provide an alternative model of a LMS design and development by reviewing important findings from research on LMS in higher education learning environments. In addition, the authors reviewed five categories of LMS features for higher education to help contribute to the design and structure of effective and efficient learning management systems. After discussing the relevant research, this study offers a model for LMS design for higher education institutions based on a variation of Davis’ (1989) Technology Acceptance Model (TAM). Due to the widespread use of Blackboard as an LMS in higher education, this system will be utilized as a baseline to discuss design applications required for higher education learning curriculum. Learning and Performance Quarterly, 1(4), 2013
25
Theoretical Framework The theory, used as a framework, in this study was the Technology Acceptance Model (TAM) developed by Davis (1989). It was adopted to explain the acceptance and use of technology in LMSs by end users. This theory posited that the perceived usefulness and ease of use would best determine an individual's intention to utilize a particular technology. In addition, five categories of LMS features for higher education, proposed by Malikowski, Thompson and Theis (2007), are discussed as the major features in LMSs. As applied to this study, the TAM theory and research model guided the literature review and the discussion of the findings as they pertain to the development and design of LMSs. Literature Review This section overviews and categorizes the literature related to learning management system (LMS) into several parts, including the definition of LMS, the Human-Computer Interaction (HCI), the human effect on the online environment and five categories of LMS features with HCI, the Technology Acceptance Model (TAM), and cross-cultural issues in the online environment. Each topic will provide a modified framework of LMS design and development. Definition of LMS Learning Management Systems (LMSs) have been widely used in higher education due to their many advantages including flexible learning times and unlimited distance education (Hamuy & Galaz, 2009). Research on the teaching, retention, and learner usability for learning management systems has been mounting steadily for a number of years (Coates, James & Baldwin, 2005; Ozkan, Koseler & Baykal, 2009; de Porto Alegre Muniz et al, 2012; Alhazmi & Rahman, 2012). Various definitions of learning management systems have been proposed over the course of years of research; however, Simonson (2007) provided a practical definition of LMS: Course management systems, also called learning management systems or virtual learning environments, are software systems designed to assist in the management of educational courses for students, especially by helping teachers and learners with course administration. The systems can often track the learners’ progress. While usually thought of as primarily tools for distance education, they are also used to support the face-to-face classroom (p. vii). Definition of Learner Performance Tsao et al. (2007) define learner performance as the knowledge and skills learned, partnered with the learning attitudes they display. Learners in higher education are the user of the learning management systems and should be considered during evaluation of the platforms. Further, learner performance is articulated as the knowledge and skills obtained during lessons with the resulting attitude and behavior displayed by the learner Lee (2002). This would result in the learning success and meet the learning objectives set by the course curriculum. Definition of Culture Culture can be defined in many different ways; however specifically, for the online environment, Hofstede (1998) defined culture as, “[T]he collective programming of the human mind that distinguishes the members of one human group from those of another. Culture in this Learning and Performance Quarterly, 1(4), 2013
26
sense is a system of collectively held values” (p. 478). The authors adopted Hofstede’s (1998) definition of culture to study the impact of culture on LMS design. Overview of Learning Management Systems (LMSs) An LMS is a particular type of software system designed and promoted for instructors and learners to utilize in teaching and learning activities. Typically LMSs offer various tools, such as course content organization and presentation, communication tools, student assessment tools, grade book tools, and other functions supporting classroom learning and teaching performance (Morgan, 2003). The 2009 report from the American Society for Training and Development (ASTD) revealed that ninety-one percent of ASTD respondents in their organizations utilized LMSs, and it also identified the challenges of using an LMS. Customization requirements and content integration need to be overcome in the learning management system (Ellis, 2009). To improve LMS implementations, organizations need to consider the virtual learning environments to improve learning outcomes, ensure accessibility, and engage learners in user-friendly virtual learning environments (ON24 Inc., 2012). Since the field of Information Technology (IT) is constantly growing, various LMSs and technologies to support learning management systems have been developed. The challenge included evaluations of benefits and functional factors for effective consideration for web-based learning management systems that will be effective for both the instructor and learner in higher education. In order to offer effective LMS evaluation methods, many researchers have proposed several methods to compare differences between each LMS. Lin’s (2010) analytic hierarchy process proposed several key issues for evaluating the LMS, such as system quality, information quality, reliability, and attractiveness. With regards to key HCI issues, Oztekin, Kong, and Uysal (2010) utilized a criticality metric analysis to evaluate a learning management system and presented the Uselearn assessment model to explain the relationship between each factor. Usability emerged as a major component to evaluate the quality of an LMS; in addition, HCI has been proposed that it was useful for improving the usability of computer systems (Schmidt, 1997). For these reasons, HCI plays an important role in developing and designing LMSs. Cross-cultural Factors in the Online Environment In recent years, cross-cultural learning research has received considerable attention. Lee and Croker (2006) explored the related research as it pertains to the cross-cultural environment. Their study evaluated the influences of expatriate characteristics, the complexities of task assignments, and observed cross-cultural differences and they proposed that the training of expatriates was one of the most vital elements for successful business globalization (Lee & Croker, 2006). In contrast, little research has been done on cross-cultural issues in the online environment, which is quite relevant since these issues also contribute to enhancement of the student’s learning performance (Cameron & Limberger, 2004). A growing number of Asian students have enrolled in online courses at universities across the U.S. (Ku & Lohr, 2003) and management of the online cultural issues should be considered. Ku and Lohr’s (2003) study addressed important claims regarding whether the Chinese culture influences Chinese students’ learning behaviors when they enrolled in online courses in the U.S., particularly during their first year. Ku and Lohr’s (2003) proposed that Chinese students believe online courses made them feel isolated from other students and rarely allowed any interaction with classmates; however, American students did not feel isolated. To address this concern, researchers proposed suggestions for designing online courses for international students as follows: (1) Increase the Learning and Performance Quarterly, 1(4), 2013
27
self-confidence and motivation of students early on; (2) offer the opportunity for students to work in small groups to increase their interaction; (3) maintain a self-paced and self-directed design of the online learning environment; (4) provide various opportunities for reading and writing; and (5) encourage student meetings with group members and instructors (Ku & Lohr, 2003). Yang, Olesova and Richardson (2010) systemically reviewed the impact of culture and social interaction within the online environment. The results of their study showed that the diversity of culture had a significant influence on HCI in the web design. Therefore, crosscultural issues are an important factor for LMS design and development. The Human Effect on the Online Environment According to a study by Southwell, Anghelcera, Himelboima, and Jones (2007), a variety of human factors, such as user control and experience, influenced user-learning patterns as they used the LMS. Hence, the LMS should consider the human effect to enhance instructor and learner experiences and satisfaction (Chrysostomou, Chen, & Liu, 2009; Kim, Kwon, & Cho, 2011). A variety of research adopted assumption-driven, statistical techniques to analyze the data in order to present useful information in regards to the human effect in the LMS; however, Chrysostomou, Chen, and Liu (2009) found the scope of the results was restricted due to statistical hypotheses and they adopted data mining analysis techniques to investigate the influential factors as consumers used the LMS. The results of this research proposed four clusters: (1) users like a single window layout with a static button, no use of icons and menus, and colors with the effect's scheme; (2) users like to use multiple window layouts with dynamic buttons, use of icons and menus, and standard color scheme format; (3) users like to utilize a single window layout with a static button, use of icons, but no menus, and multiple color schemes; and (4) users like to utilize various windows with dynamic buttons, no use of icons and menus, and colors with the effect's scheme (Chrysostomou, Chen, & Liu, 2009). The influential factors resulting in the identified four clusters, suggested a particular color scheme was most popular, which was also compatible with cognitive theory (Swelleer, Van Merrienboer, & Paas, 1998). With various human effects, such as experience and habits, users expressed different results in HCI. In addition, computer novices preferred to use particular window layouts and a static button, whereas computer experts preferred to use multiple window layouts and drop-box menus. According to research by Beckers and Schmidt (2003), computer anxiety and the avoidance of using technology makes users feel a lack of control. Moreover, the less time people are exposed to computers, the less they feel in control. A study by Chrysostomou et al. (2009) revealed that different experiences using the computer influences users’ teaching and learning performance. LMS developers should consider the impact of use and user’s length of time in a web-based learning environment when designing features for course content delivery. Five Categories of Learning Management Systems Features Malikowski, Thompson, and Theis (2007) proposed an LMS research model, which includes the following five categories with HCI: (1) transmitting course content; (2) evaluating students; (3) evaluating course and instructors; (4) creating class discussions; and (5) creating computer-based instruction. According to their study, transmitting course content was the most important function utilized by instructors. For instance, instructors utilized an LMS to announce important events, such as midterm exams, articles for reading, course information, syllabi, and assigned tasks. In contrast, certain LMS features were not utilized by instructors, specifically Learning and Performance Quarterly, 1(4), 2013
28
those who were not familiar with the computer-based applications or available interaction options with their learners (Malikowski et al., 2007). Utilizing more of the available LMS features, such as the underused synchronous discussions, would allow instructors to achieve their teaching goals and improve the learning performance of their students (Malikowski et al., 2007). Additionally, Kibaru and Dickson-Deane (2010) and Niedlsen (2003) proposed a usable online environment that could enhance web users’ performance. Abdalla (2007) adopted the TAM framework to evaluate the effectiveness of Blackboard and suggested that the convenience of technology enhanced the effectiveness of the LMS. In other words, instructors would be able to reduce the time to manage and set-up the course to improve their teaching experience, as well as the students’ learning experience (Inversini, Botturi, & Triacca, 2006). Nevertheless, previous research suggested that the development of an LMS might suffer by failing to devote a sufficient amount of attention to the overall design (Tsang, Kwan, & Fox, 2007). Integrating various appealing options might result in too many LMS features which could potentially confuse learners and stifle the LMS potential to help support the course objectives (Kidney & Puckett, 2003). Due to this fact, the development of an LMS should be systematically analyzed and designed to promote effectiveness and efficiency of the users’ learning performance. HCI in LMS A decade of HCI research has offered beneficial information on how users perform and think about the system. Research in this area provides important insight for technology usability and consideration of the user for the design element of HCI (De Lera, Fernandez, & Valverde, 2010). According to the International Organization for Standardization, usability means that users can effectively use a tool to finish a task with satisfaction (ISO 9241-11, 1998). The better HCI that LMS offers users, the greater satisfaction users will have with their systems. Hollender, Hofmann, Deneke, and Schmitz (2010) also discussed the significant influence of HCI on online learning. Although usability can improve the learning experience for students, Tselios, Avouris, and Komis (2008) pointed out that learning technology should not solely support the efficient execution of a task. By simply improving usability, one might have a negative impact on the learning experience in specific cases because implementing a task efficiently might preclude crucial learning processes. Therefore, a balanced design of HCI is one of the key components in the design and development of LMS. In order to develop and design a higher quality LMS, the five HCI categories, (1) transmitting course content; (2) evaluating students; (3) evaluating course and instructors; (4) creating class discussions; and (5) creating computer-based instruction, should be considered. As evident from the literature review, HCI can influence the effectiveness and ease of use. Paul (1989) presented a simple definition of HCI, which is “studying the interaction between computers and humans.” In addition to enhancing the usability of an LMS, HCI plays an important role in attaining the goal of improving user performance (Sung & Mayer, 2012). Moreover, each of these five categories aid in the understanding and improvement of quality LMS design. Therefore, under the premise that HCI will positively impact the usability of LMS, the authors will evaluate and develop five major categories of LMS characteristics with HCI as functions of learning management systems. In our modified model of the LMS design, HCI included five categories to help designers to develop an effective and efficient framework.
Learning and Performance Quarterly, 1(4), 2013
29
Technology Acceptance Model (TAM) Davis (1989) proposed the Technology Acceptance Model (TAM) to predict the phenomenon of a new technology adopted within a group. TAM emphasizes the acceptance and use of technology as explained by a user’s internal beliefs, attitudes, and intentions (Turner, Kitchenham, Brereton, Charters, & Budgen, 2010). Since TAM was proposed, various related research explored the topic of technology acceptance and design. This research provides the foundation of understanding and identifying a user’s technology acceptance and use of technology for various disciplines. In the field of online learning, TAM has been applied to and considered in a number of e-learning acceptance research (Šumak, Hericko, & Pušnik, 2011). According to the study by Šumak et al., (2011), TAM can effectively explain the acceptance of online learning. Consequently, this paper adopted TAM to help develop our LMS design framework. An Alternative Model of LMS Design and Development We adopted two frameworks to address the design and development of the learning management system in higher education. Transmitting Course Content Category The transmitting course content category includes three major features of an LMS: course content, announcements, and the grade book. Although the first two features are valuable, the grade book was unsuitable for complex grading or evaluation systems. Most LMSs do not offer effective grade books for instructors. Instructors adopt other techniques and tools to process more complex evaluations (Malikowski et al., 2007). This category was not as effective as the other categories in the LMS; therefore, LMS designers should consider improving these features and providing a more suitable design when building a new LMS. In other words, it would be beneficial to determine which educational theories should be applied by considering this category first. Although most of the LMSs offered features designed to support presentations and provide guidance, in practice, the components were inadequate (Gilbert & Moore, 1998; Kidney & Puckett, 2003; Ozkan, Koseler & Baykal, 2009; de Porto Alegre Muniz et al, 2012; Alhazmi & Rahman, 2012). In a study by Kumar, Gankotiya, & Dutta (2011), a comparison between Blackboard and Moodle was discussed in detail. This research classified fifty-two comprehensive features into six major categories to compare the two LMSs. Through the development of technology, Blackboard and Moodle present social and interactive features for users, such as personal blogs and Wimba classrooms (synchronous online teaching platforms). For example, Moodle incorporated social media tools (e.g., blogs) to support the diversity of the teaching and learning environment and allows students to upload their files on their blog to share with other users. Moodle also permits the instructor to share and copy the entire course with other users, or export the course shared by other instructors within the LMS. These functions can aid instructors in maintaining and developing courses. Blackboard provides options to include social media applications and other personalization tools. This personalization feature allows students to create their personal home page within Blackboard. Although the two LMSs have applied social media tools, these features are not completely integrated. Recently, researchers have proposed several studies regarding social media education, specifically utilizing social media to improve student’s learning performance (Gerlich, Browning, & Westermann, 2010). Sing and Mayer (2012) also concluded Learning and Performance Quarterly, 1(4), 2013
30
that the navigational and signaling aids in the LMS had a positive correlation between usability and recall tests. Therefore, this category was adopted in our model of the LMS development and design. Creating a Discussion Category In creating a discussion category, asynchronous discussions had higher approval ratings than synchronous discussions because participants have more time to read and respond to questions. In contrast, synchronous discussions in the LMS’s were difficult for users since the messages could be quickly scrolled away from the user’s view, and many users needed more time to focus on reading and responding to the quick discussions. In spite of this difficulty, synchronous discussions have certain benefits that could help participants feel a stronger sense of social interaction. Therefore, designing a feature that creates longer discussion formats should be considered as a viable option to provide users the opportunity to participate in discussions. A better functioning synchronous discussion feature design should decrease the difficulties in participating in synchronous discussions. Comparing Moodle and Blackboard, Kumar et al. (2011) asserted that both of LMS’s offered asynchronous and synchronous discussion functions; however, social media tools offer a better interacting approach for users (Dabbagh & Kitsanta, 2012). Traditional interaction in LMS’s are inadequate for current users and the more interactive features of LMS need to include more varied engagement approaches. Evaluating Students Category The most commonly used LMS tool in the evaluation and assessment category is the test generator, which helps instructors create course assessments and allows students to submit the test via the drop box function. The test generator allows instructors to create various types of assessments, such as multiple choice, multi-select, matching, ordering, arithmetic tests, long and short answers, fill in the blank, and true/false. In addition, the drop box function is easy to use for the student and easy to access for the instructor. A good test generator should include a flexible approach to create various evaluation methods; however, both variations of the LMS, Blackboard and Moodle, are limited in their assessment design. Fritz (2011) also emphasized the importance of interaction in the evaluation process, which would allow the learner to check and monitor their academic activities and improve student-learning performance. In contrast, simply delivering a grade without the ability to interact with the student would fail to enhance student participation, as well as their learning experience. Evaluating Courses and Instructors Category In the evaluating course and instructor category, the LMS allows students to evaluate both the course and the instructor at the end of the semester. Course evaluation serves a very important function in education because it helps instructors understand if they need to modify the course content, as well as their teaching approaches to improve the overall learning and teaching experience (Sims, Dobbs & Hand, 2002). Although this feature provides more information to institutions, it has seldom been utilized to evaluate courses by universities because most have their own evaluating systems (Malikowski, Thompson, & Theis, 2007). This feature of an LMS might not be broadly applied since colleges and universities have established evaluating systems to collect survey data. For example, the University of North Texas uses the SETE and Blackboard learning management system separately, thereby increasing maintenance costs. Learning and Performance Quarterly, 1(4), 2013
31
Developing and designing a new system needs to take this into consideration and offer alternatives which universities might be quicker to adopt. The implementation of an LMS, including a completed survey system, would enable higher education institutions to save money in both maintenance and development fees. Creating Computer-Based Instruction The use of computer-based instruction (CBI) is a feature that has been designed to help instructors transmit content for years. With the developed technology, CBI could make various options and features available in an LMS for instructors to choose from (Malikowski, Thompson, & Theis, 2007). Today, several technique tools could effectively and efficiently help instructors to create teaching content, e.g., Prezi, SlideRocket, Flickr, YouTube, Sliceshare, and Knovio. These social media tools could enhance various teaching presentations and course curriculum. Therefore, the LMS should be flexible and offer an adaptable and creative in its CBI to improve the online learning experience. Considering the literature review and consideration for the online learning frameworks, Figure 1 demonstrates a cohesive model of an LMS design and development. The HCI should consider two major factors, cross-cultural impact and the human effect, in designing an effective learning management system. Moreover, the designs of HCI, with regards to the five categories, will influence the perceived ease of use and usefulness in an LMS. Therefore, we propose a new model for LMS development and design.
Figure 1. The modified model of development and design of LMS Learning and Performance Quarterly, 1(4), 2013
32
Discussion We proposed how to consider cross-cultural impacts, the human effect, and HCI in the development for an LMS design. In this section, the characteristics of the alternative model and the implications, the interaction between HCI, the cross-cultural impact, and human effect, and limitations of the alternative model will be discussed. Characteristics of the Alternative Model The online environment provides non-linearity and hypermedia forms for the LMS users. Individuals can have access to information and knowledge without many restrictions due to the flexibility of online learning. Through interaction in the LMS, learners would be able to quickly share and gain information from other users. However, an unsuitable design of an LMS feature could result in negative impact on learning performance. For instance, learners might spend more time searching the information rather than sharing ideas or course material. In contrast, a suitable LMS design helps users minimize the barriers of utilizing the LMS to improve the learning experience. Improving the HCI, including five categories, helps the user’s perceived usefulness and ease of use would improve the intention to use the LMS (Venkatesh, Thong, & Xu, 2012). Regarding the cross-cultural impact, the LMS design becomes more complex because the crosscultural impact influences how individuals adapt to use a LMS. Moreover, human effect is also one of important factors. This is because people’s control and experience for using computer systems would influence the usability of LMS (Ku & Lohr, 2003). The well-designed HCI improve user’s perceived usefulness and perceived ease of use. Furthermore, user’s intention to use LMS presents positive results, and LMS improves student’s learning performance. The Interaction between HCI, the Cross-cultural Impact, and the Human Effect The online environment helps learners to exchange information at any time; however, the interaction between HCI, the cross-cultural impact, and human effect is complicates this learning exchange. With the cross-cultural impact and the human effect, learners have different attitudes toward using certain features of LMS’s. For example, Chinese students (collectivism culture) reported they felt isolated from others when using the discussion board. In contrast, American students (individualistic culture) did not feel isolated (Ku & Lohr, 2003). In addition, the human effect factor affects users’ perception of usefulness (Womble, 2007). The designer should be able to improve uses’ perceived usefulness by providing appropriate system design features (David, 1993). Implications and Limitations of the Alternative Model This LMS design and development study attempted to explain the human effect and the cross-cultural impact that can be emphasized in an LMS design. Moreover, the alternative model indicated the complexities of the interaction between these factors and the influence on perceived usefulness, perceived ease of use, and intention to use LMS’s. For future studies, this alternative model provides an LMS design and development along with the cultural impact and human effect. Future research should explore how to design suitable LMS’s to increase the learning performance in the LMS. From a practical aspect, this model helps instructional designers, instructors, and educational developers understand the cross-cultural impact and the human effect process of the LMS to consider the impact learning environments have on users.
Learning and Performance Quarterly, 1(4), 2013
33
Conclusions The development and design framework of a learning management system is an important foundation of learning performance and the instructional experience in higher education. The result of this paper clearly identifies the human effect and cross-cultural factors influence on LMS development and design. Based on the TAM theory and the course management model framework provided by Malikowski, Thompson, and Theis (2007), the LMS development and design needs to consider the described five categories and two factors. In this review of LMS’s for higher education institutions, much remains to be done, but we anticipate that the study will generate crucial findings in the field of LMS research. Future studies would be encouraged to test and improve upon this LMS alternative model design. We intend to continue pursuing this line of investigation in a series of studies and investigate how instructors and learners successfully utilize more LMS tools to enhance their performance. By testing the LMS framework design, we will be able to generate more specific recommendations about LMS design. With the development of LMS research, these studies can have the potential to aid LMS developers and instructional designers who support instructors and learners, and consider how to effectively use technologies to support effective learning objectives in course design.
Learning and Performance Quarterly, 1(4), 2013
34
References Abdalla, I. (2007). Evaluating effectiveness of e-blackboard system using TAM framework: A structural analysis approach. AACE Journal, 15(3), 279-287. Alhazmi, A. K., & Rahman, A. A. (2012). Why LMS failed to support student learning in higher education institutions, E-Learning, E-Management and E-Services (IS3e), 2012 IEEE Symposium,1-5. doi: 10.1109/IS3e.2012.6414943 Beckers, J. J., & Schmidt, H. G. (2003). Computer experience and computer anxiety. Computers in Human Behavior, 19, 785–797. doi:10.1016/S0747-5632(03)00005-0. Boggs, S., Shore, M., & Shore, J. A. (2004). Using e-learning platforms for mastery learning in developmental mathematics courses. Mathematics and Computer Education, 38(2), 213220. Cameron, H., & Limberger, J. (2004). Online learning: Cross-culture development in time poor environments. Journal of European Industrial Training, 28, 429-439. Retrieved from ABI/INFORM Global. (Document ID: 646658641). Chou, A., & Chou, D. (2011). Course management systems and blended learning: An innovative learning approach. Decision Sciences Journal of Innovative, 9, 463–484. doi:10.1111/j.1540-4609.2011.00325.x Chrysostomou, K., Chen, S. Y., & Liu, X. (2009). Investigation of users’ preferences in interactive multimedia learning systems: a data mining approach. Interactive Learning Environments, 17(2), 151-163. Dabbagh, N., & Kitsanta, A. (2012). Personal learning environments, social media, and selfregulated learning: A natural formula for connecting formal and informal learning. Internet and Higher Education, 15(1), 3-8. doi:10.1016/j.iheduc.2011.06.00 Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology, MIS Quarterly, 13(3), 319–340. Davis, F. (1993). User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International Journal of Man-Machine Studies, 38(3), 475–487. Retrieved from http://deepblue.lib.umich.edu/handle/2027.42/30954 De Lera, E., Fernandez, C., & Valverde, L. (2010). The emotional gap in virtual online environments. In Z. Abas et al. (Eds.), Proceedings of Global Learn Asia Pacific 2010 67-70. AACE. de Porto Alegre Muniz, M., de Moraes, A., Jacobs, K., & Soares, M.M. (2012). Usability issues in Learning Management Systems (LMS). Work, 41, 832-837. Ellis, R. K. (2009). Learning management systems 2009. Retrieved from http://www.astd.org/LC/2009/0509_LMS2009.htm Epping, R. J. (2010). Innovative use of Blackboard® to assess laboratory skills. Journal of Learning Design. 3(3). 32-36. Falvo, D. A., & Ben F., & Johnson, B. F. (2007). The use of learning management systems in the United States. TechTrends. 51(2). 40-45. Fritz, J. (2011). Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers. Internet and Higher Education, 14, 89–97. doi:10.1016/j.iheduc.2010.07.007 Gerlich, R. N., Browning, L., & Westermann, L. (2010). The social media affinity scale: Implications for education. Contemporary Issues in Education Research, 3(11), 35-41.
Learning and Performance Quarterly, 1(4), 2013
35
Gilbert, L., & Moore, D. (1998). Building interactivity in web courses: Tool for social and instructional interaction. Educational Technology, 38, 29-35. Hamuy, E. & Galaz, M. (2010). Information versus communication in course management system participation. Computers & Education, 54, 169–177. Hollender, N., Hofmann, C, Deneke, M., & Schmitz, B. (2010). Integrating cognitive load theory and concepts of human–computer interaction. Computers in Human Behavior, 26, 12781288. Huffman, H. W., & Huffman, A. H. (2012). Beyond basic study skills: The use of technology for success in college. Computers in Human Behavior, 28. 583-590. doi: 10.1016/j.chb.2011.11.004 Inversini, A., Botturi, L., & Triacca, L. (2006). Evaluating LMS usability for enhanced elearning experience. Paper presented at the EDMEDIA Conference, Orlando, Florida. ISO (1998). ISO 9241-11 Ergonomic requirements for office work with visual display terminals (VDTs) – Part 11: Guidance on usability. Retrieved from http://www.iso.org/iso/catalogue_detail.htm?csnumber=16883 Kibaru, F., & Dickson-Deane, C. (2010). Model for training usability evaluators of e-learning. Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2010 (pp. 517-522). Chesapeake, VA: AACE. Retrieved from http://www.editlib.org/p/35596. Kidney, G., & Puckett, E. (2003). Rediscovering the first principles through online learning. Quarterly Review of Distance Education, 4, 203-212. Kim, J., Kwona, Y., & Cho, D. (2011). Investigating factors that influence social presence and learning outcomes in distance higher education. Computers & Education, 57(2), 15121520. doi:10.1016/j.compedu.2011.02.005 Ku, H. Y., & Lohr, L. L. (2003). A case study of Chinese student’s attitudes toward their first online learning experience. Educational Technology Research and Development, 51(3), 95-102. doi: 10.1007/BF02504557 Kumar, S., Gankotiya, A. K., & Dutta, K. (2011). A comparative study of moodle with other elearning systems. Electronics Computer Technology (ICECT), 2011 3rd International Conference on (Vol. 5, pp. 414–418). doi:10.1109/ICECTECH.2011.5942032 Lee, L. Y., & Croker, R. (2006). A contingency model to promote the effectiveness of expatriate training. Industrial Management + Data Systems, 106(8), 1187-1205. Retrieved from ABI/INFORM Global. (Document ID: 1143355771). Lin, H. F. (2010). An application of fuzzy AHP for evaluating course website quality. Computers & Education, 54(4), 877-888. doi:10.1016/j.compedu.2009.09.017 Malikowski, S. R., Thompson, M. E., & Theis, J. G. (2007). A model for research in to course management systems: Bridge technology & learning theory. J. Educational Computing Research. 36(2), 149-173. Morgan, G. (2003). Faculty use of course management systems. EDUCAUSE. Retrieved from http://net.educause.edu/ir/library/pdf/ers0302/rs/ers0302w.pdf ON24, Inc. (2012). Is your learning management system leaving your users dazed and confused? [Whitepaper]. Retrieved from http://www.astd.org/Professional-Resources/ASTDProfessional-Partners/Professional-Partner-Content/2013/02/Is-Your-LearningManagement-System-Leaving-Your-Users-Dazed-and-Confused
Learning and Performance Quarterly, 1(4), 2013
36
Ozkan, S., Koseler, R., & Baykal, N. (2009). Evaluating learning management systems. Transforming Government: People, Process and Policy, 3(2), 111-130. doi: http://dx.doi.org/10.1108/17506160910960522 Oztekin, A., Kong, Z. J., & Uysal, O. (2010). UseLearn: A novel checklist and usability evaluation method for eLearning systems by criticality metric analysis. International Journal of Industrial Ergonomics, 40, 455-469. doi:10.1016/j.ergon.2010.04.001 Schmidt, C. T. (1997). The systemics of dialogism: on the prevalence of the self in HCI design. Journal of the American Society for Information Science, 48(11), 1073–1082. Simonson, M. (2007). Course management systems. Quarterly Review of Distance Education, 8(1), vii-ix. Sims R., Dobbs, G., & Hand, T. (2002). Enhancing quality in online learning: Scaffolding planning and design through proactive evaluation. Distance Education, 23(2), 135-148. Southwell, B.G., Anghelceva, G., Himelboima, I., & Jonesa, J. (2007). Translating user control availability into perception: The moderating role of prior experience. Computers in Human Behavior, 23(1), 554–563. Šumak, B., Hericko, M., Pušnik, M. (2011). A meta-analysis of e-learning technology acceptance: The role of user types and e-learning technology types. Computers in Human Behavior, 27, 2067–2077. Sung, E., & Mayer, R. (2012). Affective impact of navigational and signaling aids to e-learning. Computers in Human Behavior, 28, 473-483. doi: 10.1016/j.chb.2011.10.019 Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251-296. Tsang, P., Kwan, R., & Fox, R. (2007). Enhancing learning through technology. World Scientific Publishing Co. Hong Kong. Tsao, J. Z., Jang, S. T., & Chen, W. C. (2007). Research of university students' learning satisfaction of physical education- A case study on Chung Yuan Christian University. Journal of Taipei Physical Education. 15, 322-333. Tselios, N., Avouris, N., & Komis, V. (2008). The effective combination of hybrid usability methods in evaluating educational applications of ICT: Issues and challenges. Education and Information Technologies, 13(1), 55–76. doi:10.1007/s10639-007-9045-5 Turner, M., Kitchenham, B., Brereton, P., Charters, S., & Budgen, D. (2010). Does the technology acceptance model predict actual use? A systematic literature review. Information and Software Technology, 52, 463–479. Unal, Z., & Unal, A. (2011). Evaluating and comparing the usability of web-based course management systems. Journal of Information Technology Education, 1019-1038. Venkatesh, V., Thong, J. Y. L., & Xu, X. (2012). Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology. MIS Quarterly, 36(1), 157–178. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2002388 Womble, J. (2007). E-learning: The relationship among learner satisfaction, self-efficacy, and usefulness. The Business Review, Cambridge, 10(1), 182–189. Yang, D., Olesova, L., & Richardson, J. C. (2010). Impact of cultural differences on students' participation, communication, and learning in an online environment. Journal of Educational Computing Research, 43(2), 165-182. Zhang, C.X. (2004). Educational Psychology. Taipei: Tunghua Books Co.
Learning and Performance Quarterly, 1(4), 2013
37
