Critical issues for elearning delivery: what may ... - Wiley Online Library

Original article doi: 10.1111/j.1365-2729.2008.00281.x

Critical issues for e-learning delivery: what may seem obvious is not always put into practice M.A. McPherson* & J.M. Nunes† *School of Education (ECS 7.65), University of Leeds, Leeds, UK †University of Sheffield, Department of Information Studies, Regent Court, 211 Portobello Street, UK

Abstract

The successful adoption of information and communication technology to enhance learning can be very challenging, requiring a complex blend of technological, pedagogical and organizational components, which may at times require the resolution of contradictory demands and conflicting needs. The research reported in this paper investigated and analysed critical success factors (CSFs) that are required to deliver e-learning within higher education (HE) courses and programmes. The research design adopted a critical research approach, instantiated by focus group discussions with e-learning experts drawn from administrative, educational, technology and research domains. The findings revealed that staffing issues, pedagogically sound delivery models and training of both tutors and students cannot be treated as trivial issues and are critical to the success of e-learning. Furthermore, this research also shows that there is a strong relationship between these factors and inspirational institutional leadership. The findings also suggest that in order to assure the success of e-learning, this leadership should guarantee the presence of institutional enablers. It is hoped that the CSFs, described and discussed in depth in this paper, will provide a suitable theoretical foundation to underpin the successful delivery of e-learning within HE.

Keywords

critical success factors, e-learning course delivery, e-learning course co-ordination, e-tutoring, higher education, online learning.

Introduction

Most universities are being compelled to adopt technology to support learning. Even where academic staff are not wholly convinced by pedagogical arguments, institutions are still doing this, either for reasons of social pressure or in order to look modern and progressive. In doing so, many traditional universities have had a tendency to subsume e-learning activities within the strategies and resources of the broader campus-based remit (Cornford & Pollock 2002, p. 30).

Accepted: 25 February 2008 Correspondence: Maggie McPherson, School of Education, University of Leeds, Leeds, LS2 9JT, UK. Email: [email protected]

© 2008 The Authors. Journal compilation © 2008 Blackwell Publishing Ltd

For those academics and learning technologists involved in implementing e-learning projects, which inherently fall outside this traditional view of education, this poses a major problem. In fact, delivery of e-learning implies much more than a simple technical exercise in which some materials or processes are simply transferred from the offline world to some readymade online realm (Cornford & Pollock 2002, p. 12). Duke (2002) proposes that this approach calls for more in terms of pedagogy than simply ‘putting professors’ lectures onto the web’. McPherson and Nunes (2004a) argue that the adoption of these new pedagogical models implies a whole new set of both teaching and learning skills and as a consequence, tutors and learners may not necessarily be sufficiently prepared to be successful in e-learning environments.

Journal of Computer Assisted Learning (2008), 24, 433–445

433

434

Furthermore, as e-learning often starts as small-scale departmental projects (Robinson 2001), the implementation of e-learning faces a high level of risk because of its uncertain status and the unfamiliarity of staff and students with these new environments. These efforts have little chance of succeeding without a tutoring team that has appropriate online tutoring skills necessary to explore and maximize the designed environments (De Laat & Lally 2003). Therefore, the tutoring team is at least as important as the technical design team. Furthermore, both parties need to be aware of appropriate pedagogical approaches in order to maximize the benefits of tutoring and the use of learning environments by students (Whitworth 2006). This paper reports on a research project that was designed to take a critical research approach aimed at identifying critical success factors (CSFs) related to e-learning strategies, implementation, design and delivery. This paper discusses CSFs related to the delivery of e-learning and its influence and impact on the both tutor and learner experiences and their perceptions of success. These CSFs were derived through a holistic, consultative and emancipatory research process. Bearing this in mind and the necessity of carrying out research within such a complex organizational setting, we decided to draw inspiration from a generic management theory that suggests that the identification of sets of factors that are critical to successful change management is fundamental (Huotari & Wilson 2001) The research design Research question and objectives

The general aims of this research are framed by the following overarching research question: What are the underlying CFSs required to support the delivery of e-learning within higher education (HE) institutions?

CSFs as a generic concept

In actual fact, the notion of isolating critical factors as a guide for business success was first introduced by Daniel (1961), but was overlooked until Rockhart (1979) reintroduced and further developed this concept. According to Rockhart (1979), CSFs are the limited number of areas in which satisfactory results will ensure

M.A. McPherson & J.M. Nunes

successful competitive performance for the individual, department or organisation. More recently, Robson (1997) drew attention to CSFs as ‘those handful of things that within someone’s job must go right’ in order to achieve success. This implies that these are factors that practitioners and managers should keep a firm eye on. In the case of this research, CSFs emerge from the evaluation of factors in the limited sphere of e-learning, rather than placing a wider focus on the key performance indicators of HE programme provision in its entirety (McPherson & Nunes 2004b). CSF analysis in e-learning

CSF analysis, as proposed by Rockhart (1979), is a widely used top-down research framework. This type of analysis has been frequently used in investigating factors that affect informational and technological change. However, as this research concentrates on the CSFs for e-learning, particularly within HE, it is necessary to also take into consideration that this is essentially a human activity system that takes place within an educational environment. This research framework is ideally suited as a means to establish management information requirements and to define information to be managed. However, in this research it was used to identify the crucial factors that must be addressed for e-learning to do well (McPherson & Nunes 2004b). The CSF analysis described in this paper started with a characterization of five fundamental aspects of e-learning: organizational; technological; curriculum design; instructional design; and finally e-learning course delivery. This characterization emerged from an exhaustive literature review (McPherson & Nunes 2004a). As required by CSF identification and analysis methodologies, these elements are represented in the conceptual framework shown in Fig 1. This allowed further exploration and discovery of those elements that are crucial to the success of e-learning within each of the main categories. This paper focuses primarily on the ‘delivery’ component of this conceptual framework. The other components of the framework were reported in other research publications, which were as follows: organizational (McPherson & Nunes 2006a), technological (McPherson & Nunes 2006b), curriculum development (McPherson & Nunes © 2008 The Authors. Journal compilation © 2008 Blackwell Publishing Ltd

Critical issues for e-learning delivery

435

High

Organizational Setting

•

Fig 1 A framework for the study of e-learning. CMC, computer mediated communication; ICT, information and communication technology; VLE, virtual learning environment.

Organizational Knowledge

Technical Knowledge

Technological Infrastructure

•

Technical Experts - VLEs - CMC - Teaching & Learning Software

Curriculum Development

• • •

Academic Staff Educational Specialists Subject Matter Specialists

Instructional Systems Design • Academic Staff • ICT Specialists • Educational Specialists

Practical Knowledge

Delivery

• • •

Low

Organizational Factors

•

Organizational Strategy and Policy-makers Management and Administrators

Low

2007a) and instructional systems design (McPherson & Nunes 2007b). Eliciting CSFs in e-learning using a focus group interview approach Methodological concerns

Critical success factor analysis as a generic framework requires specific approaches for data collection and data analysis. Critical research was chosen as the approach for this study, since it provided the necessary holistic, consultative and emancipatory perspective that was being sought. As proposed by Schwandt (2006), critical research is an approach that aims at revealing values and beliefs as key constituents of the theoretical propositions that are made about human activity systems. In line with his thinking, the authors are of the opinion that ‘technical and one-dimensional forms of reason have subverted and eclipsed critical reasoning’ and decision making about the use, value and appropriateness of information systems adoption. This is particularly true when discussing e-learning, as corroborated by Sanders (2006), who feels that the adoption of critical research can be used as ‘a means to counterbalance the onslaught of advertising rhetoric extolling the supposed virtues of instructional technology’. Consequently, it was felt that a case-study approach centring on one HE institution would probably not enable breaking away from individual organizational culture, policies and ideologies. For a truly liberating © 2008 The Authors. Journal compilation © 2008 Blackwell Publishing Ltd

Stakeholder Involvement

Academic Staff Researchers Tutors

High

result to be achieved, it was felt a broader-based consultative approach was needed, i.e. one that would bring together different e-learning stakeholders from a variety of different HE institutions and backgrounds. Having reached the decision to adopt a wide-ranging consultative approach, the question of the data collection method emerged as the next hurdle. Survey methods or in loco individual interview approaches (e.g. grounded theory) were considered, but rejected. It was assumed that response to individual in loco inquiry would possibly not allow for cognitive conflict, breaking with principles of current daily practice and freedom from individual institutional policies that are necessary for a truly transformative process. Conversely, it was thought that in order to elicit this transformative knowledge it would be necessary to bring together practitioners, researchers, administrators and technologists in a neutral environment for the discussion and social negotiation of CSFs. Therefore, a co-operative inquiry approach, based on focus group interviews, was selected because this offered a unique and comprehensive form of participative research in which e-learning stakeholders could use the full range of their sensibilities, knowledge and experiences to discuss and negotiate the different understanding and aspects of this new approach to learning (McPherson & Nunes 2004b). As proposed by Heron (1996), this type of inquiry was thought to be an ideal way of doing research with people, where the roles of researcher and subject are integrated.

436

Focus group studies are typical co-operative data collection methods. As first proposed by Merton et al. (1956/1990), these are ideal when the research is trying to understand diverse perspectives held by different groups engaged in a particular process. Given that e-learning initiatives involve a number of individuals in distinctive educational roles (i.e. management, IT staff, academics and instructional designers), it was felt that focus groups would provide good support to elicit CSFs. Group forces or dynamics become an integral part of the procedure with participants engaged in discussion with each other rather than directing their comments solely to the moderator (Catterall & Maclaran 1997). Thus, a focus group is, in essence, a semistructured interview in which a moderator keeps the direction of discussions under control by utilizing a preset list of questions or script. There is no attempt in a focus group to reach consensus, define a majority or correct opinions on any topic (Krueger 1994). However, clarity of group purpose is key (Saulnier 2000), and while the facilitator must endeavour to create a permissive environment in which different perceptions and points of view are freely expressed, some degree of purpose and an accepted pre-understanding of the identified topic are required (McPherson & Nunes 2004b). It is not possible to discuss or negotiate a complex phenomenon like e-learning, if there is not some pre-understanding already commonly accepted to some degree. This preunderstanding coupled with the systematic processes of facilitated group dynamics and negotiation of meanings, enables deep understanding of the data (Franklin & Bloor 1999). Furthermore, ‘all the participants must have the opportunity to know what they are doing’ so that they can form a ‘common will discursively’ (Habermas 1973, p. 37). In terms of this research, the preunderstanding was established through an initial presentation by the facilitator(s) using the e-learning framework presented in Fig 1. Establishing this pre-understanding through this framework was therefore the first objective of this research, as presented above. Having established the general research approach (critical research), having selected the specific consultative method to implement that approach (co-operative inquiry) and the data collection method (focus group interviews), the final step in establishing a research


methodology for this specific project was to engage in a careful research design. Research procedure: data collection through focus group interviews

The first step in the research design for this project was to establish the settings and conditions under which the focus group interviews were to be conducted. Since a critical research approach had been adopted, it was necessary to ensure that the setting of these group interviews was conducive to socially negotiated, transformative and emancipatory outcomes. Thus, a neutral environment that was both informal and unconstrained by institutional values had to be found. The initial thought of conducting focus group interviews in selected HE institutions was rejected. Such group discussions would necessarily be biased and constrained by the organizational structure, culture and policies. After all, ‘organizational questions are not primary things’ (Habermas 1973, p. 36) and a more holistic view of e-learning within HE was being sought. After much consideration and debate, it was decided to conduct these focus group interviews with approximately 15–20 participants in four separate workshops at e-learning/educational technology conferences (Association of University Administrators [AUA] 2002, E-Learn 2002, International Conference on Advanced Learning Technologies [ICALT] 2002 and International Conference on Computers in Education [ICCE] 2002). In order to provide the pre-understanding basis discussed above, the e-learning framework, presented in Fig 1 was discussed with the participants. This represented a first theoretical formulation and the initial foundation for the understanding and proposal of the CSFs for e-learning by the participants. The findings presented in this paper emerged from these understandings and were both expanded and deepened by iterative data analysis from the different focus group interviews. Given that during the data collection period real and practice-led e-learning expertise in HE were still confined to a relatively small number of educators, the best way of bringing these experts together was seen as being through specifically targeted workshops within academic conferences. Although the study was initiated within an English HE institution, it was decided not to limit data collection to UK-based conferences. This © 2008 The Authors. Journal compilation © 2008 Blackwell Publishing Ltd


Research Question Literature Review Formulation of e-Learning Theoretical Framework Understanding and Establishing CSFs for e-Learning Conducting Focus Groups through Workshops at Targeted Conferences Data Collection Data Analysis Theory Extension Fig 2 Research design. CSFs, critical success factors.

decision aimed at allowing the collection of a wider set of opinions and expertise so that the findings could be more emancipatory and unprejudiced. It was considered that the majority of e-learning problems and opportunities facing educators internationally would not be substantially different from those faced by colleagues in the UK. As the knowledge being sought was to have an emancipatory nature, it was thought that having participants from different cultural backgrounds, as well as having a varied institutional and academic ethos, would actually be beneficial for the project. Since the main stakeholders in e-learning are educational practitioners, researchers, administrators and technologists, it was decided to select particular conferences that would be likely to attract participants from each of these groups. The strategy for data collection and exploration of participants’ views through group discussions around the issues relating to each of the five areas is outlined in the e-learning framework. At the end of each of the focus group sessions, participants were given time to consider the group discussions and to form their own views of CSFs related to each aspect of e-learning conceptual framework. Each participant was provided with a tabular form on which to record their own top five preferred CSFs in each of the five categories. A final plenary discussion, moderated by the researchers, was then scheduled to invite and encourage further discussion and the gathering of any additional comments. The research design, represented in Fig 2, was successfully used to guide and support this study. © 2008 The Authors. Journal compilation © 2008 Blackwell Publishing Ltd

437

Research procedure: analysis of focus group interview responses

The questions posed to focus group interviewees were totally open-ended, aimed at enabling total freedom of expression and individual formulation of opinion of their principal CSFs, within each category of the conceptual e-learning framework. The results from participants at all the research workshops (AUA 2002 = 15; E-Learn 2002 = 17; ICALT 2002 = 22; and ICCE 2002 = 20, i.e. 75 respondents in total) were gathered through an open-ended structured questionnaire that were filled in by each individual participant at the end of one of the focus group interview sessions as outlined earlier. In this paper, the category of the e-learning framework (Fig 1) being discussed is that of e-learning delivery. Because this research is essentially qualitative and exploratory, the method used to identify the principal e-learning CSF categories in this category was initially identified through thematic analysis (Onwuegbuzie 2003). The concept of selective coding used in this research was adapted from grounded theory as proposed by (Strauss & Corbin 1990). This does not imply that this is a grounded theory study that involves concurrent data collection and analysis; it merely means that the concept of selective coding was used in order to interpret and understand data in the open-ended responses. Selective coding involves the integration of categories that have been developed to form the initial theoretical framework (Pandit 1996). The coding was used to identify the properties, conditions and relationships between the emerging concepts and categories at each stage of data collection (Dearnley 2003). This process of selective coding implies first choosing one category to be the core category and then link to all other related concepts to that category. In this case, the core category is delivery of e-learning. Once the e-learning delivery CSFs were identified, they were then grouped in clusters related to this main category. In this context, in the core CSFs of e-learning delivery, a cluster is a subset of CSFs, within the overall universe of e-learning, which are closely related to one another and relatively far and separated from other CSFs (McPherson & Nunes 2006a). In fact, after merging the data from all workshops, initial analysis clearly showed that CSFs are not neatly bounded by the five aspects of e-learning listed in Fig 1 and it became increasingly apparent that the results of

438

the selective coding process of CSFs for instructional systems design (ISD) would be best represented using an ontology as defined below: An ontology defines a common vocabulary for researchers who need to share information in a domain. It includes [. . .] interpretable definitions of basic concepts in the domain and relations among them. (Noy & McGuinness 2001)

Furthermore, as stated by McPherson and Nunes (2006b), ontologies are often developed in order to allow for: • Sharing common understanding of the structure of a particular subject domain. • Enabling reuse of domain knowledge. • Making explicit domain assumptions underlying a change process. • Separating the general domain knowledge from operational knowledge. • Analysing domain knowledge and establishing relationships with other ontologies or reuse existing ones by extending them. In the case of this analysis, iterative selective coding was used to create a CSF ontology that is hoped can be of use for both academics and practitioners to devise e-learning delivery strategies. The purpose of this process was to identify similarities within the professional practice that emerges from e-learning (McPherson & Nunes 2006a). Thus, the e-learning delivery CSFs as presented in this paper were initially characterized through thematic analysis and sets of related CSFs brought together by the interviewees from across organisational boundaries into a grouping that makes sense to them as practitioners were identified and finally represented in ontology.

Discussion of findings emerging from e-learning CSF ontology for delivery issues

In analyzing the responses from interviewees, it became apparent that despite having been asked to specifically list what they considered to be CSFs for e-learning delivery, from time to time respondents digressed into other areas of the conceptual framework. Themes emerged around these other aspects of the framework, such as organizational, technical and curriculum design issues.


Occasionally, by rearranging words with different emphases, an interviewee referred to one CSF more than once in his/her response. Therefore, the process of clustering used in this study is not a quantitative one, such as that used by other authors in similar CSF studies, e.g. Wopereis et al. (2005), but resulted from thematic qualitative coding approach as discussed above. Four main CSFs were identified within the resultant e-learning delivery ontology relating to: staffing, delivery models, training and leadership issues; and the subthemes identified have been rephrased as questions. This CSF ontology for the category of e-learning delivery is intended to help researchers understand how individual educationalists and developers construe approaches for transforming curriculum design into a format that would be suitable for e-learning delivery. A visual representation of the main themes emerging from the e-learning delivery CSF ontology can be seen in the diagram shown in Fig 3 below. This then provides a synopsis of the key delivery CSFs and their associated subthemes. Determine attributes, experience and availability of staff and students

As discussed by McPherson and Nunes (2004a), tutoring in e-learning environments has been widely considered as a crucial factor in the success of computermediated collaborative learning activities. Thus, staffing issues has been selected as the first of the four categories and is presented in Table 1 below: Different and alternative names have been used in the literature referring to the role of the tutor in online interaction, such as coach (Murphy et al. 1998), leader (Hotte & Pierre 2002), tutor (Gerrard 2002), moderator (Feenberg 1986; Kerr 1986; Berge 1995; Salmon 2000), facilitator (Collison et al. 2000), motivator, mentor, mediator and even production coordinator (English & Yazdani 1999). However, although in many respects skills needed for such tutoring activities are similar to face-to-face (f2f) delivery, in the reality of practice, e-tutoring also differs in a number of ways (Gerrard 2002; McPherson and Nunes 2004a). Thus, it is not surprising to find that respondents consider that specific staff attributes and experience were critical to the success of e-learning. Moreover, as discussed by a number of authors, e-learning courses can be much more time-consuming © 2008 The Authors. Journal compilation © 2008 Blackwell Publishing Ltd


439

Adopt an Appropriate Pedagogical Model

Establish student attributes, experience and preparedness for e-learning

Ascertain suitability of modes

Guarantee suitable academic staff availability

Verify required staff attributes and experience

Determine Attributes, Experience and Availability (of both staff and students)

Implement Relevant Delivery Model

Address Training Requirements

Provide Inspirational Leadership

Identify needs and provide staff training

Adopt Appropriate Evaluation Approach

Realize Agreed Strategy

Understand Motivation for Engagement

Provide Student Support

Ensure Sufficient Resourcing

Pay attention to technical issues

Fig 3 Synopsis of e-learning critical success factors for e-learning delivery. Table 1. Critical success factor ontology for e-learning delivery: staffing issues. Address training requirements Verify required staff attributes and experience Check tutor experience of e-learning delivery

Guarantee suitable academic staff availability Ensure availability of tutors/instructors

Establish student attributes, experience and preparedness for e-learning Verify students’ computer literacy; independent learning abilities; motivation

Examples of issues for consideration

Are personnel prepared? Do tutors have facilitation skills? Do tutors have sufficient technical expertise and information and communication technology confidence? Are tutors being consulted about suitable design and development? Are teachers prepared for future roles; i.e. to act as coaches, tutors, mentors, content producers, facilitators, researchers, etc.? Are mentors available for online chats, etc., and are local mentors available for (distance) international students? What delivery roles are required? Are e-tutors congenial?

Are student skills sufficient? Are they computer literate? Do they have suitable learning styles? Are they independent learners? Are they self-motivated?

than their f2f counterparts, both in terms of preparation and delivery (Bernath & Rubin 2001, p. 221; Buy 2001; Strauss 2001; Barker 2002). According to Tinker (2001), the first time an online course is taught, it is common for tutors to spend 40%–50% more than their equivalent f2f colleagues. Also in this case, respondents seem to be quite aware of this problem; and the need to © 2008 The Authors. Journal compilation © 2008 Blackwell Publishing Ltd

ensure that suitable staff should be available and be given time to develop their experience and skills in this type of tutoring. However, and as discussed by McPherson and Nunes (2004a), it is not enough that tutors are prepared for online learning, the learners also need preparation. Due to the hype associated with online learning, learners

440


Table 2. Critical success factor (CSF) ontology for e-learning delivery: delivery models. Implement relevant delivery model Adopt an appropriate pedagogical model Pedagogical model should emerge from faculty

Ascertain suitability of modes Create an interactive online environment

Adopt appropriate evaluation approach Decide on relevant pedagogical evaluation and assessment


Can a variety of learning and teaching methods be used, e.g. learner-centred? Are there sufficient opportunities for interaction, collaboration, sharing results? Will it enhance performance? Can group meetings be facilitated (community of learners)? Will it be possible for students act as tutors? Can student convenience be accommodated (can students be on-campus if desired)? Are students being accorded the respect they deserve? Has the value that information and communication technology affords study been determined? Has the question of fit been determined and have at risk students been identified? Has the question of online vs. blended been fully considered – how much should be online/f2f/compact disc? Will e-learning be offered alongside traditional learning methodologies? To what extent should synchronous or asynchronous communication/discussion forums be used? Is there a good balance of f2f and virtual environments? Is mode suitable for home/school/on a global basis? Will regular face to face be provided where desirable and possible? Should there be local f2f tutors where courses are offered internationally? Will content be updated in timely way? Can achievement of CSFs be judged in relation to previously determined strategic goals? Can iterative evaluation on process for continuous improvement be implemented? How will quality of delivery be determined? What forms of evaluation are apt? Will pre-and post-tests be used? Can synchronous interaction be used to offer immediate feedback? Can feedback be drawn on to solve problems? Will levels of satisfaction be assessed, i.e. will students be given a means to ask about the system and complain?

f2f, face-to-face.

often feel compelled to engage with these new environments, without being properly equipped with the basic skills required to be successful (Nunes et al. 2000). These skills are not only required to succeed in the online learning environment to which learners are exposed, but are also an essential part of all aspects of daily networked activity. Respondents, clearly aware of this, identified the necessity to establish student attributes, experience and preparedness for e-learning, and if necessary take mitigation actions to train and prepare cohorts for e-learning.

Implement relevant delivery model

Systems and environments to support e-learning require detailed specification of learning needs, materials, activities and delivery methods and needs. This has been selected as the second CSF ontology relating to e-learning delivery models presented in Table 2. In fact, the complexity of integration of the different information and communication technology components according to these learning needs requires a sound pedagogical model (Nunes 2003). These early © 2008 The Authors. Journal compilation © 2008 Blackwell Publishing Ltd


441

Table 3. Critical success factor ontology for e-learning delivery: training issues. Address training requirements Identify needs and provide staff training Provide essential training for all levels of staff

Provide student support Make sure technological, pedagogical and personal learner support is available

Pay attention to technical issues Provide appropriate infrastructure

Ensure access to appropriate PC technology with suitable e-learning software and communication tools

Ensure adaptability (tailored to learner’s aptitude), customizability and usability of e-learning systems


Can adequate training/education of lecturers teaching courses be assured? Is staff development/user training and ongoing support in place for stakeholders? Is training offered for tutors to become mentors, conference moderators, online teachers and facilitators? Is technical and administrative support for lecturers in place? Can academic staff and tutors be trained in using the technology? Are courses such as European Computer Driving Licence in place for basic level of technological training? Are other short appropriate e-learning courses offered? Is there a supportive environment (help when needed)? Are all staff (academics educational specialists, IT support, researchers and students) involved? Have students’ user needs and/or individual learner requirements been understood? What student training needs to be provided? Has attention been paid to the intangibles (social, support, etc.)? Has user technical support mechanisms (e.g. helpdesk, FAQs) been set up? What campus services are available? Is there timely support for enrolment? Are infrastructure and communication systems robust and reliable? Are they effective and sustainable? Are IT systems sufficient seamless, transparent and ordered? Are systems reliable, available and serviceable? Are distribution systems fast enough? Is there a process of developmental testing in place? If on-campus, have sufficient labs and computers with up-to-date features, (like good Internet connections), printers been provided? If off-site, do all students have access to necessary technology? Are common and (standard) ways for delivering the content (e.g. web-based) in use? Are design and development tools project-based? Are all design, development and tutoring staff aware of the necessity to provide accessible e-learning? Have users have been consulted to ensure usability and ease of use? Are attractive interfaces been offered to encourage student use? Can systems be adapted to needs of learners? Are users are getting what they want?

FAQs, frequently asked questions; IT, information technology.

models then influence and determine delivery models and tutoring activities, and respondents identified selection and application of suitable pedagogical models as a core CSF. Furthermore, participants proposed that it is then important to adopt suitable and compatible modes of communication and tutoring. In addition to this, the participants emphasized the need © 2008 The Authors. Journal compilation © 2008 Blackwell Publishing Ltd

for compatible and appropriate evaluation and assessment approaches. Support and training

According to discussion above, the need for new tutoring and learning skills in e-learning environ-

442


Table 4. Critical success factor ontology for e-learning delivery: leadership issues. Provide inspirational leadership


Realize agreed strategy Involve staff in change processes Focus on changing role of educational professionals

Have issues of ownership and Internet protocol been clarified? Have issues of culture/class/gender been resolved? Opens up options for students but may be threatening to tutors – could a slow and gradual transition be put in place? Is it possible to encourage a culture of open and evolving commitment? Have motivational factors of the educational staff been determined? Is there a way to acknowledge dedication of teaching staff? Is motivation of virtual learning environment providers and developers same as delivery staff? Are there incentives for the application of an e-learning framework? Has it been determined whether students are sufficiently independent and motivated to able to undertake computer-based learning? Can students see the benefits? Where e-learning is deemed desirable, are targets and customers well-defined? What are motivational factors of learners i.e. rewards for learners? Can academic staff be convinced that e-learning will work, i.e. do they have a wish to use and develop new tools? How are teaching staff going to use it? Do student users want it and will they use it?

Understand motivation for engagement Offer recognition for staff commitment

Appreciate motivation of learners

Understand what is deemed acceptable and usable

Ensure sufficient resourcing Create (or at least measure) the demand for e-learning as a method of learning Guarantee sufficient funding

Can teaching staff be persuaded of the need for convergence and flexibility to enhance students learning experiences? Can a move away from expectation of two lectures, one lab, one tutorial, etc., per week be encouraged? Have issues of affordability and viability been determined? Has time resourcing, e.g. more time to teach online, been taken into account?

ments requires the provision of appropriate programmes of continuing professional development for staff and e-skills for students. This is represented in Table 3. In this table, respondents clearly recognize the necessity for systematic identification of training needs through programmes of staff review and development and see this as a crucial CSF. Additionally, the use of new technological tools and artefacts for teaching and learning require both technical support and pedagogical support by central support units. Respondents identify the quality and experience of supporters as an important CSF. Provide inspirational leadership

Over the last two decades, political and social changes have had a profound effect on teaching and learning within HE institutions, and the role of academic leadership is to balance these demands and guide

institutions through development of sound strategic change (McPherson 2003). Since all formal e-learning programmes exist within an organizational context, such as universities, corporations or virtual learning institutes, it is clear that leadership has the power to facilitate, influence or even impede the development of e-learning (McPherson 2003). This then is the last of the four e-learning delivery CSFs and Table 4 overleaf sets out the participants’ views of this crucial aspect for successful delivery of e-learning. In fact, respondents made vociferous claims that if top-down strategies are devised to implement e-learning strategies, then it is the duty of leadership to ensure that appropriate levels of staffing and support are put in place. Without these, the sustainability of e-learning cannot be guaranteed. Furthermore, along this line of reasoning, responses from participants confirmed recent claims by Jameson et al. (2006) that there © 2008 The Authors. Journal compilation © 2008 Blackwell Publishing Ltd


is a need for agreed strategies that result from the dialogue between top-down e-learning strategies and bottom-up innovation and creativity. Finally, respondents also advocated that motivational issues must be tackled through interdisciplinary collaboration, good communication, trust and creativity, thereby confirming proposals by authors such as Vaaland (2004) and Jameson et al. (2006). Conclusions Since the days of Socrates, teachers have known that it is one thing to bring students to acquire certain ways of acting – be it kicking a football, performing a multiplication algorithm, or the reciting of verbal expressions – but quite another to engender understanding. The one enterprise could be called ‘training’, the other ‘teaching’, but educators, who are often better at the first than at the second, do not always want to maintain the distinction. Consequently, the methods for attaining the two goals tend to be confused. In both, communication plays a considerable part, but what is intended by ‘communication’ is not quite the same. (Von Glasersfeld 1989)

The concern with appropriate pedagogical models and modes of communication may have been a constant concern since the times of Socrates, as proposed by Glasersfeld (1989). However, at no time since Socrates have these modes of communication changed as dramatically as nowadays, because of the advent of the World Wide Web. E-learning delivery has caught many tutors and learners by surprise and it may take some time for all HE stakeholders involved to fully appreciate the benefits and complexities added by the ‘e’. The CSFs identified in this research are clearly centred on the anxieties of academics and practitioners with this new learning paradigm. This indicates a healthy cautious attitude in the face of change, while maintaining a very pragmatic commitment towards e-learning.

References Barker P. (2002) On being an online tutor. Innovations in Education and Teaching International 39, 3–13. Berge, Z.L. (1995) Facilitating computer conferencing: recommendations from the field. Educational Technology 35, 22–30. Available at: http://www.emoderators.com/ moderators/teach_online.html (last accessed 3 February 2007).


443

Bernath U. & Rubin E. (2001) Professional development in distance education – a successful experiment and future directions. In Innovations in Open and Distance Learning – Successful Development of Online and Web-Based Learning (eds F. Lockwood & A. Gooley), pp. 213–223. Kogan Page, London. Buy U. (2001) Debunking some common misconceptions on e-learning. Proceedings of the 25th Annual International Computer Software and Applications Conference (COMPSAC’01). Available at: http://csdl2.computer.org/ persagen/DLAbsToc.jsp?resourcePath=/dl/proceedings/ &toc=comp/proceedings/compsac/2001/1372/00/1372toc. xml# (last accessed 3 March 2007). Catterall, M. & Maclaran, P. (1997) Focus group data and qualitative analysis programs: coding the moving picture as well as the snapshots. Sociological Research Online 2. Available at: http://www.socresonline.org.uk/2/1/6.html (last accessed 8 January 2007). Collison G., Elbaum B., Haavind S. & Tinker R. (2000) Facilitating Online Learning: Effective Strategies for Moderators. Atwood Publishing, Madison, WI. Cornford J. & Pollock N. (2002) Putting the University Online: Information, Technology and Organisational Change. SRHE and Open University Press, Buckingham. Daniel D.R. (1961) Management information crisis. Harvard Business Review 11, 111–121. De Laat M. & Lally V. (2003) Complexity, theory and praxis: researching collaborative learning and tutoring processes in a networked learning community. Instructional Science 31, 7–39. Dearnley C. (2003) Student support in open learning: sustaining the process. International Review of Research in Open and Distance Learning 4, 1492–3831. Duke C. (2002) Managing the Learning University. SRHE and Open University Press, Buckingham. English S. & Yazdani M. (1999) Computer supported cooperative learning in a virtual university. Journal of Computer Assisted Learning 15, 2–13. Feenberg A. (1986) Network design: an operational manual for computer conferencing. IEEE Transactions on Professional Communications 29, 2–7. Franklin J. & Bloor M. (1999) Some issues arising in the systematic analysis of focus group materials. In Developing Focus Group Research (eds R. Barbour & J. Kitzinger), pp. 144–155. Sage, Thousand Oaks, CA. Gerrard C. (2002) Promoting best practice for e-tutoring through staff development. Proceedings of Networked Learning 2002: Third International Conference, Lancaster University and University of Sheffield, 26–28 March 2002 (eds D. McConnell & S. Banks). Available at: http:// www.networkedlearningconference.org.uk/past/nlc2002/

444

proceedings/papers/15.htm (last accessed 30 December 2006). Habermas J. (1973) Theory and Practice (trans. J. Viertel). Beacon Press, Boston, MA. Heron J. (1996) Co-operative Inquiry: Research into the Human Condition. Sage Publications, London. Hotte, R. and Pierre, S. (2002) Leadership and conflict management support in a cooperative tele-learning environment. International Journal on e-Learning 1, 46–59. Available at: http://www.editlib.org/index.cfm?fuseaction= Reader.ViewAbstract&paper_id=15103 (last accessed 30 December 2006). Huotari, M.L. and Wilson, T.D. (2001) Determining organizational information needs: the critical success factors approach. Information Research 6. Available at: http:// InformationR.net/ir/6-3/paper108.html (last accessed 30 December 2006). Jameson J., Ferrell G., Kelly J., Walker S. & Ryan M. (2006) Building trust and shared knowledge in communities of e-learning practice: collaborative leadership in the JISC eLISA and CAMEL lifelong learning projects. British Journal of Educational Technology 37, 949–967. Kerr E.B. (1986) Electronic leadership: a guide to moderating on-line conferences. IEEE Transactions on Professional Communication 29, 12–18. Krueger R. (1994) Focus Groups: A Practical Guide for Applied Research. Sage Publications, Thousand Oaks, CA. McPherson M.A. (2003) Planning for success in eLearning in HE: a strategic view. In Proceedings of the 2nd International Conference on Emerging Telecommunications Technologies and Applications and the 4th Conference on Virtual University (ICETA 2003), 11–13 September 2003 (eds F. Jakab & A. Cižmár), pp. 449–452. Technical University of Košice, Košice, Slovak Republic. McPherson M.A. & Nunes J.M. (2004a) Developing Innovation in Online Learning: An Action Research Framework. RoutledgeFalmer, London. McPherson M.A. & Nunes J.M. (2004b) Critical research using focus group interviews: an approach to elicit critical success factors in eLearning. In Proceedings of the 3rd European Conference on Research Methodology for Business and Management Studies (eds. A. Brown & D. Remenyi), pp. 263–272. Reading University, Berkshire. McPherson M.A. & Nunes J.M. (2006a) Organisational issues for e-Learning: critical success factors as identified by HE practitioners. International Journal for Educational Management 20, 542–558. McPherson M.A. & Nunes J.M. (2006b) Flying high or crash landing? Technological critical success factors for e-Learning. Proceedings of the 1st Conference on Sup-


ported Online Learning for Students using Technology for Information and Communication in their Education (SOLSTICE 2006), 3 May 2006. Edge Hill University, Ormskirk, Lancashire. Available at: http://www.edgehill. ac.uk/SOLSTICE/Conference2006/documents/40.pdf (last accessed 3 February 2007). McPherson M.A. & Nunes J.M. (2007a) Kindling a passion for acquiring new knowledge: critical success factors for creating appropriate curricula for e-Learning. In Proceedings of the ED-MEDIA 2007: World Conference on Educational Multimedia, Hypermedia & Telecommunications, 25–29 June (eds C. Montgomerie & J. Seale), pp. 2921–2929. Association for the Advancement of Computing in Education. Vancouver, Canada. McPherson M.A. & Nunes J.M. (2007b) Negotiating the path from curriculum design to e-learning course delivery: a study of critical success factors for instructional systems design. In Proceedings of the 2nd European Conference on Technology Enhanced Learning: Creating New Learning Experiences on a Global Scale, 17–20 September 2007 (eds E. Duval, R. Klamma & M. Wolpers), pp. 232–246. ProLearn Academy, Crete, Greece. Merton R.K., Fiske M. & Kendall P.L. (1956/1990) The Focused Interview: A Manual of Problems and Procedures. Collier MacMillan, London. Murphy M.K., Black N.A., Lamping D.L., McKee C.M., Sanderson C.F.B., Askham J. and Marteau T. (1998) Consensus development methods and their use in clinical guideline development. Health Technology Assessment 2, 1–75. Available at: http://www.ncchta.org/fullmono/mon203.pdf (last accessed 3 February 2007). Noy N. & McGuinness D. (2001) Ontology development 101: a guide to creating your first ontology. Stanford Knowledge Systems Laboratory Technical Report KSL-01-05 and Stanford Medical Informatics Technical Report SMI-20010880, March 2001. Available at: http://www.ksl.stanford. edu/people/dlm/papers/ontology101/ontology101-noymcguinness.html (last accessed 20 December 2006). Nunes J.M., McPherson M.A. & Rico M. (2000) Instructional design of a networked learning skills module for web-based collaborative distance learning. In Proceedings of the European Conference on Web-Based Learning Environments (WBLE 2000), 5–6 June 2000 (eds F. Restivo & L. Riveiro), pp. 95–103. Facultade de Engenharia da Universidade do Porto, Porto, Portugal. Nunes J.M. (2003) Instructional design CSFs for eLearning. In Proceedings of the Workshop on the Changing Face of HE in the 21st Century: Critical Success Factors for Implementing eLearning (eds M.A. McPherson, L. Henderson & Kinshuk), pp. 25–29. Massey University, Auckland, New Zealand.



Onwuegbuzie A. (2003) Effect sizes in qualitative research: a prolegomenon. Quality and Quantity 37, 393–409. Pandit N. (1996) The creation of theory: a recent application of the grounded theory method. The Qualitative Report 2. Available at: http://www.nova.edu/ssss/QR/QR2-4/pandit. html (last accessed 3 January 2007). Robinson B. (2001) Innovation in open and distance learning: some lessons from experience and research. In Innovation in Open and Distance Learning: Successful Development of Online and Web-based Learning (eds F. Lockwood & A. Gooley), pp. 15–26. Kogan Page, London. Robson W. (1997) Strategic Management and Information Systems. Financial Times-Pitman Publishing, London. Rockhart J.F. (1979) Chief executives define their own data needs. Harvard Business Review 57, 238–241. Salmon G. (2000) E-Moderating: The Key to Teaching and Learning Online. Kogan Page, London. Sanders R. (2006) The ‘imponderable bloom’: reconsidering the role of technology in education. Innovate 2. Available at: http://www.innovateonline.info/index.php?view=article &id=232 (last accessed 1 August 2006). Saulnier C.F. (2000) Groups as data collection method and data analysis technique. Small Group Research 31, 607– 628.


445

Schwandt E. (2006) Critical research: definition. In The SAGE Dictionary of Social Research Methods (ed. V. Jupp), pp. 51–52. Sage Publications, London. Strauss A. & Corbin J. (1990) Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Sage, London. Strauss H. (2001) E-learning basics: case study. eLearn 10, 5. Tinker R. (2001) E-Learning quality: the concord model for learning from a distance. NASSP Bulletin 85. Available at: http://bul.sagepub.com/cgi/content/abstract/85/628/36 (last accessed 3 March 2007). Vaaland T.I. (2004) Improving project collaboration: start with the conflicts. International Journal of Project Management 22, 447–454. Von Glasersfeld E. (1989) Cognition, construction of knowledge, and teaching. Synthese 80, 121–140. Whitworth A. (2006) Dynamic but prosaic: a methodology for studying E-learning environments. International Journal of Research & Method in Education 29, 151–163. Wopereis I., Kirschner P., Paas F., Stoyanov S. & Hendriks M. (2005) Failure and success factors of educational ICT projects: a group concept mapping approach. British Journal of Educational Technology 36, 681–684.