Educational Technology & Society 4(3) 2001 ISSN 1436-4522
Supporting diverse learners through a website for teaching research methods Kim G. Lie Psychology Department Queen Margaret University College Clerwood Terrace, Edinburgh EH12 8TS Scotland, United Kingdom
[email protected] k Tel: +44 131 317 3609 Fax: +44 131 317 3605
V. Cano Department of Information Management Queen Margaret University College Clerwood Terrace, Edinburgh EH12 8TS Scotland, United Kingdom
[email protected] Tel: +44 131 317 3515 Fax: +44 131 317 3605 Abstract This paper presents the development, use and evaluation of a website to supplement face-to-face teaching of research methods at postgraduate level. Diversity as well as disparity in student backgrounds presented challenges in providing appropriate educational support at multiple levels. The site was developed to meet the diverse student needs with scarce teaching resources and represented an innovative use of Information Technology in a small higher education institution in Scotland. Formative evaluation methods that included questionnaires and focus groups were used to investigate the student learning experience. Positive student evaluations showed the effectiveness of developing a website to enable and empower students. Lessons drawn from the project include the importance of pedagogical principles, simplicity and user-friendliness in web design. The study showed the feasibility and desirability of integrating World Wide Web resources with on-site teaching for students who seek attendance-based courses and for initiating them to more effective use of the Internet. However, technical support and appropriate infrastructure for faculty who develop web resources remains essential. It is hoped that this case study may encourage others to harness the strengths of web technology to meet contemporary challenges in teaching students with diverse needs.
Keywords Web-based instruction/learning, Teaching research methods, Student diversity, Formative evaluation, Postgraduate education
Introduction A key challenge faced by faculty in their teaching role is how to meet the diversity of learners’ needs with limited resources without diminishing the quality of learning. This study provides an example of innovative use of World Wide Web (WWW) technology by faculty with little time and technical skills but empowered by pedagogical skills and shareware. Their experience shows the feasibility and desirability of developing Web resources to enhance onsite teaching. The paper extends the literature evaluating student responses to use of Web resources for instructional purposes in two ways. Firstly the paper highlights the features which contributes to positive learning experience for students new to Communication and Information Technology (IT). Secondly, the study reports on a range of support for difficulties and fears relating to research methodology as an interdisciplinary subject. The paper first outlines the nature of the problems facing faculty teaching postgraduates, and the benefits of Web technology before reviewing evaluations and the nature of the research methods curriculum. In the UK, the number of students in higher education (HE) has been expanding rapidly (Burgess, 1997). Postgraduate (PG) statistics available for the academic year 1998/99 show that they comprised 22% of the UK
50
student population at HE institutions (HESA, 2000). Further expansion is likely in the context of a growing awareness of the need for lifelong learning (DfEE, 1998). In addition, many professions have underlined or even specified the need for continuing professional development (Woodward, 1996) as a requisite for registration in the respective professional bodies. Universities have responded to this need by multiplying PG programmes (O’Leary, 1995).
Diversity of postgraduates and implications for support The increase in student numbers has in turn contributed to the growing diversity of the PG population. PG students are diverse in several aspects as motivation, previous study background, length of time elapsed since formal study, foundation knowledge underpinning the postgraduate course, IT skills and modes of study. Diverse motives for studying surface in PG classes. Some are intrinsically motivated by the prospect of intellectual development, while others are extrinsically motivated by better job or career prospects consequent to achieving PG qualifications (Entwistle et al 1992). Most students have mixed motives in varying proportions which will affect their approach to study and their needs for support. Differences in motivation also arise from the modular system which typically requires students to take a combination of core and elective subjects. This feature leads to some students taking core modules as a compulsory requirement of their degree programme but without having a particular interest in those subjects (Winn, 1995). Wide variations in levels of foundational knowledge and skills as well as metacognition of learning occur due to several factors. One factor is the switch to modular learning systems in the UK which offer flexibility in terms of entry and exit points. Such flexibility fosters an extremely diverse student body, since courses are open to students from a range of departments and at all stages of their study (Bradford, 1994; Rich et al., 2000). This flexible access mean that faculty can no longer assume that progress to PG education has been governed by an acquisition of prerequisite knowledge and skills and that PG courses can build on the knowledge students have already mastered. Laurillard (1993) has pointed out the dangers of such an assumption in the context of undergraduates, but her insight applies equally to PGs in the context of open access in modular programmes. The consequent disparity in levels of foundational knowledge and general study and IT skills needs to be recognised by faculty when designing teaching programmes. These wide variations in ability, knowledge and learning experience create dilemmas for teaching staff. For example, if the lecturer addresses the middle level of ability, weaker students will struggle to keep up while stronger students may lack stimulation and stretching. When conventional teaching and learning methods are used, there is often a disparity in resources available to students as a function of their mode of study. For example, full-time students are typically able to access more of the onsite facilities and resources compared to part-time students coping with the logistics of commuting to the campus and their work and family commitments.
What web technology can offer Web technology provides appropriate flexibility in higher education by providing time -location independence, a range of information types (Wild, 1999), improving the range and quality of learning activities, and shifting the responsibility to students for the selection of appropriate resources (Collis, 1996). Cookson (2000) distinguishes three levels of application for instructional purposes: enhancing face to face instruction, enhancing distance education and instruction entirely online. This paper focuses on the first level of application, where Web technology is used as a medium for supplementing learning resources and continuing discussions outside onsite classes. This application requires relatively minimal technical support. Web-based course support sites are one example of Cookson’s first level of application. Such sites are collections of Web pages, resources and tools that enhance a course (Collis, 1999). They range from the simplest forms which provide only course information to complex sites with multiple functionalities such as links to faculty-wide information, lecture presentations, synchronous and asynchronous conferencing and interactive exercises for self study and practice (Collis, 1999). These sites aim to increase efficiency of course delivery, enrich course participation and increase flexibility in meeting student needs (Collis, 1998). Flexibility in meeting the needs of different subgroups of students or individual students is facilitated because the Web medium allows for diversity of course content, support material and presentational styles (Moguilevsky et al., 1999).
51
Evaluative studies of support sites While the use of Web-based support sites has been growing, reports of evaluations are scarce and there is wide variation in evaluation methodology. In addition, formative evaluations by their very nature need to be interpreted within the contexts of their institutional systems, learner characteristics, curricula and site objectives. However, many studies provide insufficient information on one or more of these aspects, making comparisons across studies difficult. The majority of evaluations report positive responses to support sites, whereby students perceive them as enhancing or reinforcing their learning, increasing motivation to learn and giving more control to students (Fitzelle & Trochim, 1996; Meek, 1998; Rich et al., 2000; Bailey, 2000). Fitzelle and Trochim’s (1996) survey evaluated the Trochim research methods website with a convenience sample of 68 undergraduates, the majority of whom were female. In Meek’s (1998) study, the majority of students rated the Web as an effective learning tool in both pre- and mid-module surveys, with a significant increase to positive perceptions (from 69% to 93%) in one module that used more interactive Web pages. Rich et al.’s (2000) study, based on 362 undergraduates, used mixed methods including questionnaires and focus groups and found no gender differences. Rosie’s (2000) qualitative study, which used in-depth interviews found that students with deep learning approaches (Biggs, 1999) valued the Web resources as an aid to constructing their own learning framework, in contrast to devaluation of the medium with a surface learning approach. Collins’ (2000) summative evaluation, based on the students’ final course scores, did include the finding that the majority of the students were satisfied with the Web resources and would have taken another web- based course. A marked polarity of student views emerged in Usip and Bee’s study (1999). Student who did use the Web resources perceived the site as reinforcing their learning in contrast to non-users who remained unconvinced of its value. However, the high level of negative responses from the non-users could have been because students had to acquire knowledge of the Web by themselves, or were constrained financially in accessing the Web. In summary, the direction of findings is towards positive student responses to Web support sites, provided that these are user friendly and do not incur financial burdens, or require students to learn about the technology on their own (Usip & Bee 1999). In addition, Fitzelle and Trochim (1996) and Collins (2000) reported some spillover effects in that some students acquired generic computer skills such as using email.
Research methods as a subject Developing and providing appropriate forms of support for learners also has to take into account the nature of the subject matter being taught. Research methods courses pose particular challenges for teaching because of the interplay between “student-specific and task-specific factors “(Laurillard 1993, p.31). Firstly student motivation and beliefs about the subject have a powerful influence on the way they learn. Winn’s study (1995) notes the challenges of teaching research methods in a meaningful and relevant way when it forms a compulsory rather than an elective subject. Students may also have a long-standing aversion to the quantitative and technical aspects of research (Burgess, 1981; Wakeford, 1981; Schutt, 1984). This aversion can be partly attributed to the commonly held belief that statistics is a subject requiring understanding of enormous amounts of symbols and difficult calculations which students will never be able to understand (Diamond & Jeffrey, 2001). These student motivations and beliefs exacerbate any difficulties related to cognitive tasks in research methodology. For example, students may have conceptions of research ranging from the naïve to the idealistic and may misunderstand technical terms which have everyday meaning such as ‘population’ and ‘sampling’. Many often struggle to understand and use notational, graphical and numerical representations. In research methods courses focusing on skill acquisition and not just knowledge, students may also have 'phobias' about IT. These fears need to be overcome so that students become competent in using new technologies for research tasks, such as information retrieval and data analysis.
Context of study A website was developed to supplement face to face (ftf) teaching of research methods to PG students on a modular masters scheme at a small Scottish HE institution. The research methods modules serve students registered on eight different health related programmes from 3 faculties at Queen Margaret University College
52
(QMUC). The faculties of Arts, Health Sciences, and of Social Sciences and Healthcare offer awards ranging from PG certificates to MScs for a range of professional disciplines such as doctors, nurses, therapists, librarians, manager, and administrators. Thus, the students have varying interest in philosophical aspects, quantitative and qualitative research approaches and types of research question. For example, nurses may be more interested in qualitative research, whereas physiotherapists may be more interested in quantitative research. Teaching research methods to postgraduates has been problematic due to a disparity of academic backgrounds, a diversity of needs and worries regarding statistical content. The disparity relates to different levels of academic qualifications, previous research knowledge and skills, and of acculturation to higher education. Disparity in academic backgrounds arises from several factors. Many of the award programmes are a form of continuing professional development (Woodward, 1996) and minimum entry requisites are a professional qualification and some years of working experience. As a number of health professionals obtained their professional qualification before the advent of degree courses, academic backgrounds may range from a certificate level to a master’s degree. Consequently, some students may have no previous study of research principles (a normal part of the curriculum in honors degree courses, but not in pre-degree or ordinary degrees) or related skills, whereas others will demonstrate differing levels of research knowledge. In addition, some students need foundational knowledge not only pertaining to research, but also to their acculturation to higher education. Thus the challenge was to teach less well-prepared students and advanced learners to a PG level, in the same time frame, with limited teaching resources. Another factor contributing to the diversity of student needs is the mode of study. Most students are part-time rather than full-time. They juggle their study with full-time employment and family commitments. Although they choose the ftf courses and do not seek distance learning, they often have attendance problems due to their other commitments as well as geographical factors. Many of them travel from neighbouring regions to attend classes and therefore geographical factors affect their attendance to class, consequently they have difficulties keeping pace with the modules. Regardless of previous academic attainment, many students find statistics a problematic area of study. Their difficulties range from coping with the intellectual demands, to a number of fears and emotional baggage from previous struggles with mathematics. These problems required both remedial tutorials and academic counselling, with a corresponding increase in staff time and workload. The ResearchTrek (RTrek) website was an innovative attempt to provide diverse types of support transcending time and logistical constraints as well as reducing the amount of ftf tutorials. Thus the site would provide support by: Ø facilitating the incorporation of less well prepared students for PG study; Ø providing resources to complement the print-based materials and ftf teaching; Ø providing self-assessment exercises in statistics with automated feedback; Ø stimulating competent learners to explore topics further.
The RTrek Website The website is couched in a “shell” developed in-house using freely available shareware. The shell had been previously tested in undergraduate modules with positive results (Buckner & Davenport , 1996, 1997; Cano, 1998). This approach was taken because QMUC did not have a specific support policy towards the development of web-based learning material during the relevant period. Integrated software packages for the development of web-based learning material were neither available nor supported by the IT centre. Thus the web-based learning environment had to be developed mostly through shareware. The site has the following components: Subject content: Hypertext material as Study Notes (SN) and Quick Guides (QG); Module information; electronic self-assessment workbooks(WB) with automated feedback; help centre with a Glossary, FAQs and internal search engine; conference space; and links to other web-sites. The rationale behind each of the components will now be described, as well as the interaction of the site elements with the ftf sessions.
53
Study Notes The hypertext material was aimed at scaffolding the research knowledge needed to underpin the research skills practised in the workshops. In particular, the SNs were designed to serve the following functions: Ø summarize the key concepts which students needed to apply in the ftf workshops; Ø summarize the essential points to grasp when reading the core textbooks; Ø highlight the main philosophical, epistemological., and methodological aspects which underpinned the ftf sessions; Ø complement the information on the quantitative and qualitative methods covered in the ftf sessions. Questions for reflection were included at the end of most SNs to reduce passive surfing and to encourage deep learning. The latter represents an active drive for personal understanding instead of surface learning, where students merely reproduce what has been learnt (Entwistle et al., 1992; Coles, 1998). Key terms and concepts in the SNs were hyperlinked to the QGs and the Glossary.
Quick Guides The QGs provided brief overviews of the philosophies underpinning research methodologies. The aim was to provide an initial framework to guide their reading because most students have little or no previous background knowledge.
Links to in-house and external web sites Links to an undergraduate site for investigative principles and to the QMUC library were made. The former was to help the students with no exposure to research. The latter links were embedded in the SN on literature searching to facilitate access to the library databases. Links were also provided to external sites such as the web-based text Research Methods Knowledge Base at Cornell University (Trochim). The aim was to stimulate more competent learners to explore other perspectives. Other links took account of the diverse professional disciplines represented in the cohort. For example, links to a number of relevant medical sites were made for doctors or links to health promotion sites for those registered on that award programme. All links were embedded in the SNs rather than as a separate component.
The Workbooks The WBs comprised self-assessment exercises on the topics covered in the ftf statistical workshops. These tested baseline knowledge of terms and concepts so that the ftf workshops could attain a more advanced level of learning. Automated feedback gave scores and advice on revision. The WBs were designed to prepare students for the ftf workshops by Ø requiring students to do active study and not just reading; Ø shifting the responsibility for statistical learning to students; Ø reducing the disparity in knowledge for the ftf workshops; Ø eliminating the need for ftf remedial tutorials. Thus the WBs complemented the ftf workshops and the paper-based materials. The latter included a booklet called “Understanding Statistics” because resource constraints made it impossible to put it on the website.
The Conference Space The CS was envisaged as an additional support for the ‘remote’ students (travelling more than 100 miles) who could have used it for extra discussion without time and logistical constraints. Hence, use of the conference space was left optional and no activities were specifically designed. Moreover, student collaborative learning was required in the preparation for, and during, the weekly ftf workshops. Increasing the workload of both students and staff would have been counter-productive.
54
The Help Centre The Help Centre comprised the Glossary, FAQs and an internal search engine. The Glossary contained the terminology used in both the ftf programme and the website.
Design and development of the site The site was designed as an integral part of the ftf programme from its inception to its use by several methods. Firstly, design principles used for the site mirrored those used for the research methods modules, and included a needs assessment based on recurrent cohort characteristics and a consideration of the nature of the subject matter (Rowntree, 1997). A second principle was to scaffold learning, a concept derived from Vygotsky’s theory (1962).
Integration with face-to-face curriculum The use of the site was formally and explicitly integrated with the ftf programme. Students were introduced to the site during a ftf workshop with hands-on exercises and a printed manual. The module timetable directed students to specific web pages and activities as well as specific print-based materials. All topics covered on the site were mentioned in the workshops. Thus the website served to complement and supplement, but not replace, the curriculum given in the ftf programme and printed materials. The Workbooks constituted requisite preparation for the statistical workshops ni order to attain a more homogenous level in the class. Advanced exercises in class would then be possible without remedial on-site tutorials to support weaker students. Two faculty members from different departments developed the site with in-house funding within a summer. They were responsible for the academic and pedagogical content while a free-lance HTML programmer carried out technical implementation. The site was piloted on graduates from the previous cohort.
Evaluation methods The evaluation undertaken in this study was formative rather than summative. The latter type of evaluation seeks to assess the learning outcomes achieved, typically by final module scores. By contrast formative evaluation has “diagnostic purposes, to uncover strengths and weaknesses, to improve learning and teaching” (Cox, 1994, p106). In particular the evaluation focused on “what learning benefits students demonstrate from the material per se and what kind of learning accrues from student use of it” (Laurillard 1993, p248, 252). This kind of evaluation is not concerned with demonstrating “improved performance” in terms of grades or marks, but with “improving performance” (Laurillard 1993). Hence formative evaluation represents an integral part of good teaching practice. Formative evaluation was carried out through questionnaires and focus groups. The questionnaire was administered halfway through, and, at the end of the module. The 18-item questionnaire assessed userfriendliness of the site, usage, views of the usefulness of components, and satisfaction with the learning experience. At the end of the module, focus groups were used to further explore issues raised in the mid-module questionnaire. These were the underuse of the links to other websites, the impact of the website use on their perceptions of the Internet, and student perceptions of ftf and web-based learning, Data on the last aspect will be reported in another paper ( Lie & Cano, in preparation). Analysis of the qualitative data was undertaken using NUDIST 4.0. The data was examined for recurrent categories by both researchers. The students then scrutinized the analysis of categories and illustrative quotes and made amendments where they felt their views had not been captured accurately.
55
Results Student profile From a cohort of twenty students, eighteen students representing a 90% response rate returned questionnaires. As can be seen from Table 1, the majority of the respondents was female, part time and had not previously studied research methods. More than three quarters of them reported prior exposure to the WWW, but their familiarity with the medium varied as evidenced in their comments (see Impact of site usage on perceptions of the WWW). Fifteen students (75%) took part in the focus groups. Student characteristics No. of students Female 13 (72%) Male 5 (28%) Part time study 12 (67%) Full time study 6 (33%) Had prior exposure to the WWW 14 (78%) Had nil exposure to the WWW 4 (22%) No prior study of research methods 13 (72%) Previously studied research methods 5 (28%) Table 1. Characteristics of respondents
User-friendliness of the site There were initial teething problems in ensuring access to the website. These problems involved technical factors and infrastructure such as passwords and servers. At mid-module, 66% (n=12) reported easy access. However, securing the site with passwords had led to 2 students (11%) being unable to access the site from their home or workplace, while 6 students (33%) had great difficulty. These access problems were resolved within days, and subsequent occasional difficulties coincided with general difficulties on the institutional network. As regards ease of navigation, 78%(n=14) found it ‘Very easy’ and 22%(n=4) rated the navigation as ‘Satisfactory’. Fast downloading was experienced by 89% (n=16), and 11% (n=2) rated it as slow. Spontaneous comments in the focus groups corroborated that students found the site user-friendly in terms of its design, easy and convenient access in one’s own time (outwith the initial difficulties), easy navigation and satisfactory loading speed. The site structure was seen as coherent, the shallow hierarchy “made it easy to find what one was looking for”, allowing quick choice of topics. Students also commented on the advantages of quicker and more convenient access to information “at the click of a button”, and access to more information than from a textbook. Below are some illustrative quotes. Transcript references are shown in brackets. (Instead of reading a book) it's a lot quicker to click on a reference and get it, flick through and soon. (B#416418) For me, learning from the web is a lot easier than picking up a book and sitting reading a chapter. If you have got a spare ten minutes at work when you were eating your sandwich, you just log on to a site and have a quick flick. ( B#455-491) You don't have to remember to take the books with you and you think "Oh I have left them in the car - Can I be bothered going .....(B#488-491) I am lucky in that I have got (the Net) right at my fingertips at work. Obviously, if you can connect on to something that you are doing in your own time, it is so much easier than having to come into College. (A#33-36) …….it meant that when I was in the office at work if I had literally five minutes or ten minutes in my lunch hour and when I was eating my sandwich I would think "Well I'll just have a go now and I'll have a look through and sort of wander around". (B#461-465)
56
Ways in which students used the site Students used the site according to its purposes of providing a range of information types and catering for different learner levels. As can be seen from Table 2, all students reported using the site to check their understanding of the preparatory reading. By the end of the semester, all reported using the Workbooks to prepare for the workshops (compulsory preparation). The number of students using the site to reflect on questions rose from 11(61%) to 15 (83%) by the end of the semester. Usage of the site for catching up on foundation knowledge, looking at the Quick Guides, Glossary and FAQ’s also rose (more than doubled for the first case). By contrast, the number of students using links to explore other sites halved and there was no increase in the use of the Conference Space. Uses of ResearchTrek Mid -module End of module Catch up on foundation knowledge by linking to the 4 (22%) 10(56%) undergraduate site Check my understanding of preparatory reading 18 (100%) 18(100%) Reflect on questions in Study Notes 11 (61%) 15 (83%) Use Conference Space to discuss 3 (17%) 3 (17%) Explore further via links to other sites 4 (22%) 2 (11%) Use Quick Guides 6 (33%) 12 (66%) Use Glossary 9 (50%) 12( 66%) Use FAQs 4( 22%) 6 (33%) Table 2. Student use of ResearchTrek website during module
Satisfaction and an enjoyable experience Ninety four percent (n=17) were satisfied with their web based learning experience at the end of the module. Only one student (6%) expressed dissatisfaction, as she had had no access to a computer at home nor at the workplace. 89% of students (n= 16) had found learning on the web an enjoyable experience and only 11% (n=2) did not. General comments referred to the site as “excellent”, “fab”, “a superb resource”, “a fun way to learn”, “Unusual and novel experience but easy to navigate”. The site was seen as a “good learning tool” that reinforced the learning from the weekly ftf interactive workshops. Students understood that key points were summarized on the web pages and constituted another learning resource that prepared them for the next ftf workshop. Students appreciated and valued the site and wanted it to continue and expand. They also wished that other departments would develop websites. Some illustrative comments from the questionnaires are given below: An excellent user friendly website that reinforced learning and allowed self reflection in a non-threatening way I wholeheartedly support this as a learning resource Definitely worthwhile for investment; provides a more anonymous way of learning Keep it up and keep adding content More questions to facilitate learning More, more, more; would love to see its extension to wider scope and content This is an excellent idea and will hopefully be copied by other depts. The only negative comments in both questionnaire and focus group data related to the initial access difficulties and issues, and under-use of the conference space.
Student views of the usefulness of particular components By the end of the module, 78% (n=14) rated the Study Notes as very useful, and 22% (n=4) as of some use. More students perceived the usefulness of the Study Notes towards the end of the module than about half way through (78% vs 61%). Explanations were perceived as well written, accessible, concise and conveying the key points. Questions for Reflection placed in the Study Notes were found to be useful and “thought provoking” indicating that students did not remain at the level of passively gathering information.
57
I found if I didn’t like reading the books then the Study Notes were a sort of summary to reinforce what I learnt. (B#353-356) …(The Study Notes) underpin previous reading by putting it in another context. (Questionnaire data) It provided a sort of support as in it reiterated the main points… as you had the in-depth things during your class time. It just made clear the points that you should actually be mentioning, as opposed to getting bogged down by all the different information. I mean it acts as a reminder almost. It summarizes the points that you should have taken on board. (A#553-562) Focus group participants also spontaneously reported that they found the site helpful for their assignments because it elaborated on aspects not covered in the weekly sessions or gave additional tips. An illustrative quote follows: Obviously the assignment guidelines were clear in itself, they were great but (the Study Notes on the site) told you exactly what you had to present in your assignment, so I found it helpful that way. It is just obvious in class you can’t take on everything on board. It’s another resource that you have got… instead of having to wait for the next week to ask questions. (A#495-505) In contrast to the SNs, more students found the QGs useful at mid-module than at its end (100% vs 61%). Questionnaire comments reported the QGs as helpful because they provided the key points through brief and concise information. The guides served to “brush up learning”, “top-up learning” and “to consolidate”. Questionnaire and focus group data revealed that students liked the self-assessment nature of the statistical exercises, especially the anonymous nature of the learning and the opportunity to practice several times. The immediate, online feedback served as checks on student understanding or reinforced their knowledge. The Workbooks were seen as helpful in making them aware of what they had not grasped, indicating what areas needed revision. The glossary was also found to be very helpful.
Comments on the links to external websites The mid-module questionnaire data had shown that only 4 students perceived the links to other websites as helpful. However, further exploration in the focus groups revealed a spectrum of student views: Ø Links were seen as a bonus, or non-essential; Ø Links were not used because of lack of time; Ø A resource that could be tapped when needed; Ø Students found other links on their own; Ø Each has a professional responsibility to update oneself; Ø More links would have been helpful. Some illustrative quotes are given below: Again it was a sort of bonus and additional extra type thing. I felt I had enough with what I was using without that. Obviously it is an added bonus and probably if I had more time I would have used them. (A#1191-1193) It is always the time factor,…when you have not got the pressure of time, you think ‘well I’ll go back to that and look at it’. (A#1186-1189) (Links were not used because)…you had the core text which we used quite a lot, plus we had the workbooks in statistics, plus what we were getting in class and the articles and stuff, so I think that was enough. (A#582-589) It is the sort of thing you can use again at another time; Oh yes, absolutely (A#610-614, several students) We know that the sites are there and we can go there and jump out when we need them, but just because we haven’t used them, doesn’t mean to say they shouldn’t be there. They should be there. (A#620-624) I found sites that were basically American sites. A lot of them had a slightly different angle on something we'd been taught and even diagrams. I found that very useful because it was a different way of looking at something we had already looked at and somebody else's interpretation of a methodology. (B#688-699)
58
I think we probably have to take responsibility as professionals to look. You always have to keep up-to-date with things that are going on. Whereas before you would have gone to the library and search for references, now you have it accessible, so I think we have to take some of that responsibility as well. (A#1264-1270) At the other end of the spectrum, an experienced user of the WWW felt restricted and wanted more links. The only criticism I would have is with the links. The site was very good for the purpose but if you did want to stretch it a little bit extra and explore something, it was a bit more difficult to do that. (B#238-243)
Impact of site use on learning The focus groups showed that site use widened student learning, especially for novices to the WWW. A range of subsequent student views emerged: Ø transformation of a previously negative view of the WWW to a positive view; Ø reinforcement of positive views of the WWW as an accessible and useful resource; Ø extension or increase in their use of WWW resources; Ø use of the WWW seen as an integral part of education; Ø no change from previous use of the WWW (one experienced user). Illustrative quotes can be seen in Table 3. Types of student views Using the site transformed previously negative views of the WWW to a positive view
Illustrative quotes - I have a sort of phobia for Internet and I think it’s just that all that knowledge comes up, but I got a surprise, how positive it is. (B# 6466) - For me using the Internet is a fairly new thing…. The thing I found so useful about ResearchTrek was because it was specifically for the purpose of that module, so there wasn’t anything in it that didn’t apply…With the Internet there is so much junk that you have got to wade through all the useless information to get to anything that is actually useful within it. (B# 17-31) Reinforced view of usefulness and - (ResearchTrek) has just really reinforced that the Net can be a accessibility of the WWW useful resource that’s accessible to you.(A#24-26) Extended personal use of the - (My use of the Net) has increased because it has made me more WWW aware of what use it can provide basically. (A#32-37). - I am using the Net more in general as I do more modules. We have just finished a module and the first thing I would normally do would be to go the library to try and get the books or the journals. Last week I found that there weren't very many books, the next thing I thought was to use the Net… maybe a year ago I wouldn't have either known or done that.(A#70-80) - I know certainly from my own work …I am much more likely to use Internet as a tool for getting information. ( B# 200-204) - I was using the Net before on and off but was not very well versed in that. I wasn’t very good at using it, but after using ResearchTrek the Net became handy (B#188-191) WWW skills seen as part of wider - I think that it is probably part of the learning process,…of education education in general. You become more confident in using things and accessing other sources(A#88-93) -The site helped me get on my feet with regard to accessing information. (B# 47-49) No change in perceptions -I do use the Net a lot anyway so it wasn’t a great change for me. (B# 214-216 Table 3. Impact of site use on wider learning
When the views of students were analyzed with regard to their degree of familiarity with the WWW, the site appeared to help those with none or little experience most. Newcomers to the WWW found the site a positive
59
experience because they found it “very simple”, “straightforward to use” and “easily accessible”. One student reported that the site “helped me to get on my feet with regard to accessing (the Internet)”. Those with a little experience also had positive experiences about the site. Some had had positive prior experience of using the WWW (e.g. cheap flights) but others a negative experience. Hence the site use reversed their previous negative experience of the WWW. Worries about using the WWW due to lack of familiarity and difficulty in obtaining useful information were overcome. Finding quick and easy access to useful information on the site reversed perceptions that the WWW seemed “filled with junk information”. Others found a heightened awareness of the usefulness of the WWW and acquired more confidence in accessing other sources. Searching the WWW when printed materials were scarce became a new method of finding information and added to their study repertoire. One student was motivated to develop further webskills. You know you weren’t really using it to its full potential before whereas I am beginning to realize that (the Web) is very useful (A#235-237) The first thing I would normally do would be to go to the library to try to get the books and journals… the next thing I thought was to use the Net… maybe a year ago I wouldn’t have either known or done that. (A#69-79) Experienced users had contrasting views. One user found the site restrictive because of the minimal number of links to external sites. By contrast, another student appreciated the “shallow hierarchy” because it permitted quick access to the essential information, fast downloading and easy navigation
Discussion With regard to the study methodology, triangulation of data through questionnaires and focus groups strengthened the study. In addition, involving the students in the qualitative data analysis of the focus groups served as a check against researcher bias. Due to resource constraints, the faculty members who had developed the site facilitated the focus groups. In addition, one was also the main tutor for the course. However, checks through the data for both groups showed that negative comments were made in both groups indicating that students felt free to express their views. The high level of student satisfaction with their web-based learning experience compares favorably with the 71% found in Meek’s study (1998) of geological science students. Perceiving the site as a useful learning tool echoes similar findings from students across a range of subjects (e.g. Usip & Bee, 1998 – economics, Collins, 2000 – biology, Kapur & Stillman, 1997 – arts & social science). Student perception of the site as an exc ellent learning tool agrees with the enhanced learning reported by students using the Trochim Research Methods website (Fitzelle & Trochim, 1996). Student perceptions of the site as user-friendly are consistent with its conformity to web design guidelines (Collis, 1999). Technophobias were overcome through the simple site design of a shallow hierarchy as well as through the introductory workshop with structured exercises. Integrating the site with the ftf programme appeared essential to its success. Students felt empowered as learners through several components of the website. They preferred the electronic workbooks instead of the self-assessment exercises in a core textbook. A possible reason may be that the automated feedback benefited students in terms of perceiving more learner control. The importance of learner control as perceived by students when using the WBs and choice of links echoes similar findings in Keller and Knopp(1987) and Fitzelle and Trochim (1996). The site also initiated and fostered students’ academic use of the WWW. The reported impact on developing student use of the WWW as an academic resource can be regarded as application of skills learnt to other course (Fitzelle & Trochim, 1996). The study showed the feasibility and desirability of developing a website for postgraduates to complement ftf courses that already have existing print-based materials. The two faculty staff created the academic hypertext material over a summer. In some respects, the academic task of writing the hypertext material was relatively easier than writing lectures because of the shorter length (Collis, 1999). In addition the content was pared down to the essentials to aid the beginning student. The bulk of the time was taken up by searching for and reviewing ext ernal websites that might represent the next step forward for the progressive learner or for the more competent learner.
60
The support site was desirable from both the student and staff perspective. From a student’s perspective, the website was a success because it answered their diverse needs and their preferences for a minimalist approach. Student preferences for concise summaries rather than elaborated information agree with constructivist theories of knowledge (Hoffman, 1999) and with Collis’ criteria for good design (1999). From the faculty view, staff were freed from running remedial individual tutorials and the ftf statistical workshops started from higher baseline knowledge. However, the study showed the importance of support structures for innovative faculty. Academics who pioneer innovative use of web technology need to be supported by adequate training and technical backup (Collis, 1996, p332). The negative taint on student perceptions relating to password problems during the early weeks could have been prevented by sufficient technical backup to ensure access. Political and institutional statements on using IT in higher education need to be supported by concrete training and support actions. Nonetheless, this project showed that the advantages to students and faculty of having time- location independent learning resources did outweigh the attendant problems of infrastructural issues.
Conclusion The site did provide support for the diverse range of information needs and of learner levels in the cohort. It was especially effective in leveling the playing field for less prepared learners. These came better prepared to participate actively in the ftf workshops. Faculty were spared from giving ftf remedial tutorials in order to bring students up to threshold levels. The site served as a valuable time -location independent resource particularly for the part time students. These felt empowered by the technology because they could make more efficient use of their time to fulfill both study and other commitments. For example they valued use of the site in short time periods (e.g. lunch/ tea break). They also made fewer calls on faculty time outside the ftf programme compared to previous cohorts. The kind of learning that accrued from student use of the site, as indicated by their evaluation, was active and orientated towards deep learning. The support site appeared to encourage greater independence and self-reliance by allowing them to control their learning activities more consciously. Students took more responsibility for their own learning by deciding when, where and for what purpose they accessed the site. It enabled them to make fuller use of the range of learning resources including paper-based resources. Lessons were learnt from the experience of site development. The basic lesson was the importance of simplicity and user-friendliness. In other words: keep it simple for both students and faculty. Students with minimal IT skills, or technophobias were empowered to use the site and had positive experiences. Faculty who were not technically minded found it feasible to develop a site within a reasonable time period. Another lesson was that pedagogical principles needed to design a web-based learning environment are similar to those required for designing ftf curriculum. The need for sound structure to scaffold learning applies as much to designing course-support sites as to preparing lectures. This may seem obvious but there may be a tendency for faculty who are new to web technology to perceive it as an alien domain requiring considerable technical skills in the first instance, rather than pedagogical skills. The experience of the developers was the reverse, who found that the process also furthered professional development in teaching. The positive student response to the site may be partly attributed to its integration with the ftf programme by the formally explicit indications to its use and timings. In addition there was no duplication between materials on the site and the paper-based materials. Rather, the two types of material and learning activities contained within them complemented each other. Student comments showed that they did perceive and did use the site as supplementary to, and not as a replacement for, the ftf workshop programme. In brief, the positive student evaluation showed the desirability of developing a web based course support site to enable and empower students, and enhance their leaning. However, technical support for both students and staff remains essential to ensure reliable and convenient access. It is hoped that this case study may encourage others to harness the strengths of web technology to meet contemporary challenges in teaching students with diverse needs. Future plans are to expand the site content and scope as well as designing electronic conferencing activities. The latter will be aimed at developing more reflexivity in group work and improving the quality of writing skills.
61
References Bailey, D. G. (2000). Web based tutorials for teaching electronics. In Courtois , B., Guillemot, N., Kamannos, G. & Stehelin, G. (Eds.) Microelectonics Education, London: Kluwer, 155-158. Biggs, J.(1999). Teaching for quality learning at university, Buckingham: Open University Press. Bradford, M. (1994). Arena symposium: searching for good practice in postgraduate education. Journal of Geography in Higher Education, 18 (3), 335-372. Buckner, K. & Davenport, E. (1996). Support issues for case-based learning in an undergraduate human factors class. Education for Information, 14, 331-341. Buckner, K. & Davenport, E. (1997). Using a shell for delivery and support for case-based learning in a networked environment. In Judson, R., Maslin, S., Prothero, S. & Oates, L. (Eds.) Flexible learning in action, London: Kogan Page, 39-43. Burgess, R. G. (1981). Objectives in teaching and using research methodology. Sociology, 15, 490-495. Burgess, R.G. (1997). Beyond the first degree: graduate education, lifelong learning and careers, Buckingham: Society for Research into Higher Education, Open University Press. Cano, V. (1998). Measuring the organisational impact of a web-based information literacy module. Paper presented at the On-line Educa conference, December 2-4, 1998, Berlin. Coles, C. (1998). How students learn: the process of learning. In Jolly B. & Rees L. (Eds.) Medical Education in the Millennium, Oxford: Oxford University Press, 63-82. Collins, M. (2000). Comparing Web, correspondence and lecture versions of a second year non-major biology course. British Journal of Educational Technology, 31 (1), 1-27. Collis, B. (1996). Telelearning in a digital world, London: Thomson Computer Press. Collis, B. (1998). New didactics for university instructors: why and how. Computers and Quality Education, 31 (4), 373-395. Collis, B. (1999). Designing for differences: cultural issues in designing WWW based course support sites. British Journal of Educational Technology, 30 (3), 201-215. Cookson, P. S. (2000). Implications of internet technologies for Higher Education: North American perspective. Open Learning, 15 (1), 71-80. Cox, B. (1994). Practical pointers for university teachers, London: Kogan Page. Department for Education and Employment (1998). The learning age: a renaissance for a new Britain, London: Stationery Office. Diamond , I & Jeffrey, J. (2001). Beginning statistics, London: Sage. Entwistle, N., Thompson, S. & Tait, H. (1992). Guidelines for promoting effective learning in higher education, Edinburgh: Centre for Research on Learning and Instruction, University of Edinburgh. Fitzelle, G. T. & Trochim, M. K. (1996). Survey evaluation of website instructional technology: Does it increase student learning? http://trochim.human.cornell.edu/WebEval/webeval.htm Higher Education Statistics Agency (2000). Higher education statistics for the UK 1998/99, Cheltenham: HESA. Hoffman, B. (1999). Encyclopaedia of educational technology, San Diego, CA: San Diego State University, http://coe.sdsu.edu/eet
62
Kapur, S. & Stillman, G. (1997). Teaching and learning using the WWW: a case study. Education and Training International, 34 (4), 316-322.
Innovations in
Keller, J. & Knopp, T. (1987). Instructional theories in action: lessons illustrating theories and models, Hillsdale, NJ: Erlbaum Associates. Laurillard, D. (1993). Rethinking university teaching: A framework for the effective use of educational technology, London: Routledge. Lie, K. & Cano, V. (2001). Student perceptions of web-based learning compared to onsite learning (in preparation). Meek, J., Garrett, M. & Grattan, J. P. (1998). Evaluating the impact of new technology on students information gathering strategies. Alt-J, 6 (1), 57-63. Moguilevsky, A., Brown, G. H. & Grattan, J. P. (1999). Meeting students' needs: teaching initiative and experience at Aberystwyth. Active Learning, 20, 52-56. O'Leary, J. (1995). Coming in from the cold. The Times, 2nd Degree Supplement, September 11, 1. Rich, D. C, Pitman, A. J. & Gosper, M. V. (2000). Integrated IT based geography teaching: a Macquarrie University case study (Arena symposium). Journal of Geography in Higher Education, 24 (1), 109-115. Rosie, A. (2000). Deep learning: a dialectical approach drawing on tutor-led Web resources. Active Learning, 1 (1), 45-59. Rowntree, D. (1997). Preparing materials for open, distance and flexible learning, London: Kogan Page. Schutt, R., Blalock, H. M. & Wagenaar, T. C. (1984). Goals and measures for research methodology courses. Teaching Sociology, 11, 235-258. Trochim’s web-based text Research Methods Knowledge Base at Cornell University, http://trochim.human.cornell.edu/KB/ Usip, E. E & Bee, R. H. (1998). A discriminatory analysis of students perceptions of web based learning. Social Science Computer Review, 16 (1), 16-29. Vygotsky, L. (1962). Thought and language, Cambridge, MA: MIT. Wakeford, J. (1981). From methods to practice: a critical note in teaching of research practice to undergraduates. Sociology, 15, 505-512. Wild, M. (1999). Editorial: Addressing issues of culture and diversity in the application of new technologies. British Journal of Educational Technology, 30 (3), 195-199. Winn, S. (1995). Learning by doing: teaching research methodology through student participation in a commissioned research project. Studies in Higher Education, 20 (2), 203-214. Woodward, I. (1996). Continuing professional development: Issues in design and delivery, London: Cassell.
63
