Wiki-based Process Framework for Blended Learning Marija Cubric Senior Lecturer, Business School, University of Hertfordshire, UK +44 1707 285 546
[email protected]
providing wiki-spaces within the institutional virtual learning environments, but less so in educating the teachers and students on the effective use of the technology. With a few exceptions, currently published research on the institutional use of wikis does not include how the learning activities should be “shaped, planned or enforced” in a wiki [11]. In this paper we aim to fill that gap by providing a framework for learning and teaching process supported by the use of wikis.
Abstract With few exceptions, currently published research on the educational use of wikis does not include how the learning activities should be “shaped, planned or enforced” in a wiki [11]. In this paper we aim to fill that gap by providing a framework for learning and teaching processes supported by the use of wikis. An instance of that process framework ("feedback-driven" process) was formulated and implemented through a series of trials performed at University of Hertfordshire Business School during the course of the last two academic years to 2006/7. The results of the trial have been collected and analyzed using the quantitative and qualitative methods and have led to the conclusion that students’ engagement with wiki-based learning activities is directly proportional to the quality and frequency of tutor’s feedback and the clarity of the underlying learning and teaching process.
The main motivation for this work comes from the following pedagogical needs: •
to provide tutors with more regular feedback on students’ comprehension and progress;
•
to help tutors in identifying “troublesome knowledge” [14] and to enable them in reinforcing the understanding of those areas;
Categories and Subject Descriptors H.5.3 [Group and Organization Interfaces]: Computer supported cooperative work, Web-based interaction K.3.1 [Computer Uses in Education]: Collaborative learning.
•
to provide students with an open “structured bulletin board” [10] for reflection, meta products, analysis and feedback that is easy and fun to use;
•
to extend the official “contact time” to 24/7, by using other students as teachers, reviewers and role models;
General Terms Experimentation, Design, Human Factors, Theory
•
to focus on continuous feedback in order to respond to students’ needs and enhance their learning experience;
Keywords E-learning, blended-learning, MediaWiki
•
to facilitate acquisition of transferable and noncognitive skills and prepare students to be not only readers and writers, but also editors, reviewers and collaborators. To facilitate development of research, organizational, and negotiating skills [16]; to help student employability by preparing them for teamwork, global audience and peer reviews and in general for the new business model where “collaboration is the expectation rather than exception” [16]; to facilitate “connective writing” [16] with emphasis on criticality, clarity, structure and linkage; to provide support for different learning styles via an “inherently democratic medium” [10]; to support international students by providing examples of good writing; to reduce plagiarism by making students’ work public.
1. Introduction Since the mid 1990s wikis have been used in businesses and educational institutions as a tool that promotes sharing and collaborative creation of the Web content. The popularity of wikis was largely due to the meteoric rise of Wikipedia, the famous wiki-based on-line encyclopedia. With the beginnings of the Web2.0 movement, wikis have being promoted to one of the defining tools of the new Web, the one that will replace the “old-Web” ways of managing WWW content [13]. The educational institutions have largely reacted to this relatively new phenomenon by
•
• • •
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. WikiSym’07 October 21–23, 2007, Montréal, Québec, Canada. Copyright © 2007 ACM 978-1-59593-861-9/07/0010…$5.00.
•
11
These pedagogical requirements have been realized through implementation of a truly blended learning process that combines off-line and on-line activities and is “ founded on transparent communication amongst all parties involved with a course” [7].
2.1 Wiki as a communication and collaboration platform The first trial was conducted during the academic year 2005/6 with a group of 17 students studying “E-business system interactions” module. The objectives of the trial were to explore various ways in which wikis can be used to support students’ learning with the emphasis on using wiki’s communication and collaboration capabilities.
Theoretical foundations for this work are based on the following re-occurring themes from the constructivist learning theories: Vygotsky’s theory of social development [17] that defines learning process as a problem solving through “adult1 guidance or in collaboration with more capable peers”; Laurillard’s “conversational approach” to learning that defines learning as a continuous discussion with a student that helps in guiding the student towards the solution [9]; Gravett & Petersen’s “structured dialogue” approach to learning via carefully designed learning tasks [6] and Novak & Patterson’s “just-in-time” teaching method [12].
The trial consisted of weekly wiki updates by students in the context of the following learning scenarios: collaborative development of the module glossary, bibliography and FAQ list; post-lecture discussions on the current study topic; individual coursework (essay) development and open “multi-way” feedback (tutor to student, student to student, student to tutor) on the module wiki. Students’ work was assessed contributing towards 20% of their final grade.
In addition to that, this work was inspired by Tim BernersLee original idea of the “read/write web” where “students can work together … interact with simulations, with teachers, but particularly with each other” (T.Berners-Lee quoted in [3]) and Will Richardson’s description of the “classroom of the read/write web” [16].
The role of the tutor has been that of an observer, mediator and reviewer. The “scaffolding” provided by the tutor was limited to: creating initial page templates with simple guidelines on the content inclusion (e.g. “Add your entries in alphabetical order using the Harvard referencing style”); setting the topics for discussion (e.g. “Discuss reasons for e-commerce project failures. Support your answers with references.”); and providing feedback on the (coursework) work in progress.
In subsequent sections, we describe and evaluate two case studies based on the use of MediaWiki in University of Hertfordshire Business School curriculum delivery in two consecutive academic years (Section 2) that have led to the formulation of the wiki-based blended-learning process (defined in Section 3) and we conclude with a summary of benefits and issues in our current approach.
The outcomes of the trial were evaluated using a purposely created questionnaire and have shown the following interesting results. The Glossary entries were of varying quality, but have helped the students to get common understanding of the terminology used in the module (69% of students agreed that building a wiki based glossary has helped in their learning of the subject). Although many Glossary entries have been repeatedly modified, in majority of cases they were modified by the same person, giving the impression that students have established ownership on particular Glossary items and did not attempt to “step to each other’s feet”. In that sense the collaboration was limited to collaborative creation of the Glossary, but not individual Glossary items.
2. CASE-STUDIES The common characteristic of all trials was that they were based on weekly wiki (MediaWiki) updates by students that were triggered by tutor-set questions and assessed. The details of the assessment strategy for wiki contributions have been discussed in [4]. Each of the four trials was focused on a single, onesemester, post-graduate (MSc) module at the University of Hertfordshire Business School delivered in a context of 21 hours of (class-based) “contact-time” and 129 hours of “independent learning” time.
Students’ responses to the post-lecture topic discussions were ranging from non-objective opinions (10% of students), to fully referenced, objective, critical assessment of the topic (30% of the students were awarded extra marks because of the quality of their contributions). 69% of students agreed that on-line discussions have helped their learning and 54% of students agreed that the topics were challenging. In order to facilitate the assessment, all contributions were signed. That has lead to a “threaded discussion forum” style of contributions, rather than collaborative creation of “connected” and “link-rich” content. Other reasons could be attributed to students’
The common characteristics of the students groups in the modules included in the trials were: small class size (up to 17 students), large proportion of international students (100% in some of the modules) and relatively small percentage (less than 20% overall) of students with an ITrelated background.
1
Teacher
12
familiarity with the traditional linear style of discussion as well as the lack of guidance on how to collaboratively create content (see Section 2.1.1.1. for more discussion on this point).
and movement of pages; “recent changes” function for monitoring weekly contributions by individual students; “history” function and user contributions statistics - for assessment purposes. All contributions were added to “article” pages apart from reviews and feedback that were included in the corresponding “talk” pages.
The idea of on-line coursework development and using students as reviewers has been particularly interesting and has helped in enhancing students’ learning experience and quality of their work (65% of students awarded grades B or higher for their coursework). Despite initial “shyness”, 85% of students have eventually submitted their essays on the wiki; but 38% of students were not comfortable in making their work visible to other students. 62% of students have agreed that tutor reviews of their essays have helped them in improving the overall quality of the coursework, but surprisingly few students (38%) have agreed that peer reviews were helpful. This “lack of trust” could be attributed to the very mixed structure of the group (students from 10 different countries!) and the lack of group bonds (first semester together). However, this finding requires more experimentation and analysis before any further conclusions can be made.
In summary, this initial trial has helped students achieve better results in their final essay coursework, but has indicated many areas of “improvement”.
2.1.1 Lessons Learned This section includes discussion on the most important “lessons learned” that were identified in the first trial, presented in the order of importance.
2.1.1.1 Provide necessary “scaffolding” Many reports on the use of wikis have concluded that less “scaffolding” results in better quality of the created content [16]. However, we believe that some guidance on the content is essential, but the amount of scaffolding depends on various factors, such as: the study level (e.g. postgraduates or undergraduates), group composition (“digital natives” vs. “digital immigrants” [15], local vs. international students) etc.
It is interesting to notice that unlike in previous years, there were no attempts to plagiarize2, which we believe is due to completely transparent nature of students’ contributions and the subsequent feedback.
2.1.1.2 Provide more regular feedback
The rest of the scenarios used in the trial were mainly information pages, such as help pages on how to use and build wiki, students’ personal pages, coursework information and discussion pages. Students were told that all coursework inquires would be answered on the wiki, rather than through the individual e-mails and private discussions.
The results of the questionnaire show that students value tutor’s feedback and that feedback should be continuous rather than sporadic. Therefore, in order to increase students’ engagement the role of the tutor should be extended to that of an” active reviewer”.
2.1.1.3 Provide students with examples of “objective language”
The main issue encountered in this trial was a difficulty to engage students in wiki-based activities (53% of students made in-sufficient number of contributions and 54% of students responded that they wouldn’t be contributing to the wiki if it was not linked to the assessment). Similarly as reported by Lund and Smørdal [11] initial students’ interest had to be constantly fueled by new “incentives” (e.g. provide comments for improvement to your colleagues in order to get comments from tutor). Although the questionnaire did not include question on reasons for low interest, some possible reasons could be contributed to insecurities (the group consisted of 60% international students from various countries), but also to unfamiliarity with the concept of collaborative learning and the value of peer feedback.
As observed by Esenbach et al [5] “striving towards objectivity is a form of self-education”, however, students need guidelines for expressing “neutral point of view”, and that is an important pre-condition for collaborative editing.
2.1.1.4 Insist on quality of presentation Use of images and other visuals in wiki pages, could help in making the pages less “dull” and student work more interesting (only 50% of students had found wiki work to be a “fun” activity). Students should be actively encouraged to submit grammatically correct contributions.3.
Regarding the usage of wiki features the following special MediaWiki functions were used during the trial: deletion 2
3
HTML version of the wiki pages was fed into JISC’s TurnitinUK (http://submit.ac.uk) tool to check originality of the submissions and for the detection of plagiarism.
At the time the MediaWiki editor did not provide spelling check functions, but since then the spelling check of wiki text has been added to browsers such as Firefox 2
13
2.1.1.5 Insist on un-signed contributions4
2.1.1.11 Separate wiki administrator from teacher’s role
Signed contributions are appropriate for some types of work (e.g. discussions and feedback) but should be avoided in any type of collaborative writing.
Wiki philosophy assumes “flat hierarchies” [5] and therefore the tutor’s role shifts towards a learning collaborator and facilitator from the traditional one of an authority and a ruler. However, in order to enforce certain rules of behavior (e.g. delete content that does not conform to the page standard) teacher might temporarily “turn into” a ruler; it is advisable that those authoritative actions are perceived as coming from an anonymous wiki administrator, rather than from a “collaborator”.
2.1.1.6 Incorporate wiki in the assessment The questionnaire has confirmed the well-accepted theory that majority of students are assessment-driven and would not engage in activities that are not directly relevant to the assessment (e.g. 54% of students would not use wiki if it was not linked to the assessment). The inclusion of wiki in the assessment could further be justified by the fact that 69% of students think that use of wiki has helped them in learning. Students’ engagement and interest should be further extended by increasing the wiki-based assessment weight.
2.1.1.12 Use web-feeds (RSS) MediaWiki provide web-feeds in the form of a listing of “recent changes”. This is a very useful feature for all participants as it enables quick review of the latest updates without redirecting the browser to the specific wiki site.
2.1.1.7 Separate article page from talk page Formal division between article and talk pages facilitates two different styles of writing: collaborative content creation (article pages), and discussions, reviews and feedback (talk pages).
2.1.1.13 Disable anonymous access Anonymous access should be disabled in order to be able to trace and evaluate individual contributions, for example, by using the “history” function.
2.1.1.8 Provide early access to module wiki to all “stakeholders”
2.2 Wiki as a collaborative learning space The second trial was conducted during the academic year 2006/7 with a group of 27 students studying “E-business modeling”, “Developing Information Systems” and “Ebusiness system interactions” modules. The objectives of the trial were to implement enhancements in the process identified in the previous case-study with the emphasis on using wiki as a space for collaborative learning.
Internal and external moderators, authors and domain experts could provide valuable contributions to the wiki and students would benefit greatly from their involvement (“many, many teachers” [16]).
2.1.1.9 Use wiki not only from outside, but also inside the classroom
All activities that were used in the previous trial were employed in the new one, apart from the “On-line discussions” that have been replaced with “Topic Analysis” activity (Figure 9). The difference between the two scenarios is in emphasis on collaborative analytical writing, rather than “threaded” discussion. Two new learning activities were introduced: literature reviews, in order for students to provide evidence of reading related to the weekly topics (Figure 7) and in-class practical problemsolving work documented on the module wiki (e.g. diagrams for business process models, use-case diagrams, domain models etc).
This is a nice activity in a classroom and it reserves time for students who lack motivation or confidence, to focus on the wiki work.
2.1.1.10 Reserve enough time for introduction to wiki Despite the underlying simplicity, using wiki and understanding its “rules of conduct” (e.g. “wikietiquette”) still need to be learned [5]. Wikipedia provides very good resources that can be recommended to students for reading before the start of the module. Especially important is to introduce students to the concept of “concurrent updates” and demonstrate what happens when two or more users attempt to edit the same page at the same time.
4
Please notice that “un-signed” contribution is different from “anonymous” contribution: the author of an “un-signed” contribution is a registered user, and the authorship of an “unsigned" contribution can be traced using the “history” function.
14
Table 1. Comparison of the trial results Question Wiki was easy to use Wiki supported me in preparing the coursework Wiki has helped in my learning of the subject Participation in topic analysis has helped my learning Reading & reviewing other students work has helped my learning Tutor feedback has helped me in improving the overall quality of my work Other students reviews of my work have helped me in improving the overall quality of my work I was comfortable in making my work visible to other students I would have used the wiki even if not directly linked to assessment I would recommend the use of wiki for this module in the next academic year Using wiki was fun
st
nd
1 trial 77%
2 trial 73%
54%
80%
69%
93%
69%
87%
69%
87%
62%
100%
38%
80%
62%
80%
46%
67%
69%
93%
46%
67%
•
Similarly to Wikipedia’s practices, entries that did not conform to standard (e.g. Harvard referencing) were removed after a short grace period. Students responded positively to this measure by eventually submitting the corrected content;
•
Use of wikis inside the classroom has proved to be an interesting compliment to the lecture and has enabled instant access to students’ work for the purpose of class discussions and reviews (2.1.1.9);
•
Last but not least, extensive use of images and other visuals have made the pages much more interesting and easier to follow (2.1.1.4).
trend
The outcomes of the trial were assessed using the same questionnaire as in the previous trial and have demonstrated improvement in all areas except in “ease” of use (Table 1). This might have been caused by increased tutor’s expectations regarding the presentation aspects of students’ contributions (e.g. use of internal and external links, use of images etc). Regarding the assessment results, 32% of students have made insufficient number of contributions, and 34% of students have been awarded grades B or higher for the overall quality of their contributions. 61% of students have been awarded grade B or higher for the work that they have developed on the wiki using the feedback from tutors and other peers (e.g. essays and design documents).
All of the “lessons learned” identified in the previous trials have been addressed in the new trial, and in particular the following “interventions” have made the most significant impact on the positive outcomes of the second trial: •
•
Students’ feedback on the use of wikis included in the final module reflection was very encouraging. Some typical positive responses are quoted below5:
The “scaffolding” provided by the tutor (2.1.1.1) was extended to include more guidelines on content creation e.g. provided examples of good literature reviews, guidelines on collaborative and “connected” writing, as well as guidelines and techniques on “objective” writing (Figure 8)
“The Wiki page is a good part of learning because it encourages class members to integrate, which was quite helpful…” “I learnt a lot from other people's input, whenever I got stuck alone, I would hop onto Wiki page and get the answers I needed…”
The role of the tutor has been extended to that of “active reviewer”. A weekly work of the entire group was reviewed (2.1.1.2) and feedback and points for improvement were discussed in-class with students. This has helped in particular in addressing the areas of “troublesome knowledge” [14] in more depth.
•
To facilitate collaboration, all contributions were unsigned (2.1.1.5) but to make the assessment work easier, students were asked to provide weekly “work records” (Figure 10) linking to the submitted content;
•
Wiki work has contributed directly to 40% of the final grade, and indirectly (development of essays or design documents on the wiki) to further 30%-60% of the final grade (2.1.1.6);
“Looking at my input on Wiki made me feel quite good, because I could see my input to the whole course.” “(Use of wiki) made us concentrate more in class and read more about the topic in order to know how to answer the
questions, therefore, understand things better” “The things which I have learned beside this module is that how should I conduct my work in organized form.” “Knowledge and skill I gained from using Wi-ki page will be useful for me in the future because Wi-ki page ca be use not only in studying purpose but also in business purpose. It could be use inside a company allow employee to share and exchange information need in business process.” 5
15
All reflections are included in their original form.
“I also learned how to reach to a single conclusion or decision….because our work is posted and everyone can read, this encouraged us to gain more knowledge in order to have a better quality of posted material.” “The structured wiki activities provided a core to the course and ensured an in depth coverage of the deeper areas of the course. It encouraged me to look into specific areas in greater depths and learn from the contribution of others and my own research. From reviewing the contribution of others it was also possible to better understand the dimensions of a topic and also some of the inherent ambiguities. It also presented a challenge to re write information to be more succinct and focused.” “This module is become more interesting because the use of wiki as a sharing, research, and studying medium among the … students….The weekly tasks allow us to refocus and encourage us to do more work or further reading about a particular subject each week.” “Initially, I think working with the wiki is waste time, but now I find the wiki can help me to increase reading and practice what I have learn. I quite enjoy work within wiki.” “Wiki is very nice tool. My group used it to help in developing (coursework) website. We posted all information about the website there. And it is easy to access. We can track final version of work with out sending email again and again. I think this tool is really useful in my future. I will use it.” “As a class I believe everyone enhanced their writing skills as well as Harvard Referencing style.” “I think it is very wise to spread out assessment in the wiki and the coursework, so students who are weak in one assessment will be able to do their best in another assessment.”
each student will have to do more in dept reading and research.” “… for me who have poor English skill, I like to read more than add, and I feel scar when I need to delete some thing there. However, I edit something since it quite wrong.” I suggest that student should study the next lesson and post the background knowledge to wiki before go to class. I believe that it will improve understanding of students.” “Other thing is group page that allow students post what they want, no construction, no serious in there. It’s kind of common room that they can use the tools to communicate to each other.” The results of the questionnaire and the assessment have both demonstrated more active students’ engagement, better understanding of the importance of feedback and more appreciation for the collaborative learning requirement. Most importantly, they have shown that wiki activities need to be planned and structured with care in order to engage students and make positive impact on their learning experience. This second trail paved the way for a conceptualization of the underlying learning and teaching process, supported by wikis. We describe that process in the next section, as an instance of a more generic framework and we provide examples of alternative processes that can be instantiated based on the same framework.
3. PROCESS FRAMEWORK A “blended learning process framework” defines basic learning activities, location of the activities (in-class, outside of class), ownership of the activities (individual or group), guidelines for completing the activities, grouping of related activities into learning tasks, frequency of the tasks, teacher’s and learners’ responsibilities relative to the activities and task, and technologies used to support the completion of the activities.
Students’ reflections have also included some interesting observations on alternative uses of wikis in future module deliveries:
A “learning activity” could be either of the following:
“…there is advantage from the wiki work, but in my opinion there is also disadvantage or I should say weakness. The last person who want to contribute to the weekly topic would find themselves have to do extra work than other person who has contribute earlier. But then again, it will benefit other students because more depth of information and analysis are provided by doing so.” “…to improve wiki effectiveness is maybe to make separate pages for each student to contribute to the weekly topic. Therefore it might become … fairer. One way to overcome this is to assign each student to a different sub topic from the weekly topic. Therefore,
16
•
Add definition of one or more items to the module glossary
•
Provide solution to a practical problem (e.g. develop diagram solution, program solution, informal description of the proposed solution)
•
Add (referenced) contribution to the weekly topic analysis
•
Develop coursework
•
Review an article/web-site/standard relative to the weekly topic
•
Review the work of your colleague and provide comments.
include 21 hours of (class-based) “contact-time” and 129 hours of “independent learning” time.
The activities listed above map to increasing levels of competencies in Bloom’s “cognitive domain” taxonomy [2] with “Glossary contributions” being at the first level (“fact recollection”) and reviews being at the top level (“evaluation”).
The process progresses through a series of weekly iterations, where each iteration consists of the following ordered set of activities:
A learning activity can be developed in-class, outside of the class, individually or in groups. Each of the learning activities is accompanied with a specific set of “guidelines” provided by the tutor. For example, for the “Topic Analysis” activity, students are advised to work together to formulate an in-depth analysis of the topic and to link the content to the internal “Literature review” and “Bibliography” pages as well as to external glossaries. They were also provided with links to some useful examples of academic writing [16].
•
Day 1 (lecture day): After the lecture is delivered, tutor publishes a “weekly task” on the module wiki, relative to the lecture topic.
•
Day 2 –Day 5: In response to the weekly task, students add individual contributions to the module wiki and update their “work record” on the wiki.
•
Day 6: Tutor reviews weekly wiki contributions and provides group feedback. The feedback is documented on the module wiki and serves as a basis for topic review discussion in the next lecture (day 7).
The state-transition diagram of the weekly process is displayed in Figure 1 where the following colour coding6 is used: blue, yellow and green states correspond to teacher’s, students’ and combined activities respectively. Similarly, all on-line activities are marked with the MediaWiki logo.
A “weekly task” is a set of related learning activities (e.g. grouping can be based on the subject topic, specific casestudy, difficult concept etc) set by the tutor and completed by students outside of the classroom. A weekly task “status” can be: “not-started”, “in-progress” or “completed”. A “work record” is a record of student’s weekly activities relative to the pre-set weekly task, maintained by the student.
Task published
A “task feedback” is the feedback that includes strengths and points for improvement for a particular completed task. Feedback is provided by tutor or students, individually, per group or per class and can be provided in-class (“face-toface”) or outside of the classroom (“on-line”).
START
An “assessment” is an evaluation of a task (or group of tasks) completed individually or in a group, that is based on particular set of grades and grading criteria.
feedback
“Wiki-based framework” for blended learning is a process framework that uses wiki as a platform for publishing weekly tasks, completing weekly tasks, maintaining work records and providing feedback.
Contrib’s added
Lecture delivered
Contrib’s reviewed
Figure 1 Feedback-driven L&T Process
A “blended learning process” is a particular instantiation of the process framework that prescribes ordering and frequency of learning tasks, ordering and frequency of the feedback, specific assessment criteria as well as context such as duration, number of contact hours, study level etc.
In addition to the weekly tasks, the process includes the following special activities: •
Week 1: Students familiarize with simple wiki editing, wiki “etiquette” (as defined in Wikipedia) and module assessment strategy.
6
Alternatively, text in bold, italic and bold-italic font style correspond to the teacher’s, students’ and combined activities respectively.
In the next sub-section we describe the blended learning process implemented in the second trial, as well as several alternative processes that are all instances of the same wikibased L&T (Learning and Teaching) framework.
3.1 Feedback-driven L&T Process The wiki-based L&T process used in the second case study has been defined in a context of 11 teaching weeks that
17
•
Week 3: Tutor provides short individual feedback to each student via e-mail underlying areas for improvement in their wiki work.
•
Week 6: Tutor provides another short individual feedback, but this time it includes grade prediction e.g.” the work so far has been of grade B standard” .
•
Task published
Week 8 –onwards: Students start to develop the rest of their coursework “incrementally” on the module wiki using “transactional” writing style (Glogowski K. quoted in [16]) – writing based on comments and feedback (e.g. trigger, write, feedback, reflect cycle), where the feedback is provided by tutor or peers
•
Week 10 – onwards: Students and tutor provide feedback (on the wiki) to their colleagues on their work in progress.
•
Week 12: Students submit final wiki contributions.
Contrib’s added
Lecture delivered
Contrib’s reviewed
feedback
Figure 2 On-line L&T Process START
The core of this process is frequent and regular feedback, in particular - the detailed weekly group feedback that is used as a basis for re-visiting those areas of core knowledge that students had difficulties with, supplemented with short individual feedback (weeks 3 and 6) for the purpose of motivating further students’ engagement and directing better their efforts.
Task published
Contrib’s added
Lecture delivered
3.2 Alternative Processes The same framework can be used to instantiate alternative “feed-forward” JITT (Just-in-Time Teaching) process (Figure 3). The only difference with the process already described is in the starting point of a weekly iteration i.e. the “weekly task” is published before the lecture (“feedforward”) and students are asked to complete their contributions on the new topic rather than on the topic already covered in class.
Contrib’s reviewed
feedback
Figure 3 Feed-forward JITT Process
That enables the teacher to gather “students’ conceptions just-in-time to help re-shape or guide the up-coming lecture” [17].
Task published
A variation of the described processes can be used for delivery of on-line modules (Figure 2). For example, feedback discussion could be performed on the module wiki or in combination with web conferencing tools such as Elluminate (.http://www.elluminate.com/).
Lecture delivered
Teacher’s and students’ roles and ownership of activities can vary resulting in a variety of learning and teaching processes, ranging from more traditional models (e.g. Figure 1 and Figure 3) to the learner-centric models such as “facilitative learning” where the role of the teacher is changed to that of a “facilitator”, ready to receive and accept constructive feedback [8].
feedback
Contrib’s added
Contrib’s reviewed
Figure 4 Facilitative and collaborative learning process
18
Furthermore, according to Richardson [16] use of collaborative technologies such as wikis and blogs, creates a shift in the role of teachers to not only facilitators but to collaborators and “change agents” (Figure 4).
-
Highlight all contributions by a specific user on an individual page or selected set of pages
-
Provide word counter for a “highlighted”) content on a page
4. FUTURE WORK
-
Provide text matching capability for a selected (or “highlighted”) content on a page (for the purpose of detecting plagiarism).
In this section we discuss some open questions and areas for future work such as: whether and how to assess wiki contributions, whether or not to make wiki open to users outside of the module group, how to scale the approach for large groups, what to do with the content once the module is finished and how to use the historical information available for each wiki page.
•
4.1 Assessment dilemmas
selected
(or
“Work collaboratively but (be) assessed individually” the two requirements are to a certain extent contradictory and make the assessment of individual contributions difficult. However, implementation of some of MediaWiki enhancements listed above could ease this problem as well.
4.2 Externalization
Assessing students’ wiki contributions is an important driver for their engagement, however, not an easy task for the tutor and even more difficult one for the moderators and external examiners. MediaWiki support for assessment is limited to “history” and “user contributions” pages and the corresponding “compare versions” and ”diff” functions. However, for updates submitted over a period of 10-15 weeks, and by a group of 10 or more students, traceability of individual contributions quickly becomes an unmanageable task for the tutor, and a practically impossible task for internal and external moderators. During the trials we have tried several different assessment strategies [4], each of which has been designed to remedy certain deficiency in the previous approach. However, the following assessment dilemmas remain to be re-addressed in future trials:
Richardson’s suggestion on “expanding the walls of the classroom” [16] by extending wikis over institutional firewalls and opening it to external contributors’ has many obvious merits, but also it poses certain threats. Most obviously, the potential for malicious use is certainly increased, and creates additional work load for the tutor (to monitor, revert to previous version, block users etc). But also, the role of the tutor becomes less clear (facilitator? moderator?) in the presence of “many, many teachers” [13] and that could lead to confusion. A plausible alternative is to open wiki to a registered set of external users such as: internal and external moderators, textbook authors etc. This opens up another question i.e. what is the motivation and incentive for engagement of these external groups.
•
4.3 Scalability
•
“To assess or not to assess” – ideally, wiki contributions should be assessed for formative (practice), rather than summative (grade) purposes. The danger however is that students will fail to engage and therefore not gain any benefits from the approach: if students do not submit contributions, tutor cannot provide the feedback. Alternative approaches could be to make wiki contributions pre-requisites for formal assessment or to make the content developed on the wiki available to students during the formal assessment?
Both of our wiki trials have non-intentionally dealt with small group sizes (to the maximum of 17 students in the first trial). The natural question to ask is - how to scale up our approach to support large groups? For example, should all users be registered? Is it possible to assess individual contributions, or should group contributions be assessed instead? Should the overall module wiki space be divided into separate tutorial wikis? And most importantly, would increase in group size mean that the process undertaken was not possible because of the increased workload on the teacher? Current wiki trials in the UH Business School, involve several large students groups (up to 180 students) and the results of these trials will provide more insight into the methods that work and don’t work with large groups.
“Assess weekly or assess at the end” – weekly individual assessment would provide more detailed account of students work and enforce regular contributions, however this approach creates significantly increased marking load, especially with groups of 10 and more students. Assessing overall quality of students’ work (at the end of a semester) is a more practical task, from the tutor’s perspective, and with the following MediaWiki extensions it could be made more manageable:
It is important to notice that the weekly feedback, which is the core element of our approach, is relative to the collaborative work of the entire group, rather than individual contributions and therefore independent of and equally applicable to any group size.
19
Regarding the further MediaWiki support, the following would be valuable general extensions or enhancements8 relative to the existing set of features:
4.4 Retired Content Finally, what to do with the content of the module wiki, once the module is finished? With additional MediaWiki support the content could be exported into PDF or HTML format7 and distributed to all participants of the module for future referencing. It could also serve as a model and point of reference for the next year cohort, or being added to some global repository of learning resources to serve the larger learning community (educational Wikipedia?)
•
Better navigation capabilities (e.g. easier inclusion of page links in the left-hand side navigation bar);
•
Better support for discovery and repair of broken links;
•
Support for internal image re-sizing.
The following MediaWIki extensions would be beneficial for the assessment of wiki contributions:
4.5 Personalization The historical information available in each wiki page can be used by “data analyzers” and “pattern finders” programs and tools to discover the learning patterns of individual students and subsequently to adapt the weekly tasks to personal development needs. For example, the tool could discover that a particular student is always amongst the first ones to add the content, but rarely participates in adapting or extending the existing content. The tool could then provide the student with a specific task to deal with the specific deficiency (e.g. “Extend the argument in this analysis …”). This idea was discussed by Aroryo and Dicheva [1] in the context of general web-based educational systems but with no specific references to wikis.
•
Configuration of different “skins” for different (sets of) pages - this will allow students to use personalized “look & feel” for their coursework submissions;
•
Page export to different formats such as PDF or HTML for the purpose of archiving (“retiring”) entire wiki site; or to ASCII text format for the purpose of feeding it into text matching tools
From the pedagogical perspective, our main conclusions are:
5. CONCLUSIONS In this paper we have described and evaluated two consecutive trials of the use of wiki technology as a support tool for curriculum delivery and assessment, as well as for students’ learning. The trials have provided a substance for formulation of a wiki-based learning and teaching framework that can be used to instantiate a range of blended-learning or e-learning processes, based on the parameters such as: level of study, methods for teaching and learning (e.g. learner-centric vs. teacher-centric), learning styles, ratio of off-line and online activities etc. One of the described processes (“Feedback-driven L&T Process” in Section 3.1) has been implemented and evaluated in the second trial. It is important to emphasize, that the frameworks and the example processes are independent of the subject area.
•
Students’ engagement with wiki-based learning activities is directly proportional to the quality and frequency of tutor’s feedback, clarity of the underlying learning and teaching process and weight of the activities in the formal (“summative”) assessment.
•
The “contract” between learner and tutor should include incremental “value-added” targets supported by carefully planned learning tasks, rather than one final “verdict” in the form of final exam and final grade.
6. ACKNOWLEDGMENTS This work has been supported by University of Hertfordshire Learning and Teaching Development Fund for 2005/6 and (UK) Higher Education Academy (Subject Centre for Business Management Accountancy and Finance) Teaching Research and Development Grant for 2006/7.
7. REFERENCES [1] Aroyo L. and Dicheva D. The new challenges for e-
The benefits of wiki as an e-learning tool are well known and documented in [5], [10] and [16]. In addition to that we would like to emphasize its suitability for development of those non-cognitive skills that are increasingly required by employers: giving and receiving feedback, working towards consensus, easy creation of web content etc
learning : the educational semantic web. Educational Technology & Society, 7(4), 59-69, 2004 [2] Bloom B. S. Taxonomie of educational objectives,
Handbook I: Cognitive domain. New York: David McKay Co Inc.1956
The choice of MediaWiki engine was based on the assumption that students would be familiar with its basic features and page layout because of its use by Wikipedia.
[3] Carvin A. Tim Berners-Lee: Weaving a Semantic Web,http://www.andycarvin.com/archives/2004/09/tim_bern erslee.html 2005
7
8
External tools for mapping the content of a Mediawiki site to an HTML or PDF document are already available,
20
Some of the extensions could easily already be implemented, but not trivial to use or configure
[4] Cubric, M., Using wikis for summative and formative
APPENDIX – Samples of wiki pages
assessment, Re-Engineering Assessment Practices (REAP) International Online Conference, May 2007 [5] Ebersbach A. et al Wiki : Web Collaboration,
Springer, 2005 [6] Gravett S. and Petersen N. Structuring dialogue with
students via learning tasks, Innovative Higher Education, Vol. 26, No. 4, 2002 [7] Heinze A., Procter C., Reflections On The Use Of
Blended Learning, Education in a Changing Environments, Conference Proceeding, 2004 [8] Laird, D. Approaches to training and development
Addison-Wesley, Reading, Mass. 1985 [9] Laurillard D.
Rethinking University Teaching: a framework for the effective use of educational technology - 2nd edition. London; RoutledgeFalmer, 2002
Figure 5 Main Page
[10] Leuf, B., Cunningham, W. The Wiki Way Quick
Collaboration on the Web, Addison Wesley, 2001 [11] Lund A. and Smørdal O. Is There a Space for the
Teacher in a WIKI? WikiSym’06, August 21–23, 2006 [12] Novak J. and Patterson E. Just-In-Time Teaching:
Active Learner Pedagogy With WWW, IASTED International Conference on Computers and Advanced Technology in Education , May 27 -30, 1998 [13] O'Reilly T., What Is Web 2.0 - Design Patterns and
Business Models for the Next Generation of Software. http://www.oreillynet.com/pub/a/oreilly/tim/news/200 5/09/30/what-is-web-20.html, 2005 [14] Perkins, T.D. The many faces of constructivism,
Educational Leadership, 57 (3). 1999 [15] Prensky
M. (2001) Digital Natives, Digital Immigrants, On the Horizon, NCB University Press, Vol. 9 No. 5, 2001.
Figure 6 Weekly Task Page
[16] Richardson W. Blogs, Wikis, Podcasts and other
powerful web tools for the classroom. Corwin Press, 2006 [17] Russell M. Preliminary Exploration into just-in-time
teaching. Journal for the Enhancement of Learning and Teaching, Volume 3 Issues 2, University of Hertfordshire Press, 2006 [18] Vygotsky, L.S. Mind in Society: The development of
higher psychological processes. Cambridge, MA: Harvard University Press, 1978
Figure 7 Literature Review Page
21
Figure 8 Topic Analysis Page
Figure 11 Coursework Information Page (all names changed)
Figure 9 (Weekly) Topic Analysis Page
Figure 12 Essay feedback page (all names changed)
Figure 10 Individual Work Record Page
22
