Department of Information Science, University of Bergen,. Box 7800, N-5020 ... Table 1: Research Questions for Master's theses related to the VisArt scenario.
Position paper submitted to workshop on «Evolving use of groupware» at ECSCW’99 Copenhagen DK, 12/9-99
Dynamics of groupware use in a collaborative telelearning scenario Anders Mørch & Barbara Wasson Department of Information Science, University of Bergen, Box 7800, N-5020 Bergen Norway {anders, barbara}@ifi.uib.no
Project DoCTA DoCTA (Design and use of Collaborative Telelearning Artefacts) is a multidisciplinary research project that focuses on the design and use of technological artefacts to support collaborative telelearning. The research is not limited to only studying these artefacts per se, but includes social, cultural, pedagogical and psychological aspects of the entire process in which these artefacts are an integral part. This means that we both provide and study virtual learning environments that are being deployed to students organised in geographically distributed teams. From a research perspective, the exploratory studies being carried out within DoCTA will provide us with insight into the processes of collaboration enabling us to identify collaboration patterns. The community of study includes teachers, students and facilitators participating in three collaborative telelearning scenarios (IDEELS, Demeter, VisArt). The main research question has been formulated to ask how these students, teachers and facilitators organise their work. In order to collect data about the activities in which the student are engaged, different methods and techniques are being used. The most important sources of information are derived from observing the students as they collaborate using artefacts, from interviewing them, from the artefacts they create/design, and also from electronic logging of the artefacts used for collaborating (e.g., email, whiteboards, chats, to-do-lists, postit notes). A number of the research questions being asked are relevant to the workshop focus. In particular, one study (Guribye, 1999, Wasson & Guribye, 1999) of the IDEELS scenario has used Activity Theory to analyse how a team of students divided their work, and how this work was mediated by the various collaboration technologies they had access to. In this study both Engeström’s (1987) activity triangle and Leont’ev’s (1978) levels of activities were used as analytical tools. Currently our work is focused around the VisArt1 scenario. The VisArt scenario involved students taking courses at three educational institutions (University of Bergen, Nord-Trøndelag College, Stord/Haugesund College) in Norway. The VisArt activity took place during February
1
VisArt is an abbreviation for Visual Artefact; the kind of artefact students created with TeamWave
Position paper submitted to workshop on «Evolving use of groupware» at ECSCW’99 Copenhagen DK, 12/9-99
and March 1999. VisArt began with one week of training in using TeamWave Workplace2 (TW) and in team collaboration. This was followed by a three weeks design and collaboration activity where the team was to design a learning room in TW. Each team comprised three students, one student from each institute. There were no opportunities for the teams to meet face-to-face. TW was used as the main information and communication medium, and was supplemented with email. Figure 1 shows one of the learning rooms designed by Team07.
Figure 1: Team 7’s room for learning about polar bears. This room is part of a series of rooms constructed for the purpose of teaching elementary school children about polar bears
Two VisArt studies are using Activity Theory (one focused on teams of students and the other on the instructors and facilitators). We are supplementing this overall focus on collaboration patterns and how the participants organise their work, with several studies that will focus on evaluation of the technological environment. For example, one thesis is focusing on the usability of TW by monitoring how students’ opinions about the utility of the tools changes from their initial impression to a final judgement after a semester of use. Another thesis is focused on the analysis of TW data logs to look at how the different artefacts were used for 2
http://www.teamwave.com
Position paper submitted to workshop on «Evolving use of groupware» at ECSCW’99 Copenhagen DK, 12/9-99
collaboration activities. A third thesis is concerned with coordination. Table 1 below gives an overview of the research questions being asked in the various studies. Table 1: Research Questions for Master’s theses related to the VisArt scenario Domain/theoretical framework Usability of TeamWave Workplace
Research Question
Usability – effectiveness
Can students working in collaboration reach their team goals with TW?
Usability – efficiency
To what extent do the tools enable students to meet their task?
Usability - satisfaction
Are students satisfied of TW tools?
Efficiency from a qualitative perspective Comparison of anticipated use and actual use of the tools
What are the differences between the anticipated use of the tools and the actual use of the tools? Which are the differences of tool use when alone and when others in the room? How are the rooms used?
Research methodology
What implications do the use of the tools have for electronic data-collection?
Formative evaluation Tailoring for instructional design
How to support pedagogical room design in TW?
User centred design
How do students use TW tools to design a room for teaching? What is to be improved?
Coordination theory Supporting coordination
How to support coordination in a collaborative telelearning environment?
User centred design
What kind of coordination do students do?
Activity Theory Organisation of work – student perspective
How do the students organise their work? How instructors and facilitators organise their work
Organisation of work – Instructor’s perspective
Design of training and assistance Assistance design/ improvement
How effective is the training designed for collaborating and learning to use TW? How to improve help and assistance?
Theoretical discussion
Position paper submitted to workshop on «Evolving use of groupware» at ECSCW’99 Copenhagen DK, 12/9-99
In order to study human activity and evolution of technology in VisArt, we take as a starting point the «use of technology» and not the technology per se. Our working hypothesis is that the interplay between change of technology and change in activity (through the use of technology) is significant. We thus design our case studies to explore this interplay in various ways. Whether or not change of technology and change in activity can be explained within the same conceptual framework is an open issue that we would like to discuss. Our experience with Activity Theory is that it provides a lens through which we can see the world. It is useful to orient thoughts and research questions. It provides a number of methods/tools shaped by a general theoretical approach. It can be used to guide methodological decisions regarding evaluation (how technology is used) and to a lesser extent regarding design (how technology will be used). In fact, from an Activity Theory perspective we should always talk about redesign because we are "redesigning" work/learning praxis. Activity Theory reminds us to always focus on context and suggests the articulation of the design of artefacts as «change in activity». From this perspective activities evolve and change and artefacts evolve and change. As Susanne Bødker has been heard saying, «the work of the user is not to use the information system (using an artefact is not an activity) rather work role brings artefact into the activity; gives it a context. We interpret this to mean that Activity Theory needs to be integrated with a design perspective. The perspective should be that design is seen as continual redesign, a kind of artefact evolution. The rationale for this is that evolution of artefacts needs to be informed by the practice carried out by users when they use the artefacts in genuine human activity. Otherwise the redesign will be of little practical value. The diagram in Figure 2 is an attempt to put Activity Theory in such a context in order to help us focus our discussion. constrains/ enables
Evolution of Artefacts
Use of Artefacts
breakdown
breakdown action
operation
reframe requirements
work role
Human Activity
reframe
Meaningful tasks
division of labour
Figure 2: Interplay between evolution of artefacts, use of artefacts, meaningful tasks, and human activity in an attempt to integrate a design perspective with Activity Theory
From evolution of artefacts to meaningful tasks (and back again) Figure 2 can be seen as an extension of Carroll and Rosson’s task-artefact cycle developed in HCI (Carroll et al., 1991). The task-artefact cycle describes the interdependence of tasks and artefacts in human activities. There is a reciprocal relationship between the two. When tasks
Position paper submitted to workshop on «Evolving use of groupware» at ECSCW’99 Copenhagen DK, 12/9-99
change, artefacts tend to change as well. Carroll and Rosson give as an example the evolution of writing tools. When the typewriter was replaced by the word processor this was not only an evolutionary change in technology — it also changed the task of writing. For example, with a word processor it is now possible to correct a misspelled word without the reader ever knowing that it was once misspelled. What consequences this have for the evolution of groupware and whether or not we can find similar or other kinds of relationships between the other components in the diagram is something we would like to explore. From use of artefacts to evolution of artefacts (and back again) In the beginning of the DoCTA project we attempted to tailor TW to create our own specialized version of it. This soon came to an end due to difficulties we had with Groupkit toolkit and its scripting language (Tcl). Instead the developers of TW tailored it for us and provided us with new releases of the software. The kind of evolution we saw during the scenario was evolution of the artefacts the users created themselves; that is the artefacts they created with the TW tools. We can call these artefacts 2nd level tools to distinguish them from the (1st level) tools developers used to build and tailor TW. The 2nd level tools are first of all the specialized rooms the students created (see Figure 1) as well as the tools they used to accomplish this (chat, postit, to-do-lists, etc.). Each room can be specialized to a particular purpose and filled with content specific to that purpose. The purpose can change over time. In addition new rooms can be created and old ones deleted. One interesting use of a TW tool is the student’s use of the postit note (like a yellow postit). This tool was used from leaving messages for one another or for oneself, to giving directions and finally as a “chat” tool. Open issues for discussion Collaborative telelearning seems to work from a technological point of view. Our students were able to use many of the TW tools and e-mail without much difficulty (e.g., chat, postIt notes, shared artefacts). Following the use of the tools over the scenario has been interesting as they moved back and forth between the tools depending on their needs. Our findings regarding the ease of use of many collaborative tools tell us that the technological problems are no longer the prime issue in CSCW/groupware design. The main issues now seem to be related to the broader organizational contexts in which the tools are designed and used (cultural, sociological, pedagogical, and psychological). At this stage we are not sure how to evaluate the success of our project. We need to examine our results from several (maybe conflicting) perspectives. Questions we would to raise for discussion include: • How can Activity Theory be used in a prescriptive manner? • What do our narrative descriptions of learning activity tell us for the design of future collaborative technologies? • How should/can data be collected when the participants are distributed over wide geographical areas? • How do we measure (if at all) the quality of the products they have produced?
Position paper submitted to workshop on «Evolving use of groupware» at ECSCW’99 Copenhagen DK, 12/9-99
• How can we design for spontaneous, instant, and informal use, since this seems to be the preferred way for students to use collaborative tools (e.g. chat)? • How to integrate product-oriented tools (e.g. applications) with informal collaboration tools? • How can we create activities to take advantage of the way tools are integrated? • How should actors organize their work to take advantage of the way tools are integrated? • How do we encourage creative (spontaneous, instant, and informal) learning? • Can creative learning activities be designed for or should it emerge as a result of practice? • What is collaborative telelearning practice? • What are conditions for creative activities to emerge? • How do we measure what students have learned? References Carroll, J.M., Kellogg, W.A., & Rosson, M. B. (1991). The task-artefact cycle. In J. M. Carroll (Ed.) Designing Interaction: Psychology at the Human-Computer Interface, 74102. Cambridge, UK: Cambridge University Press. Engeström, Y (1987). Learning By Expanding: An activity-theoretical approach to developmental research. Helsinki: Orienta-Konsultit Oy. Guribye, F. (1999). Evaluating a collaborative telelearning scenario: A sociocultural perspective. Masters dissertation, Department of Information Science, University of Bergen. Guribye, F. & Wasson, B. (1999). Evaluating collaborative telelearning scenarios: A sociocultural perspective. In (Eds.) Proceedings of Educational Multimedia & Educational Telecom ‘99. Charlottesville, VA: AACE. (Outstanding Paper Award). Leont’ev, A. N. (1978). Activity, Consciousness, Personality. Englewood Cliffs,NJ: Prentice Hall. Biography Anders Mørch is an Associate Professor in Information Science at the University of Bergen. His research interests are: CSCW, HCI, End-User Tailorability, Pedagogical Agents, Computer Supported Collaborative Learning (CSCL), and Computer Supported Collaborative Design. He is co-editor of the JCSCW special issue on «Tailorable Systems and Cooperative Work» to be published in 1999. Barbara Wasson is a Professor of Pedagogical Information Science in the Department of Information Science, University of Bergen, Norway. Her research interests include collaborative telelearning, advanced IT Support for Teaching and Learning on Campus, Artificial Intelligence Applications in Education, Instructional Planning, and Pedagogical Agents. She is project leader for DoCTA.
