situations of others and to reflect, in a non-threatening setting, on the nature of their own social location .... The interview protocol followed this basic outline: ... Strategy games can be further divided into six categories: Adventures, D&D games,.
Using Sociological Research In Educational System Design By Angela Coco, Gavin Edie and Lesley Jolly
Background Currently, one of the most difficult problems facing students, teachers and their parents in educational settings is the high and growing incidence of bullying. A plethora of materials has been developed to assist people to deal with bullying but a review of the literature reveals that most approaches are didactic in nature. That is they take the teacher’s, parent’s or designer’s (expert’s) authorative point of view and give instructions about the best ways to deal with issues (Damarin 1993). Situated learning needs to proceed from the point of view of the learner (Brown and Duguid 1993). Therefore an approach built upon the experiences of those people who actually bully and experience bullying should form the basis of a learning tool that helps students devise their own strategies for working out problems with bullying. Our pilot program is designed to address the bullying experiences of twelve-year olds. We are building a prototype to assist students to think through consequences of options for actions (with respect to situations of bullying) in a safe and supportive environment. That environment can be provided by mediated experiences in a virtual world. This strategy provides a problem based approach to the learning task as students are ‘virtually’ immersed in situations where they need to make choices for action, see these choices enacted by virtual characters, reflect on the consequences and decide upon further or alternative courses of action. The challenge for design occurs when we ask how to make the content of the virtual world relevant to students’ personal experiences. Some designers have begun to engage the methods of sociologists, particularly ethnographers, to understand the social contexts in which computer systems are used, and the research contexts, such as collaborative interdisciplinary partnerships out of which they are created (cf Crabtree et al. 1998; Rouncefield et al. 1997; Viller and Sommerville 1997). These studies generally address interface issues, how to design an architecture and notation that is user friendly and takes into account the interactional dynamics of the lived environment. In many studies the latter are referred to as ‘situated learning’ (e.g. Crabtree et al. 1998; Hughes 1998) but the situated learning we have in mind is not the place in which the computer and student interact. (This is an important feature of many reported educational software and work situation programs but for this project the place can vary, as students may access the program from anywhere they can gain access to the web). We refer to ‘situated learning’ from the view of educational theory where students undergo ‘authentic activity’ in which they solve problems that help them develop their lifeskills (Hughes 1998). Most readers would acknowledge that one of the most helpful strategies for working through a difficult situation is finding someone else who has experienced the same situation who will tell us about it. Identifying with their definition of the situation, their feelings about the event and the actions they took or didn’t take helps us to work out what strategies would be successful and unsuccessful. We are aiming for a similar experience for users in a virtual environment. The story line content of the three dimensional worlds we are building is created from scenarios from real life that people have experienced and described to us. In those stories we aim to mine the rich detail of experience in its bodily, affective and mental processes. Mental processes can be further detailed as motivations, perceptions, will and
Page 1 of 11
characteristics of personal disposition that are built from previous experiences and personality traits.
VNET Taking a broadly constructivist approach we propose that a three-dimensional software program that provides simulations of real life situations would add a useful tool to teaching/learning environments. We are designing a virtual network education template (VNET) using the Active world’s software (http://www.activeworlds.com/) which will allow us to construct role playing situations. The program will enable students to play out situations in order to develop strategies for dealing with difficult scenarios. In order to create realistic scenarios we use sociological methods, particularly interviewing and observation to gather rich experiential data that is then used to inform both the design and content of the program. Many scenarios can be represented thus catering for individual differences in motivation, agency and lifeskills. We believe this approach will add a new dimension to educational software design as it not only addresses interface issues, but also uses a method that combines content and methodology with educational theory. Teaching methods are most effective when they engage processes that foster deep learning (Marton and Ramsden 1988; Marton and Säljö 1984). Deep learning is facilitated by interactivity between student and material, relevance to their personal experiences, and interactions that aim at meaning-making and problem solving rather than rote learning. In some contexts these processes are referred to as ‘experiential learning’, ‘problem-based learning’, ‘action learning’ or ‘situated learning’. Situational learning has characteristically been devalued in formal education which more often uses didactic rather than inquiry approaches (Hansen 2000). One reason for teacher’s reluctance to provide experiential learning is the preparation time and open-endedness that the development of such learning tasks entail. In the regular classroom large groups of students need to be taught at a time, often leaving little space for individual differences and self-paced learning and assessment. Programs created with the VNET are designed to stand-alone and to be used by students when they choose. One pedagogical strategy for encouraging situated learning has been in the use of simulation games. Simulation games seem successful at allowing students to empathize with the social situations of others and to reflect, in a non-threatening setting, on the nature of their own social location and cultural choices (Maidment and Bronstein 1973). In simulations, all aspects of experience, mental, affective and physiological are evoked. For software and system designers, the challenge has been to understand different pedagogical approaches and the ways these need to be incorporated into the specification and design process (Hinostroza and Mellar 2001; Hughes 1998; Mishra et al. 1999). In software and system design the term simulation tends to evoke a different meaning from that usually understood in the educational literature. Simulation with computer systems can refer to the manipulation of environments, simulation of chemical reactions, growth or digestive processes. For our purposes the simulation refers to social role-playing activity, usually called simulation games in the pedagogical literature. For such ‘simulations’ designers need to provide templates that allow for user experiences and decisions that are qualified by context and contingency (Miral 1998) rather than task-based scenarios.
Page 2 of 11
The Active Worlds software provides an rich architecture for creating simulated experiences in which students may role-play situations designed to help them to reflect on interactions in particular situations. Participants use the Active Worlds browser to access an Active Worlds server located on the Internet. The server component stores information about the world and all the participants and objects within it. The users “view” this world from their own unique perspective. This perspective is then rendered by the participant’s machine (client side) and displayed in the 3D world viewer in the Active Worlds browser seen illustrated below (Fig 1): #4 3D World
#5 Web Browser
#1 View Buttons #2 Expression Buttons #3 Side bar containing tabs for: World List Teleports Contact List Telegrams Help
#6 Chat Window (with whisper)
Figure 1 - The Active Worlds Browser
In the design of VNET it is important to remember the difference between software and systems. Members of the group have at times confused the two and speak as if we were designing of a piece of software. VNET is not a piece of software. If we were designing the Active Worlds Browser itself then we would be designing software. VNET is a system; a system that is built using the Active Worlds software. That said, the Active Worlds software, contains features that we as designers may use to mould the system required to fulfill the specifications laid down by the stakeholders of the project. Considering figure 1, you will notice that the browser is divided into a number of different areas. Each area performs separate function that are woven together to form the VNET system. There are a number of possible interactions a participant may have with the VNET software and these interactions are used along with the scenarios elicited from sense-making Page 3 of 11
methodology to construct the gameplay and gently guide the student through the learning objectives. They are: 1. The Chat window – participants may communicate with those in their local area through the chat window. The whisper function allows private conversations. Communication here is in a synchronous fashion (Refer to Figure 1). 2. The Emote buttons – upon pressing these buttons, the participant instructs their avatar with the 3D world to perform a set of movements which indicate their emotional state e.g. anger, punch, dance, sing etc. These are the equivalent of emoticons in 3D worlds. 3. The Web Browser - this be linked to objects in the 3D world and web pages can be made to appear according to cues within the world. 4. Communications Functions – Contact List and Telegrams allow communicative functions at a distance and/or asynchronously. 5. Navigation Functions – World List and Teleports. These allow the quick movement from one point of interest in the world to another. 6. The 3D World Viewer - This is the focus of most of the design effort of VNET. The participants move within the world, interacting with objects, avatars and bots. Actions withing the worlds include: a. Building b. Moving within c. Interacting with object/s d. Interacting with avatar/s e. Interacting with bot/s Other functions of the browser software are not immediately evident e.g. networking, rendering, and synchronization etc. It is indeed the fact that these aspects of the Active World Multi-User Virtual Environment (MUVE) are taken care of in the background, that we may design our virtual world with relative ease compared to doing so from scratch. VNET is an educational system in which the curricular corpus is cast as a strategy game. The end goal of the game is “getting help to deal with bullying” as measured by standard indicators. The game consists of a number of scenarios organized into levels which are based on narratives provided by respondents who were drawn from the population of people who will use the system. A wide variety of topics could be the base of the VNET program. We have chosen to begin in the health education field. Scenarios form the conceptual commons on which system designers and social scientists may play out and integrate their respective approaches to the design process. We note that the design literature uses the term ‘scenario’ in a variety of ways that address content as well as procedural issues. For our project it is important to delineate the various ways scenarios are used in the design of the VNET. Traditionally, scenarios are used to inform numerous aspects of system design (K. Weidenhaupt, K. Pohl, M. Jarke, and P. Haumer): 1. 2. 3. 4. 5. 6.
System vision Design rationale Usability specification Functional specification User Interface metaphor Prototype Page 4 of 11
7. Object model 8. Formative evaluation 9. Documentation, training and help 10. Summative evaluation Scenario elucidation tools often include storyboarding, ethnography/ethnomethodology and visual prototyping tools. These processes are commonly used for vision development concerning functionality and user-friendliness of the system. The main feature of VNET and that which we are focussing on in this paper is our use of scenarios to inform the content of the virtual environment which falls into a sub-set of the functional specification (#4).
Experience and Sense-Making To facilitate experiential learning, one needs to address the procedural issue of how sociological methods can inform the content of the program as well as interface and sociality aspects. While we acknowledge that interface and social context of the program use are important, in this paper we will focus only on how we are approaching the content issue. In promoting situated learning the greatest difficulty lies in evoking and reflecting individual’s phenomenal worlds. Sociological exploration of people’s experiences, their tacit assumptions about the world, their everyday meaning-making and interactions is the domain of phenomenology. Phenomenology can give rich and detailed descriptions of individual experiences, but as yet there is no typology for organising and generalising from individual behaviours to more abstract generalised situations that people might identify with. Individual experiences are many and varied. If such experiences are to provide the content of an interactive software program a structure is needed that can explore many levels of people’s experiencing in a systematic way as well as provide means for organising and analysing data in ways that can be transposed into the specification and design of the system. In order for situated learning to work as a learning strategy, students need to be exposed to problems that are personally relevant and that they experience an intrinsic motivation to solve, so our data gathering method needs to identify and explore such situations. Sense-Making theory and method developed by Dervin (1983) focusses on the way people make sense of dissonant situations. It provides both a way of conceptualizing lived experience and a way of dialoguing about it (Cardillo 1999). Although Sense-Making is grounded in communication studies and explores the ways people go about seeking and understanding information, Coco (1999) suggests that the interview method is also useful for understanding people’s phenomenal worlds, their inner reasoning processes and construction of meaning. Mira(1998) reflects that “experience is defined by problems and by the knowledge gained from solving them”. Sense-Making Methodology focusses on gaining in-depth insights into people’s reflective examination of personal critical situations or ongoing situations. For the purpose of gaining data for the content of the VNET, the critical incident or ongoing situation was an event that could be described as an incidence of ‘bullying’. However the language we used in interviews was described as behaviour that people had difficulty managing. The situation participants described was one that had taken place in the school environment and in which the person participated in some way. Sense-Making allows for many different levels of experience, mental, affective and physical, to be interrogated. It focuses on individuals' situation defining and their communicative moves. In this way we gain information about the ways people intentionally frame situations Page 5 of 11
which govern their courses of action (Harley 1993). Sense-making then enables investigators to identify strategies that helped and/or hindered their attempts to re-negotiate balance within self and between self and others. The research is actor centred, the researcher does not set the boundaries of a situation but seeks to gain a perspective on the experience from their points of view (Dervin 1992; Dervin 1993). Sense-making then provides a theoretical model of how to conduct interviews with respondents. It is composed of three parts: Situations; Gaps; Uses. Circling each situation involves description of the situation, identification of the gaps in knowledge or understanding and the means by which attempts were made to bridge the gap, and assessing the ways attempts at bridge-building helped or hindered progress in or resolution of the situation (Dervin 1992). While there is an interviewng structure that directs respondents to think about and make connections between aspects of experiences, the focus is on verbing, how they got from here to there. Dervin and others have devised a variety of Sense-making interviewing techniques but we have used the core technique, the Micro-moment Timeline Interview (Dervin 1983), which she describes as "most clearly theoretically derived and most isomorphic with the Sensemaking assumptions" (Dervin 1992). In Dervin's words "experiences can be infinitely variable across respondents, [but] they are given systematic order because recollections are guided in terms of the Sense-Making metaphor and focused on gap defining and gap bridging" (Dervin 1992). The Micro-Moment Time-Line interview, Sense-Making’s foundational interviewing approach, was used to explore people’s experiences. The interview protocol followed this basic outline: Each participant was asked to recall the situation and to describe it in time-line steps; What happened first? What happened next? What happened next? And so on. Then for each time-line step, participants were asked: 1. What thoughts, questions or confusions did you have at this time? 2. Did you have any reactions, emotional or physical at this time? 3. Did these thoughts, (reactions etc) remind you of other things that have happened to you? How did they remind you? Answers to each of these three questions for each time-line step were pursued further with: How did these thoughts etc (reactions, etc., connections etc.,) help or hurt/hinder/constrain you in dealing with the situation? The final question focused on future time If you had a magic wand and could make this situation just right for everybody, what would it look like? Each interview concluded with attention to demographic details. The participants who provided the raw data for the role-playing scenarios were students, their teachers and other participants from three year seven classrooms in the Brisbane area, Page 6 of 11
Queensland Australia. One researcher sat in each classroom for three weeks. During the first week they observed and recorded features of the physical environment, patterns of movement and interaction, and generally became ‘part of the furniture’. They also recorded aspects of students interactional styles on matrices designed as recording sheets. It was essential that students become comfortable with researchers’ presence before researchers began interviews during the next two weeks. In the second and third weeks researchers identified incidents of what could be termed bullying behaviour as they observed in the classroom. They noted the students and any others who were involved with the incidents and the next day conducted micro-moment timeline interviews with them about the incident.
Data Analysis and System Design Organising and reducing data is a normal feature of qualitative analysis and explanation. However in usual sociological analysis and reporting, the telos is to provide explanations of lifeworlds and social structures. For system design we have to approach this stage differently. Rather than generalising to a macro-situation, the data needs to be organised in ways that can give the designer the methods and attributes that they will apply to objects in the program. Several aspects of the design process are informed by the sociological data - the character of the software, the design procedure, the design phase and the design techniques. Character Of The Software VNET structures the interactions within the 3D world using ideas from game theory. While a taxonomy of game genres is still a matter of debate (cf Turocy and von Stengel 2001), the gaming aspects of VNET are similar to strategy games as children must strategize in order to become more proficient in dealing with difficult/bullying behavior. These games emphasize cognition rather than manipulation vis a vis Skill/Action games. As can be seen from the interviewing technique, Sense-Making provides several structural elements that make the data amenable to scenario scripting. Firstly, the people’s experiences are ordered according to chronological ‘steps’. These steps may be emotional, mental or physical, but importantly they are ordered as experienced. This feature facilitates scenario building. Further, we can identify thoughts and confusions and associated feelings and physical actions. This means we have the actual words of the experience (a twelve year old uses language differently from an adult) and movements and environmental descriptions to use in scripts for avatars, bots or players and for constructing the physical environment for the virtual classroom or playground. The emotion data is used for ‘emote’ functions and to manipulate reactions of bots. Strategy games can be further divided into six categories: Adventures, D&D games, wargames, games of chance, educational games, and interpersonal games. The gaming aspects of the VNET system resemble a combination of adventure genres (where players move through complex worlds accumulating tools for dealing with obstacles they encounter) and interpersonal genres (where they deal with positive and negative aspects of feelings and actions). The sense making data combined with phenomenological analysis can give detailed delineation of cognitive processes including players motivations, intentions, perceptions and dispositional characteristics unique to the person. For example, when participants are asked about their thoughts and questions in a situation and how these helped or hindered they Page 7 of 11
respond in such a way as to reveal the content of these mental processes. A helpful strategy would be coded into the simulation as a tool for dealing with obstacles presented by bullying behaviour. Similarly when respondents report feelings and physical reactions we gain an understanding of the ways emotional and physical phenomena affect actions and reactions. They would be used in the program to code the demeanor of avatars which then trigger the next action. Design Procedure The design procedure involves choosing a goal and topic, research and preparation, the design phase and, of course, evaluation of the design (Crawford 1982). The design phase is further divided into three parts: the I/O structure, the game structure and the program structure. The data from interviews also informs the research and preparation phase of the design procedure, as well as providing scaffolding for the game structure. However a detailed consideration of these aspects of the application of data gained from sense-making interviewing is beyond the scope of the present paper.
Conclusion Our project is in its infancy. So far the research and collaborative process has involved creative problem solving as issues arose. At the writing of this paper we are working with the issue of envisaging ways to create game goals that youth would find interesting and motivating. There are some limitations that we feel will be continually present, in the collection of data, the transformation of data and in the construction of the system. Inconsistencies in the collection of data will vary according to the substantive content being explored in the game. In the case of bullying, despite our attempts to be egalitarian and approach the matter in non-judgemental ways, we have found, unsurprisingly, that the parents of children (and mostly also the children themselves) who bully do not agree to be interviewed. We have the recollections of a limited few to gain the points of view of persons who exert their will in socially unacceptable ways. This issue may in part be solved in the traditional ethnographic way by building long-term personal relationships with the children, but such an approach presents other familiar problems regarding to time, funding and interruptions to the work of the school. The transformation of data is one of the most difficult tasks for this project. Details of this process will be the subject of future reports. We see however important theoretical issues arising from the process. Most noticeable is the way creating an interactive computer game forces us to work with the classic tensions in sociological theory. We find ourselves simultaneously working out ways to blend structure and agency, micro and macro, event and process, individual and group in order to script the game scenarios and build their related contexts. A dichochotomy which invokes psychological theoretical issues, is the relationship between physical and virtual worlds. The possiblitlities of the VNET system are somewhat limited by the Active Worlds software itself. Some components are not open to manipulation by programmers or designers e.g. the Side Bar and Chat Window. Alteration of existing components or addition of new ones would require the cooperation of the ActiveWorlds software development teams. Although the software development kit in its current form provides a high degree of exposure of internal methods and variables, not all are so. Also, some aspects of the game play are difficult to implement, for example, fine manipulation of objects, collaborative projects, close-ups, break out screens and would benefit from further development of the software. Page 8 of 11
The VNET design team have already suggested improvements and add-ons to expand the functionality of the gameplay while and at the same time have created adequate innovative solutions to overcome these barriers and produce a system that we hope children will both use and enjoy.
Page 9 of 11
References Activeworlds Corporation, 95 Parker Street, Newburyport, MA 01950, Telephone: (978) 4990222, Facsimile: (978) 499-0221, http://www.activeworlds.com/ Brown, J. S., & Duguid, P. (1993). Stolen knowledge. Educational Technology, XXXIII, 1015. Cardillo, L. W. (1999). Sense-Making as theory and method for researching lived experience: an example in the context of health communication and adolescent illness. The Electronic Journal of Communication/La Revue Electronique de Communication, 9. Carroll, J. M. (2000). Making Use: scenario-based design of human-computer interactions: The MIT Press. Coco, A. (1999). Catholics' meaning-making in critical situations. Unpublished Docotral dissertation, University of Queensland. Crabtree, A., Nichols, D. M., O'Brien, J., & Rouncefiled, M. (1998). The contribution of ethnomethodologically informed ethnography to the process of designing digital libraries. Lancaster: Lancaster University. Crawford, C. (1982). The Art of Computer Game Design (pp. 96). Damarin, S. K. (1993). School and situated knowledge: travel or tourism? Educational Technology, XXXIII, 27-32. Dervin, B. (1983). An overview of the sense-=making research: concepts, methods and results to date, International Communication Association Annual Meeting. Dallas: Ohio State University. Dervin, B. (1992). From the mind's eye view of the user: The sense making qualitativequantitative methodology. In J. D. Glazier & R. R. Powell (Eds.), Qualitative research in information management. Engelwood, Col: Libraries Unlimited. Dervin, B. (1993). Verbing communication: mandate for disciplinary invention. Journal of Communication, 43. Hansen, R. (2000). The role of experience in learning. Jounral of Technology Education, 2. Harley, S. (1993). Situated learning in classroom instruction. Educational Technology, XXXIII, 46-51. Hinostroza, J., Enrique, & Mellar, H. (2001). Pedagogy embedded in education software design. Computers and education, 37, 27(4). Hughes, M. A. (1998). Active learning for software products. Technical Communication, 45, 343-252. Maidment, R., & Bronstein, R. H. (1973). Simulation games: design and implementation. Ohio: Merrill. Marton, F., & Ramsden, P. (1988). What does it take to improve learning? In P. Ramsden (Ed.), Improving learning: new perspectives. London: Cogan Page. Marton, F., & Säljö, R. (1984). Approaches to learning. In F. Marton & N. Entwistle (Eds.), The experience of learning: implications for teaching and studying in higher education (pp. 39-58). Edinburgh: Scottish Academic Press. Miral, B. (1998). 'Applied constructivism' for user documentation: alternatives to conventional task orientation. Journal of Business and Technical Communication, 12, 7. Mishra, P., Zhao, Y., & Tan, S. (1999). From concept to software: developing a framework for understanding the process of software design. Journal of Research on Computing Education, 32, 220-238. Rouncefield, M., Highes, J. A., & O'Brien, J. (1997). Ethnography: some practicalities of ethnographic analysis. Lancaster: Lancaster University. Turocy, T. L., & von Stengel, B. (2001). Game Theory (pp. 39). Page 10 of 11
Viller, S., & Sommerville, I. (1997). Coherence: an approach to representing ethnographic analyses in systems design, HCI Journal. Lancaster: Cooperative Systems Engineering Group, Lancaster University. K. Weidenhaupt, K. Pohl, M. Jarke, and P. Haumer, "Scenario Usage in System Development: A Report on Current Practice". IEEE Software, March 1998. Also available as CREWS Report Series No. 97-16. http://citeseer.nj.nec.com/weidenhaupt98scenario.html
Page 11 of 11
