understanding users' avatar activities in virtual worlds - Springer Link

Education Tech Research Dev (2008) 56:575–593 DOI 10.1007/s11423-007-9081-2 DEVELOPMENT ARTICLE

Mixed methods for mixed reality: understanding users’ avatar activities in virtual worlds David F. Feldon Æ Yasmin B. Kafai

Published online: 4 December 2007 Association for Educational Communications and Technology 2007

Abstract This paper examines the use of mixed methods for analyzing users’ avatarrelated activities in a virtual world. Server logs recorded keystroke-level activity for 595 participants over a six-month period in Whyville.net, an informal science website. Participants also completed surveys and participated in interviews regarding their experiences. Additionally, the study included online ethnographic observations of Whyville and offline observations of a subset of 88 users in classroom and after-school settings during their participation. A mixed-methods analysis identified a major user emphasis on avatar appearance and customization that was invariant across user typologies. Implications for the use of mixed methods in online environments are discussed with regard to three challenges resolved during the study: (1) appropriate reduction of the vast quantity of data, (2) integrated analysis of online and offline events, and (3) interactions between qualitative and quantitative data. Keywords Avatars Contextual data analysis Massive multiplayer online games Virtual environments Mixed methods

Introduction The study of virtual worlds is an emerging topic in education that presents fascinating opportunities to observe learning in non-traditional, large-scale environments. However, the distributed nature of users’ interactions also presents challenging methodological issues that must be addressed before robust inferences can be made (Jenkins et al. 2003; Moore D. F. Feldon (&) Department of Educational Studies, College of Education, University of South Carolina, 1801 Shadowood Drive, Columbia, SC 29212, USA e-mail: [email protected] Y. B. Kafai Psychological Studies in Education, Graduate School of Education and Information Sciences, University of California Los Angeles, Los Angeles, CA, USA e-mail: [email protected]

123

576

D. F. Feldon, Y. B. Kafai

et al. 2005). When learners participate in a virtual environment, they are simultaneously interacting in two worlds—the online (virtual) environment and the offline (real-world) environment. Consequently, the collection and analysis of data are complicated and require careful considerations that are unique to the study of virtual environments. The purpose of this article is to examine these challenges and describe the manner in which a mixed methods approach can provide robust insight into facets of user activity and experience that cannot be attained through single-method approaches.

Three challenges for interpreting data from virtual worlds Collecting data in the virtual environment has the advantage of complete data capture for all interactions among all users through automated user and server logs. This ‘‘virtual omniscience’’ (Moore et al. 2005) yields vast quantities of data to be analyzed (often tens of millions of data points; e.g., Kafai et al. 2007b). Unfortunately, the analysis of these large data sets also presents three primary challenges. The first challenge lies in determining an appropriate scheme for reducing the massive quantity of data. For example, simply determining the level of user engagement within a particular area of a virtual world could entail measuring time spent by users in the environment, counting the frequency of mouse clicks or page hits, or gauging the quantity of chat text generated by users individually or collectively (Chi 1997; Hakkinen 2000). However, depending on the structural affordances of the environment and the prevalent modes of interaction within the specific virtual location, each of these metrics can provide very different pictures of users’ engagement (Williams et al. 2006). Further, the cultural norms of user interactions can vary widely across different regions within a virtual world— even when the structural affordances provided to users do not change (Ducheneaut and Moore 2004; Steinkuehler and Williams 2006). The second challenge for data interpretation is the lack of real-world analogues for some kinds of virtual events (Salomon 1994). For example, some virtual worlds permit ‘‘teleporting’’ between locations within the environment (Fields and Kafai 2007). When users teleport, they disappear from one virtual location and appear in another without following a contiguous path. While much of the activity in virtual worlds can be described and interpreted through reference to real-world behaviors (e.g., walking, chatting, buying, etc.), there is no offline action with a situated meaning equivalent to teleporting. Other users present in the virtual environment are unable to determine whether the disappearance of someone’s avatar is due to teleportation, intentional termination of an online session, or an undesired loss of Internet connectivity without explicit communication through the avatar in question (Cherny 1999). Users may also choose to establish multiple avatar identities to navigate the virtual world under different personae. However, unlike the offline world where assumption of an alternative identity precludes concurrent use of the original, users can open multiple browser windows and present themselves as unconnected avatars that may differ from each other in demeanor, capabilities, and gender identity (Taylor 1999, 2006). Thus, it is possible for a single individual to act as multiple people in such a way that neither other users nor server-based activity logs can identify the actions as those of a single individual. These exclusively virtual phenomena can cause discontinuities in longitudinal data collected in a virtual world and could result in inaccurate conclusions about user behavior. The third challenge of online data collection is capturing relevant offline events that occur unobserved in users’ physical surroundings which influence, complement, or give

123

Mixed methods for virtual worlds

577

alternate meaning to their online actions1 (Castronova 2005; Fields and Kafai 2007; Hine 2000; Kafai in press; Leander and McKim 2003; Lin in press). Conversations between two users who are present in the same physical environment can prompt changes in online action (e.g., ‘‘Meet me at the moon!’’ [Fields and Kafai 2007, p. 194]). Unfortunately, when users who are only observable in the virtual environment interact, the full scope of events is lost, because it is not known if there are additional, offline interactions between them (for examples of such interactions that include different users taking physical control of a keyboard or mouse, see Kafai [in press]). Likewise, offline observations of users’ activities cannot offer a complete picture of interactions with users in other, unobserved physical locations. Thus, studies that fail to integrate real-world observations of online interactions with server-based records sever participants’ actions from their full context.

Qualitative and quantitative methods Tensions between the selection of qualitative and quantitative methods mirror the logistical and interpretive quandaries of researching virtual worlds. Participants in a study may move from one virtual ‘‘location’’ to another in meaningfully quantifiable patterns. Analysis of these actions typically allows inferences to be drawn based on time in each place and intensity of activity within it (e.g., Berendt and Brenstein 2001; Loken et al. 2004; Zaıäne and Luo 2001). However, virtual interactions in each location may also entail participation in unique subcultures that influence not only the frequency and type of interactivity, but also the meaning ascribed by the participant to those online actions (Steinkuehler in press). Further, the evolution of such subcultures and norms for groups of users in virtual worlds typically emerge independently of, and occasionally in opposition to, virtual world designers’ intended affordances and participation patterns (Steinkuehler 2006). Therefore, there is no a priori standard or framework through which to interpret server-generated data. For example, the virtual world Lineage is an environment that houses a role-playing game in which players battle one another to gain status and power in that context (Steinkuehler 2006). Certain areas of the environment act as entryways into the community for new members who lack experience with both the game and the larger virtual environment. Because new players are not as skillful, they are easy targets for more advanced players. As a result, some experienced players will search these areas for newcomers and kill them before they gain enough skill to defend themselves successfully. Other senior members of the community see this as an unacceptable practice and routinely seek out the newcomer-focused predators to kill them or force them to flee to other regions of the virtual world. Thus, there are regular patterns of navigation and interaction that occur within the novice areas. However, without qualitative inquiry to provide context for server-generated statistics, key aspects of the interactions are lost. In such cases, attempts to make generalizable claims about patterns of user navigation and interaction would lead to invalid conclusions, because in such instances, similar patterns describe distinct activities. Further, the patterns may not exist in other virtual worlds or even in other areas of Lineage.

1

This issue also arises with the use of synchronous communication technologies that are not embedded within the virtual world being observed, such as third party instant messaging software, voice over IP, or telephone (e.g., Williams et al. 2006).

123

578


The current study The study reported here illustrates the value of using mixed methods2 to address the methodological and analytical challenges of virtual world research. As one component of a larger study of phenomena within the environment (see also Feldon and Gilmore 2006; Fields and Kafai 2007, submitted; Kafai in press; Kafai and Giang 2007; Kafai et al. 2007b; Neulight et al. 2007), we attempted to characterize the manner in which users engaged with their avatars (graphic self-representations) in the context of a youth-oriented, informal science education virtual world, Whyville.net. Specifically, we sought to understand the role that the customization and control of avatars plays in the interactions between users and with particular affordances of the environment. While the role of avatars in virtual worlds and student engagement is an important topic in its own right (Schroeder 2002; Schroeder and Axelsson 2006), the major focus of this article is to report the methodological considerations and approach that led to our findings. Therefore, the empirical results reported here serve primarily to illustrate the benefits of a mixed methods approach to data collection and analysis. Different facets of these findings are reported in greater depth elsewhere (see Kafai et al., 2007a, b, c).

The Whyville environment Whyville.net is a virtual world designed for users between the ages of 8 and 16 to provide a socially interactive space for children to engage in science-related activities. In addition to typical online affordances, such as chat and intra-Whyville email (called y-mail), its activities include both science-based and recreational games. At the time of the study (Winter, 2005) Whyville had about 1.2 million registered users and could handle up to 4,000 users concurrently. Whyville users (called Whyvillians) receive a virtual salary for each completed science activity with which they can purchase face parts, clothes and accessories designed by fellow Whyvillians for their online avatars. On a typical day Whyvillians log into Whyville and check their y-mail accounts for new messages and their salary ledger for current account status. They then go to popular locations such as the virtual beach or one of the planetary colonies to chat with others. They can also play games with other users or complete more science-focused activities to increase their salaries. Whyvillians often read articles written by members of the community that are posted weekly in the Whyville Times for updates on community life and go to the virtual mall, Akbar’s, to browse through the latest offerings of avatar face parts for purchase (e.g., eyes, hair, lips, clothes, accessories, etc.). They can also sell and trade face parts at a virtual trading post. Several screenshots from Whyville are shown in Fig. 1.

Previous research on avatars In Whyville, users’ avatars are visible to each other within the virtual space as graphic representations that they can customize. Preliminary research examining the impact of 2

Onwuegbuzie and Johnson (2004) suggest that studies utilizing multiple methodologies within or across stages of research (i.e. defining the research objective, data collection, and data analysis) be referred to as ‘‘mixed models’’ rather than ‘‘mixed methods.’’ However, to maximize consistency in terminology across scholars, these phrases will be considered interchangeable for the current study.

123


579

Fig. 1 Scenes from Whyville (clockwise from upper left): Trading post; moving through Myville, a Whyville suburb built and occupied by Whyville users; Socializing at the Beach Grotto

avatars on users’ virtual world experiences suggests that the nature of avatar-mediated interactions can significantly influence the level of presence users experience (Lee 2004; Mirkopoulos and Strouboulis 2004). In turn, presence is positively associated with successful learning outcomes for virtual educational and training environments (Moreno and Mayer 2004; Picciano 2002; Shin 2003; Shin and Chan 2004). Although definitions of presence vary slightly across researchers, the general notion communicates an experience of authenticity during interactions within a computer-based environment (Biocca et al. 2003). Lee (2004) differentiates three primary facets of the construct: physical presence, social presence, and self presence. Physical presence is the extent to which ‘‘virtual…physical objects are experienced as actual physical objects in either sensory or nonsensory ways’’ (p. 44). Similarly, social presence indicates the extent to which users do not notice the mediation of social interactions embedded within the virtual space.3 Self presence reflects the user’s experience of agency and direct interaction with salient elements of the virtual environment. Self presence can manifest through either physical authenticity (e.g., visual perspective, point-of-view) or the ability to generate 3

This sense of presence does not differentiate between actual users who are represented within a virtual space and agents or other fabricated social entities that interact with a user as part of the virtual environment.

123

580


appropriately representative responses to events within the environment via avatars or other mechanisms. Accordingly, previous ethnographic and survey-based research suggests that users in virtual worlds typically place a great deal of emphasis on their avatars’ appearance as extensions of their identities within the virtual space (e.g., Taylor 1999, 2006; Turkle 1995; Yee 2006a). High value is placed on avatars’ appearance and capabilities to the extent that users in some virtual worlds will purchase avatars with particular attributes using real currency in offline environments (Castronova 2005; Taylor 2002) or solicit the input of peers in making avatar-related decisions (Fields and Kafai 2007; Kafai in press; Kafai et al. 2007c). However, self-reported conceptions of avatars’ roles and offline market trends cannot adequately characterize the influence of avatars’ importance to users on activity and general patterns of use within the virtual world itself. Therefore, it is also necessary to integrate qualitative data with analyses of user logs. Previous research into avatar-related online activities has not attempted such extensive integration. Early work by Turkle (1995) rejected the notion of gathering data from the physical world, because her emphasis lay in the positioning of online identity and representation within the virtual context. However, subsequent research (e.g., Taylor 1999) has expanded this perspective to understand better the relationship between online and offline identities. Toward this end, Taylor’s work exclusively uses interviews from participants in both their virtual and physical identities. The strength of this approach is a rich understanding of the multiple identities assumed by users in virtual worlds. However, it lacks the context provided by observations of participants’ ongoing interactions within the cultural contexts that these identities inhabit. Leander (Leander and Lovvorn 2006; Leander and McKim 2003) utilizes ethnographic and interview data to create comprehensive examinations of individuals’ literacy practices across physical and virtual settings. Through interviews and periodic observations of crosscontext activity, he demonstrates that individuals possess continuous identities and practices both online and offline. The nature of their activities changes in response to the social environment, but consistent personal perspectives remain intact. This method takes advantage of situated activity across online and offline settings. However, it is limited by intermittent periods of observation. Dependence on individual ethnographers entails potential observer effects and unitary perspectives on the nature of cultural activities within the virtual world. In an effort to understand the norms and large-scale patterns within virtual worlds, Yee (2006a, b) conducted surveys of approximately 30,000 online gamers over a period of three years. His quantitative approach provides evidence that most users in virtual worlds report interacting with others in the virtual environment much the same as they would in the physical environment (Yee 2007). This perspective suggests that most users (in contrast to Taylor’s [2006] findings from small, purposefully selected samples) perceive their virtual identities to be equivalent to their offline identities aside from issues of appearance. The previous research on avatars in virtual environments is currently inconclusive about their personal meanings and social functions. Intensive qualitative data provides a picture of highly nuanced personal meanings that center on representation within individual virtual worlds. However, survey research from a quantitative perspective does not offer evidence to support the assumption that avatars’ appearances typically carry such personal significance for their users. To generate a more complete understanding of avatars’ roles for users in a virtual world in the current study, we rely on an integration of qualitative and quantitative methods to provide a contextualized understanding of broader trends within Whyville.

123


581

From this perspective, our study addresses the following research questions: 1. How much user activity in Whyville pertains to avatar modification? 2. Does the level of avatar-related activity differ for different types of users? 3. What meanings do users ascribe to their avatars and related activities?

Methods The research questions posed in this study represent a multi-tiered investigation of the interrelationships between students’ identities and perspectives, online behavior, and offline interactions. As each of these elements manifests distinctly in both qualitatively and quantitatively observable ways, a full examination of avatars within Whyville requires an integrated capture and analysis of the data across methods. Specifically, we integrate server-generated logs with interviews, online and offline observations, and participants’ survey responses. As such, accumulation and analysis of data from different research methodologies provide a unique and enhanced representation of the relevant phenomena. Participants Participants in this study were 595 children, ages 8–18, who volunteered in response to solicitations within the Whyville.net environment and obtained parental permission. The sample was representative of Whyville’s gender and age distributions (68% female; median age = 13 years). A subset of 88 users from two sixth grade science classes in a private elementary school and two after-school programs (one at the school and one at an after-school drop-in clubhouse) participated through loosely organized activities and in-person recruitment by researchers.

Design We collected online tracking data and surveys from all participants. We also videotaped observations of Whyville use within the offline environment and conducted offline interviews with the subset of users described above. It was necessary to collect each of these diverse forms of data for three reasons. First, as described above, previous single-method research on avatars generates conflicting findings that cannot be reconciled without using multiple perspectives to examine data from a single cohort of participants. Second, our goal is to identify broad trends in user activity without losing a richer understanding of the context and meaning associated with the behaviors. Therefore, qualitative data plays an important role in gaining users’ perspectives on their experiences. Third, informing the appropriate classification and reduction of log file data requires a thorough understanding of user perspectives, social norms, and structural constraints within Whyville.

Procedures We collected data over a six month period through several mechanisms described below. During the course of the study, virtual epidemics occurred twice within Whyville. These

123

582


events modeled communicable diseases that, when contracted, affected avatars’ appearances by covering them in red dots and interfered with online chat by replacing typed text with the word ‘‘achoo.’’ Outbreaks of the disease, Whypox, typically occur about once per year in the Whyville virtual world (see Kafai et al. 2007b; Neulight et al. 2007). Server logs Server log files transparently recorded all Whyville activity of consenting participants (*70 million lines of data). These files included records of locations visited within Whyville, quantity of time spent there, and text-based communications (i.e., public and private chat, postings to threaded discussions, etc.). Online surveys We administered 30-item online surveys before and after each outbreak of Whypox during the study (i.e., 4 administrations) to elicit general background information and explore the possible impacts of the disease outbreaks on activity and interest levels. Likert response items (e.g., I feel more at ease using a computer when alone than with a group of people [5-point Likert: Strongly Agree…Strongly Disagree]), multiple choice items (e.g., Do you and your friends make plans to log in at about the same time? [Yes, all the time./Yes, some of the time./No, I log in whenever I have the time.]), and open-ended items (e.g., What was the worst thing about having Whypox?) asked participants about their practices, experiences, and preferences in Whyville activities, including specific items pertaining to Whypox and relevant issues with their avatars. Additional multiple-choice questions elicited estimates of frequency and comfort with computer use and online activity (e.g., How much time do you spend each day visiting chat rooms? [No time/0–15 min/ 15–60 min/1–2 h/More than 2 h]), as well as the degree of interest in science and technology (e.g., How often do you read books about science or science fiction? [I do this now./ I used to do this but don’t anymore./I never do this.]; Do you know how to make a web page? [yes/no]). Participants were compensated in ‘‘clams’’ (Whyville’s virtual currency) for completing each survey. The aggregate response rate across all survey administrations was 73.64%. Overall reliability of the multiple choice and Likert items using Cronbach’s alpha was computed to be 0.722.4 Any open-ended item with an inter-rater reliability less than 80% was excluded from analyses.

Offline observations Observations of the classroom and after-school environments included researchers’ field notes and video recordings of participants using Whyville. Whenever possible, video cameras were positioned to record the activities of all participants in the room. In situations where this was not possible, priority was given to those participants with high levels of offline interaction related to Whyville use. All participants were videotaped at least once, but the total number of instances for each participant depended on their attendance, 4

Reliability of these items was likely hindered by the unstable relationship between interest in technology as a discrete activity and the amount of time spent using technology for schoolwork and social communication.

123


583

self-selected activities during unstructured periods, and the extent of their interactions with others in the room while online. Researchers attempted to balance the desire to remain unobtrusive during observation with the occasional need to assist student participants with technical difficulties that arose during computer use. Given the extended period of data collection and the regularity of filming, students seemed to acclimate to the researchers’ presence quickly and did not seem unduly influenced by their presence with regard to peer interactions or choices of activities. However, it is not possible to definitively know the impact of our presence in the study’s settings, so this may be a possible limitation for the interpretation of our findings.

Interviews We conducted in-depth, face-to-face interviews with 35 willing classroom and after-school participants about their Whyville experiences at the conclusion of the study (other participants were unavailable or uninterested in specifically discussing their avatars). Openended questions emphasized issues pertaining to avatar use, design, and identity in both online and offline contexts, such as: ‘‘How is your avatar like you and/or not like you?’’ and ‘‘How often do you change your avatar?’’ Based on responses to those questions, we asked opportunistic follow-up questions to elicit more detail and clarify meaning. Using a grounded theory approach, we analyzed transcriptions of the interviews for reasons why youth created and interacted with their avatars the way they did, listing every reason they gave and grouping them into themes. Since some had more than one reason for making a particular look, or some changed their looks periodically, the themes are not mutually exclusive. Thus for the 35 participants, we listed 44 reasons for creating a particular look and grouped those into six major themes, with 2–4 sub-themes each. The primary coding and grouping themes were rechecked by the other authors (Kafai et al. 2007b).

Ethnography One member of the research team conducted ethnographic observations that captured Whyville community life before, during, and after Whypox. Her role was that of an embedded reporter visiting different popular places in Whyville daily with the goal to observe movement patterns, chat interactions of online users, and postings in the Whyville Times. Her experiences and observations provided experiential characterizations of various community events, social norms, and subcultures that provided context for interpreting users’ actions. Kafai et al. (2007b) report these findings in detail.

Analysis Greene et al. (1989) articulate a framework of rationales for the incorporation of multiple methods and identify five distinct categories that we employ in the collection and analysis of our data: expansion, triangulation, complementarity, initiation, and development. In each category, the collection of both quantitative and qualitative data inform the analysis and interpretation of results to provide a more comprehensive, accurate, or refined account of the phenomena observed. By virtue of the differing natures of the phenomena captured, it is necessary to embrace each of these rationales at different stages of the study.

123

584


Expansion Employing multiple methods is a common means for expanding the breadth or scope of a study. In this case, our overall purpose is to provide an integrated perspective on avatars in Whyville that encompass aspects of the various methodologies that have been used previously to investigate these phenomena. Triangulation Triangulation is the identification of converging evidence that corroborates the validity of inferences drawn from different methods. In our analyses, this approach sheds light on the role that avatar modification plays in Whyville use. Because structural constraints, cultural norms, and personal inclinations converge to affect the frequency of avatar-related activity, it is necessary to bring together data from the log files (quantitative hit counts), survey data (quantitative and qualitative), interviews (qualitative), and ethnographic observations (qualitative). Log files indicate the number of virtual interactions in avatar-related areas, but they cannot differentiate between those hits that were required due to the software’s mechanisms for changing an avatar’s appearance (purchasing, trading, arranging parts on avatar, etc.) and those that represent participants’ independent choices to engage with their avatars. Further, a user’s commitment to changing an avatar might result in a need to navigate across a different number of pages than another activity to which she was equally committed (e.g., playing checkers). Complementarity The purpose of examining multiple data sources for complementarity is to identify both overlapping and distinct aspects of avatar-related activity for elaboration and clarification. We synthesize the interviews, surveys, and observational data to understand user interactions around avatars in shared physical and virtual spaces. The conversations and meanings attached to the process of valuing face parts both for purchasing/trading decisions and for changing avatars’ faces and accessories present a multi-faceted view of participants’ interactions.

Development Development repurposes data from one method to support the development or analysis of another method of data collection. In this case, the observations and analysis obtained from the online ethnography are essential to developing meaningful categories of Whyville locations in order to reduce the location (URL) data from the log file appropriately using cluster analysis. The understanding of both sociocultural and structural functions from the various places allows us to assess activity-related differences in users’ allocation of their time in Whyville. URLs in the server log are coded as specific to Whypox (i.e., Whypox), avatar-related (i.e., Avatar), social (i.e., Socializing), dedicated to civic participation in Whyville (i.e., Whyville Community), science-related (i.e., Science), dedicated to playing games with other users (i.e., Games), dedicated to managing users’ own financial resources within Whyville (i.e., Finances), and miscellaneous (i.e., Other).

123


585

In a more typical application of development, we also use the survey and observational data as background information to frame interview questions meaningfully around both common and unusual experiences. Further, the patterns of survey responses and ethnographic trends provide a context to determine the typicality of the participants interviewed.

Initiation Concurrent analysis of data from multiple methods permits the identification of contradictions, paradoxes, and alternative explanations for an observed event. In this case, as explained in more detail during the discussion of results, the large proportion of navigation dedicated to avatar-related activities seems to contradict the moderate affect and value for them expressed during the interviews.

Results Quantitative results User log data Of the 6.93 million individual screen-locations (URLs) visited within Whyville by the 595 participants, 33.4% were locations specifically related to the creation, sampling, trading, or purchasing of face parts (i.e., avatar features and accessories). The mean percentage of avatar-related hits was 33.43% (SD = 14.82). Socializing, which includes chatting and sending messages, is the next greatest category of locations and represented 27.5% of the total page hits. The mean percentage of socializing hits was 25.27% (SD = 11.42).

Cluster analysis Exploratory k-means cluster analysis identified three clusters of users (Table 1). Based on the total Whyville activity in each cluster and the relative distributions of the location types, we have labeled them as Casual Users, Social Users, and Heavy Users. Casual users visited locations within Whyville about one fourth as much as Social users. In turn, Social users visited locations within Whyville less than half as much as the Heavy users. However, the proportion of page hits related to avatar-based activities does not vary significantly across the three groups. Likewise, the clusters do not indicate significant differences in the survey-based measures of science and technology interest. Several parameters of the Whyville environment are relevant to the characterization of the clusters. Whyville users receive a virtual salary in proportion to the number of science game activities that they complete. However, there are a finite number of science games, so there is a maximum possible salary. Once users achieve this maximum by completing all of the science tasks at the highest levels of difficulty, there is no extrinsic reward mechanism for continued activities in science locations.5 5

This observation was also made by one of the participants interviewed, who said ‘‘At first I really liked this gator game, so I started to really get into it. But then I couldn’t get any more salary from that game so I got really bummed out….’’

123

586


Table 1 Final cluster centers for descriptive categories of Whyville users Clusters

ANOVA results

Casual users (n = 409)

Social users (n = 131)

Heavy users (n = 32)

Science interest (z-score)

1.0428

0.8416

0.9492

2.44 0.0879a

Technology interest (z-score)

1.1548

0.9338

1.2813

2.65 0.0717a

Total Whyville activity (Hits)

5437.8

21451.57

53762.5

Whyville community (%)

4.7364

4.1843

3.7836

4.67 0.0097

Whypox (%)

1.6022

0.3703

0.2058

18.79 0.0000

Socializing (%)

23.7548

29.5084

26.1892

13.30 0.0000

Science (%)

4.3229

2.8852

2.3262

11.30 0.0000

Finances (%)

2.4401

1.3093

0.9222

19.80 0.0000

Other (%)

7.2251

5.5925

9.1237

9.95 0.0000

Avatar (%)

33.2769

34.7083

30.7410

1.02 0.3606a

Games (%)

2.8563

2.0089

1.5346

6.36 0.0018

a

F

Sig.

1579.39 0.0000

Nonsignificant differences between clusters

In relation to general navigation patterns within Whyville, it follows that attention to finances and visits to science activities wane as users sustain high levels of activity, because the concomitant likelihood of achieving maximum salary increases. Further, those users who spend the most time on Whyville are also most likely to find unique social niches in less common social locations, like individual user’s houses in Myville (see Fig. 1). Thus, the relative increase in activity in miscellaneous (i.e., Other) locations within Whyville is unsurprising.

Survey data Surveys indicate that 28.3% of participants report spending at least half of their time in Whyville on avatar related activities, and 36.2% report changing their looks every few days or more. Most participants (63.8%) report changing their look no more often than every few weeks. 36.6% feel that it is important to have many ‘‘clams’’ (i.e., virtual currency) to buy avatar accessories or face parts, and 48.2% cite avatar appearance as the reason for spending clams on their most expensive face part. Overall, 54.4% of Whyvillians reported spending at least half of their virtual salary on face parts for their avatars. However, 20% of respondents indicate that avatar appearance is not the most important aspect of Whyville. A respondent typical of this group commented that ‘‘there are other ways to win fame and respect.’’ This is consistent with the 65.7% of Whyvillians who did not try to cover up their Whypox with new face parts during the epidemics. ‘‘I didn’t do anything (about the spots from Whypox) because it doesn’t mader [sic] what you look like, & if you covered it up...that doesn’t mean you don’t have it ne [sic] more!’’ However, nearly one third (32.3%) of respondents did try to hide their symptoms, citing concern for their virtual appearances.

123


587

Qualitative results Field notes and videotaped observations Based on the offline data, two primary types of interactions centered on avatars: critiquing or admiring others’ avatars and donating or trading face parts. Of these, the activity of critiquing or admiring avatars, which consisted of both blanket statements such as ‘‘he’s ugly’’ or ‘‘she’s hot’’ and more refined critiques like ‘‘Oh my god you’re gonna look so nice like since it matches your hair,’’ were recorded about twice as often as donating or trading face parts. Designing avatars in the after-school club often consisted of long stretches of time (15 min or more) devoted to searching for face parts in Whyville’s mall, calling over friends to get their opinions, trying various parts on, purchasing parts, and carefully positioning each face part ‘‘just right’’ on the avatar. Such interactions were typical both in the classroom during unstructured time (or when students were off-task during independent work times) and in the after-school settings. Donating or trading face parts could be pragmatic (e.g., exchange of used face parts) or a way of solidifying friendships (e.g., a girl gave a special ‘‘discount for her friends’’ on face parts that she owned). Some participants even donated face parts to others as a type of prank. In one instance recorded in the field notes, a male participant continuously sent stereotypically female face parts to his male friends as a prank with the suggestion that their avatars wear them. Eventually, his friends chose to retaliate physically, and one of the researchers needed to break up the scuffle. Video data provided another example of offline interactions surrounding online events as a form of teasing in the following transcription: Sam (to computer screen): ‘‘ ‘Cootie eyes’ give to Masher 47…’’ Sam (yelled to classmate across room): ‘‘Aidan I just gave you face parts. Check ‘em!’’ Sam (to three friends clustered around his computer): ‘‘This is gonna be funny.’’

Interviews None of the participants we interviewed thought that their avatars looked like them, except for occasional similarities in accessories (e.g., ‘‘We both like to wear necklaces’’). About one third indicated that they changed their avatars rarely (once per month or less). Another third indicated that they changed sometimes (every two weeks). The remaining third indicated they changed often (once a day to once a week). Similarly, the participants varied in terms of which parts they changed when they did modify their avatars. One third of those interviewed said that they changed everything—their whole look. Reasons for a complete change included playing with a different look (‘‘If I see somebody I like again then I copy them’’), wanting to remain unknown on the site (‘‘I want to be mysterious’’), sneaking up on friends to throw something at them (‘‘Pie in the face’’), or boredom with the old look. The remaining participants did not change their entire faces, and generally kept some parts all the time. However, some reported changing parts they considered important (e.g., hair), while others only changed parts they thought were unimportant (e.g., clothes). Still, among participants who changed everything and among those who changed little, almost all indicated that it was unimportant for avatars to be visually recognized on

123

588


Whyville, because it was easy to see an avatar’s user name by positioning the mouse over it. When asked directly about their motivations for the choices they made, participants commonly responded with one or more of the following six rationales: simple aesthetic considerations (10), to make the avatar in part like their ‘real’ selves (8), to affiliate with something or someone (7), to have a feature they cannot have in real life (6), to conform with or rebel against a popular trend (7), and for functional reasons like disguise (6). Many Whyvillians put things on their avatars or created a whole avatar that represented some affiliation with a movie or video game, sport, or desire. These included having a Storm Trooper head (a reference to Star Wars), Matrix sunglasses, (‘‘I wanted to be, I was Trinity, like the glasses’’), a skateboard (‘‘I skateboard’’), or two dogs (‘‘I really want to have them because I don’t have a dog’’). Some also experimented with characteristics that they did not have in real life (‘‘I don’t have blond hair’’). Others seemed to be going after a whole look that expressed an abstract quality, like being angelic or cool (‘‘I could look cool and nerdy at the same time’’).

Discussion Our mixed methods approach to studying the role of avatars in Whyville provides important insights that would not have been attainable through other approaches. The identification of broad trends that are contextualized by rich understandings of (1) Whyville’s cultural and structural affordances and (2) participants’ personal meanings for actions and events offer insights about the nature of youths’ interactions with avatars in Whyville. These understandings may also provide useful insights into the design of other educationally focused virtual worlds.

How much user activity in Whyville pertains to avatar modification? Overall, one third of participants’ navigation in Whyville occurred in avatar-specific locations as recorded by the log files. Further, 28.3% indicated in survey responses that half of their time spent in Whyville was dedicated to avatar-related activity and 36.2% reported that they changed their avatars’ looks at least every few days. Despite the fact that the majority (63.8%) responded that they did not change their appearance more than every few weeks, avatar-related activity accounted for at least 20% of total page hits for more than 80% of the participants. Through a quantitative lens, it is clear that these trends in Whyville use are robust. However, a definitive interpretation of the data is ambiguous. It is possible that Whyville.net was designed in such a way that users must navigate through a disproportionate number of pages to accomplish minimal avatar-related tasks. However, it is also possible that avatar activity is a highly valued aspect of social and civic life in Whyville that drives engagement independent of users’ desire to alter their avatars’ appearance. By integrating the quantitative and qualitative data, however, the evidence strongly suggests that the latter interpretation is the correct one. Not only do avatars’ appearances occupy a great deal of focus for most Whyvillians online, but they also figure prominently in offline conversations when children visit Whyville from computers in the same offline spaces. Reports on avatar-related issues have been published in The Whyville Times by different authors 587 times over the past 7 years of Whyville’s existence (an average of

123


589

almost once per week), discussing topics ranging from fashion trends and the logistics of making face parts to broad issues of racial equity (Kafai et al. 2007b).

Does the level of avatar-related activity differ for different types of users? In light of the broad trends discussed above, it is important to determine if Whyvillians’ avatar-related interactions are uniform or if different types of users might participate differently. In this regard, the cluster analysis (generated through quantitative analysis of variables qualitatively identified as relevant within the context of Whyville) provided a key insight: Regardless of other differences in participants’ profiles, the proportion of online activity related to avatars did not vary. Because avatars command such high levels of attention from heavy and casual users alike, avatar customizability seems to be an important feature of the Whyville environment overall. Furthermore, it reinforces the finding that avatars are a dynamic aspect of life in Whyville, because even those users who do not change their appearances frequently engage with relevant facets of the virtual world. If the level of navigation pertinent to avatar-related activity differed across clusters, it would suggest that relevant forms of engagement with Whyville would wax or wane as a function of overall use. For example, casual users might have been less inclined to modify their avatars because they were not as vested in interactions on Whyville as heavy users. Conversely, it could have been the case that avatar-related activity tapers off as heavy users crystallize their avatars’ features over time. However, as these possibilities did not manifest, it seems that avatars hold broad appeal for the Whyville population.

What meanings do users ascribe to their avatars and related activities? In contrast to the findings of Taylor (1999, 2006) and Turkle (1995), most participants in the current study did not seem to vest their avatars with deep personal meaning or attributes of identity for personal exploration. Instead, the emphasis tended to be on normative social dynamics on Whyville. Whyvillians largely wanted to be interesting to their peers on the basis of aesthetics and affiliation with popular trends or activities of interest. On occasion, they used their appearance as a tool for accomplishing a particular goal. Of particular interest was the socially complex ways in which face parts were used in interactions between users. Giving away or trading/selling a particular avatar feature carried very diverse meanings depending on the nature of the part, the identities of the giver and receiver, and preexisting social relationships that all provided a meaningful interpretation of the transaction. Here again, the use of mixed methods provided a rich view of the ways in which elements of avatars’ appearances carry a highly contextualized meaning. The frequency of these transactions was evident in the log files, but it was only upon observing both the online and offline interactions surrounding them and following up with interview questions that the full complexity of the phenomenon emerged.

Conclusions Avatars’ appearances in Whyville are clearly an important and central focus of activity. The high frequency of relevant activity was invariant across the three classes of users derived through cluster analysis, despite significant differences in the proportions of

123

590


scientific and social pursuits on the site. The prevalence of avatar-related customization and exchange is not typical of other virtual worlds (e.g., Yee 2006a). However, Whyville is an unusual online setting (in part) because it allows extensive customization of avatar appearances beyond selection from a limited catalog of features. Thus, the meaning constructed by users during their avatar development and interactions is particularly sensitive to the nuances of cultural influences within and across contexts and the desire of individual users to be perceived in certain ways relative to those contexts. In addition, the majority of Whyville users are young adolescents (median age = 13.1 years). Therefore, emerging issues of appearance and related social interactions are typical of the demographic (Berk 2006), but unusual for other virtual worlds like EverQuest and Star Wars Galaxies where the median age is 25 (Griffiths et al. 2003; Yee 2006b). Avatar-related activity is clearly a significant element of Whyville activity based on quantitative analyses. However, the qualitative data reveals the diversity of motivations that underlie it (expansion; Greene et al. 1989). Some participants used avatars to communicate information that they wanted others to know (e.g., ‘‘I skateboard’’). Others used their avatars to have qualities that were not possible in their daily lives (e.g., ‘‘I don’t have blond hair’’, ‘‘I don’t have a dog’’, etc.). Much avatar activity primarily served as a focus for social interaction and conversation, both online and offline (e.g., critiquing avatars collaboratively). The integration of qualitative and quantitative data also revealed some surprising discrepancies (initiation; Greene et al. 1989). For example, two-thirds of the participants interviewed indicated that they changed their avatars’ looks in Whyville no more frequently than every two weeks, despite the large proportion of online activity in avatarrelated locations. The interview, video, and observation data suggests that a great deal of time in avatar-related locations is spent browsing or ‘‘window shopping’’ rather than actually changing avatar appearances. Further, the execution of simple trades or donations of parts for avatars may carry with them extended social meaning that is not evident in the online environment or from the perspective of user logs. The interaction regarding the ‘‘Cootie eyes’’ discussed above is a clear example. Without the ability to observe the offline aspect of the interaction, it would have taken on the appearance of a simple gift with a possible motivation of friendship or charity, rather than a prank involving online activities for an offline audience. The mixed methods approach we used to study Whyville offers insights about users’ situated activity and their broad behavioral trends. Interpreting each source of data in light of the other and integrating the analyses of both allow us to understand users’ engagement with their virtual world as a complex cultural entity from which generalizations can be drawn to inform future research and the development of new computer-mediated educational environments. However, the inferences we draw in this work are limited by the cross-sectional nature of the data collection. Our assessment of actions and social meaning through the combination of tools reported here cannot provide a continuous perspective that captures the dynamic interaction between personal meaning and behavior for a given user. To better understand this linkage, it may prove beneficial to integrate multiple types of data in the application of connective ethnography (Leander and Lovvorn 2006; Leander and McKim 2003). This method explicitly emphasizes ‘‘tracing the flows of objects, texts, and bodies’’ (Leander and McKim 2003, p. 211) and analyzing the construction of boundaries within and between virtual and physical environments. Initial applications of this approach have provided useful insights on the sharing of knowledge about different avatar functions in Whyville (Fields and Kafai 2007, submitted).

123


591

In future work, it is also critical to design and implement experimental and quasiexperimental studies to assess the causal mechanisms underlying trends found in descriptive studies. Such research will clarify the causal role that structural affordances may play in shaping users’ online behavior and interactions. For these studies, attention to the integration of qualitative and quantitative data will be especially important. Because metrics used to compare different experimental conditions must be equivalent across conditions but meanings ascribed to virtual actions may vary significantly—even within a single virtual world, researchers must integrate thorough qualitative analyses with any attempts to quantify outcomes for statistical comparisons. Implications The pervasive engagement with avatars in Whyville and users’ nuanced interactions surrounding them suggest that this facet of virtual worlds may be an optimal focus for the design of informal learning events like Whypox. Research in academic motivation indicates that learning is more likely to occur when the target content is entailed by activities of personal and cultural value and relevance (for reviews, see Eccles and Wigfield 2002; Locke and Latham 2002). These trends are mirrored in the emerging research on virtual presence and learning (e.g., Moreno and Mayer 2004; Picciano 2002; Shin 2003; Shin and Chan 2004). As further research examines the role of avatar-related affordances in virtual worlds and their links with user engagement, we may be able to identify specific design principles to improve learning outcomes for events based in virtual worlds. This is the first study of a virtual world to incorporate user log data with online and offline observations, interviews, and surveys. The use of mixed methods to augment the meaning and phenomena captured through individual facets of the study provides a strong example of their powerful role in the study of web-based environments specifically and other complex learning environments. Indeed, our conclusions suggest that mono-method studies are not only limited in scope, but may generate inferences that are not supported by the full complement of available data. As future research progresses to identify those aspects of virtual worlds that we can leverage effectively for educational purposes, it will be essential to attend to the full context of events through mixed methods approaches. Acknowledgements An earlier version of this paper was presented at the Annual Meeting of the American Education Research Association in April 2007, Chicago, Illinois. The analyses and writing of this paper has been supported by a grant of the National Science Foundation (NSF-0411814) to the second author. The views expressed are those of the authors and do not necessarily represent the views of NSF, the University of South Carolina, or the University of California. We wish to thank Melissa Cook, Deborah Fields, Michael Giang, Linda Kao, and Kylie Peppler for their assistance with data collection and analysis.

References Berendt, B., & Brenstein, E. (2001). Visualizing individual differences in web navigation: STRATDYN, a tool for analyzing navigational patterns. Behavior Research Methods, Instruments, and Computers, 33(2), 243–257. Berk, L. E. (2006). Development through the lifespan (4th ed.). New York: Allyn & Bacon. Biocca, F., Harms, C., & Burgoon, J. K. (2003). Toward a more robust theory and measure of social presence: Review and suggested criteria. Presence: Teleoperators and Virtual Environments, 12(5), 456–480. Castronova, E. (2005). Synthetic worlds: The business and pleasure of gaming. Chicago: Chicago University Press.

123

592


Cherny, L. (1999). Conversation and community: Discourse in a social MUD. Stanford: CSLI Publications. Chi, M. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6(3), 271–315. Ducheneaut, N., & Moore, R. J. (2004). The social side of gaming: A study of interaction patterns in a massively multiplayer online game. In Proceedings of the 2004 ACM Conference on Computer Supported Collaborative Work (pp. 360–369). New York: ACM. Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–132. Feldon, D. F., & Gilmore, J. (2006). Patterns in children’s online behavior, scientific problem-solving: A large-N microgenetic study. In G. Clarebout & J. Elen (Eds.), Avoiding simplicity, confronting complexity: Advances in studying and designing (computer-based) powerful learning environments (pp. 117–125). Rotterdam, the Netherlands: Sense Publishers. Fields, D. A., & Kafai, Y. B. (2007). Tracing insider knowledge across time and spaces: A connective ethnography in a teen online game world. In C. Hmelo-Silver & A. O’Donnell (Eds.), Proceedings of the 2007 International Conference on Computer Supported Collaborative Learning (pp. 193–195). Mahwah: Lawrence Erlbaum Associates. Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11, 255–274. Griffiths, M., Davies, M., & Chappell, D. (2003). Breaking the stereotype: The case of on-line gaming. Cyber-Psychology and Behavior, 6, 81–91. Hakkinen, P. (2000). Neural network used to analyze multiple perspectives concerning computer-based learning environments. Quality & Quantity, 34, 237–258. Hine, C. (2000). Virtual ethnography. London: Sage Publications. Jenkins, H., Klopfer, E., Squire, K., & Tan, P. (2003). Entering the education arcade. ACM Computers in Entertainment, 1(1), 1–11. Kafai, Y. B. (in press). Gender play in a tween gaming club. In Y. B. Kafai, C. Heeter, J. Denner, & J. Sun (Eds.), Beyond Barbie and Mortal Kombat: New perspectives on gender and gaming. Cambridge: The MIT Press. Kafai, Y. B., Cook, M. S., & Fields, D. A. (2007a). ‘‘Blacks deserve bodies too!’’ Design and discussion about diversity and race in a teen online world. Paper presented at the Digital Games Research Association International Conference. Tokyo, Japan, September, 2007. Kafai, Y. B., Feldon, D., Fields, D. A., Giang, M., & Quintero, M. (2007b). Life in the time of Whypox: A virtual epidemic as a community event. In C. Steinfield, B. Pentland, M. Ackerman, & N. Contractor (Eds.), Communities and technologies 2007 (pp. 171–190). New York: Springer. Kafai, Y. B., Fields, D. A., & Cook, M. S. (2007c). Your second selves: Resources, agency and constraints in avatar design in a teen online world. Paper presented at the Digital Games Research Association International Conference. Tokyo, Japan, September, 2007. Kafai, Y. B., & Giang, M. T. (2007). Virtual playgrounds: Children’s multi-user virtual environments for playing and learning with science. In T. Willoughby & E. Wood (Eds.), Children’s learning in a digital world (pp. 196–217). Oxford: Blackwell Publishing. Leander, K. M., & Lovvorn, J. F. (2006). Literacy networks: Following the circulation of texts, bodies, and objects in the schooling and online gaming of one youth. Cognition & Instruction, 24(3), 291–340. Leander, K. M., & McKim, K. K. (2003). Tracing the everyday ‘sitings’ of adolescents on the internet: A strategic adaptation of ethnography across online and offline spaces. Education, Communication, & Information, 3(2), 211–240. Lee, K. M. (2004). Presence, explicated. Communication Theory, 14(1), 27–50. Lin, H. (in press). Body, space, and gendered gaming experiences: A cultural geography of homes, cybercafe´s and dormitories. In Y. B. Kafai, C. Heeter, J. Denner, & J. Sun (Eds.), Beyond Barbie and Mortal Kombat: New perspectives on gender, games, and computing (pp. 54–67). Boston: The MIT Press. Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57(9), 705–717. Loken, E., Radlinski, F., Crespi, V. H., Millet, J., & Cushing, L. (2004). Online study behavior of 100,000 students preparing for the SAT, ACT, and GRE. Journal of Educational Computing Research, 30(3), 255–262. Mirkopoulos, T. A., & Strouboulis, V. (2004). Factors that influence presence in educational virtual environments. CyberPsychology & Behavior, 7(5), 582–591. Moore, R. J., Ducheneaut, N., & Nickell, E. (2005). Leveraging virtual omniscience: Mixed methodologies for studying social life in persistent online worlds. Workshop presented at the Games, Learning & Society Conference, Madison, WI, June 23–24, 2005.

123


593

Moreno, R., & Mayer, R. E. (2004). Personalized messages that promote science learning in virtual environments. Journal of Educational Psychology, 96(1), 165–173. Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40, 120–123. Neulight, N., Kafai, Y. B., Kao, L., Foley, B., & Galas, C. (2007). Children’s participation in a virtual epidemic in the science classroom: Making connections to natural infectious diseases. Journal of Science Education and Technology, 16(1), 47–58. Onwuegbuzie, A. J., & Johnson, R. B. (2004). Mixed method and mixed model research. In R. B. Johnson & L. B. Christensen (Eds.), Educational research: Quantitative, qualitative, and mixed approaches (pp. 408–431). Boston: Allyn and Bacon. Picciano, A. G. (2002). Beyond student perceptions: Issues of interaction, presence and performance in an online course. Journal of Asynchronous Learning Networks, 6(1), 21–40. Salomon, G. (1994). Differences in patterns: Studying computer enhanced learning environments. In S. Vosniadou, E. De Corte, & H. Mandl (Eds.), Technology-based learning environments: Psychological and educational foundations. NATO ASI Series F: Computer and System Sciences (Vol. 137, pp. 79–85). Berlin: Springer. Schroeder R. (Ed.) (2002). The social life of avatars: Presence and interaction in shared virtual environments. London: Springer-Verlag London Limited. Schroeder R., & Axelsson A.-S. (Eds.). (2006). Avatars at work and play: Collaboration and interaction in shared virtual environments. London: Springer. Shin, N. (2003). Transactional presence as a critical predictor of success in distance learning. Distance Education, 24(1), 69–86. Shin, N., & Chan, J. K. Y. (2004). Direct and indirect effects of online learning on distance education. British Journal of Educational Technology, 35(3), 275–288. Steinkuehler, C. (2006). The mangle of play. Games and Culture, 1(3), 199–213. Steinkuehler, C. A. (in press). Cognition and literacy in massively multiplayer online games. In D. Leu, J. Coiro, C. Lankshear, & K. Knobel (Eds.), Handbook of research on new literacies. Mahwah: Erlbaum. Steinkuehler, C. A., & Williams, D. (2006). Where everybody knows your (screen) name: Online games as ‘‘third places’’. Journal of Computer-Mediated Communication, 11, 885–909. Taylor, T. L. (1999). Life in virtual worlds: Plural existence, multimodalities, and other online research challenges. American Behavioral Scientist, 43(3), 436–449. Taylor, T. L. (2002). ‘‘Whose game is this anyway?’’: Negotiating corporate ownership in a virtual world. In F. Maÿra¨ (Ed.), Computer games and digital cultures conference proceedings. Tampere: Tampere University Press. Taylor, T. L. (2006). Play between worlds: Exploring online game culture. Cambridge: The MIT Press. Turkle, S. (1995). Life on the screen: Identity in the age of the Internet. New York: Simon & Schuster. Williams, D., Ducheneaut, N., Xiong, L., Zhang, Y., Yee, N., & Nickell, E. (2006). From tree house to barracks: The social life of guilds in World of Warcraft. Games and Culture, 1(4), 338–361. Yee, N. (2006a). The psychology of massively multi-player online role-playing games: Motivations, emotional investment, relationships, and problematic usage. In R. Schroeder & A. Axelsson (Eds.), Avatars at work and play: Interaction in shared virtual environments (pp. 187–207). London: SpringerVerlag. Yee, N. (2006b). The demographics, motivations and derived experiences of users of massively multi-user online graphical environments. Presence: Teleoperators and Virtual Environments, 15, 309–329. Yee, N. (2007). The Daedalus project. Available at http://www.nickyee.com/daedalus. Accessed 27 November 2007. Zaıäne, O. R., & Luo, J. (2001). Towards evaluating learners’ behaviour in a web-based distance learning environment. In Proceedings of the IEEE International Conference on Advanced Learning Technologies (ICALT 2001), Madison, WI, USA, 6–8 August 2001.

David F. Feldon is an assistant professor of educational studies at the College of Education, University of South Carolina, Columbia. Yasmin B. Kafai is an associate professor of learning and instruction at the Graduate School of Education and Information Studies at the University of California, Los Angeles.

123