assessment of a web-based learning game in an ... - CiteSeerX

ASSESSMENT OF A WEB-BASED LEARNING GAME IN AN INFORMATION SYSTEMS COURSE ZHENYU HUANG and JAMES J. CAPPEL Central Michigan University Mount Pleasant, Michigan 48859

ABSTRACT For decades, instructors have used games to facilitate student leaming. More recently, the availability of the World Wide Web and Web authoring tools have greatly expanded the opportunities for instructors to utilize online leaming games. Web-based leaming games would appear to have particular appeal to the "digital generation" of leamers who have grown up playing computer and video games. However few previous studies have evaluated the use of online leaming games. This study addresses this issue through the application of an online game called FlashGame, developed using Macromedia® Flash''''^, in two mass sections of an introductory infonnation systems course. A survey was administered at mid-semester to assess participants' usage and perceptions of this game. Since web-based leaming games are at an early stage of development, the fmdings provide important insight and raise implications for instructors about the use of online games to support student leaming. Keywords: Leaming Games for IS instruction; Online Leaming; Web Authoring Tools; Macromedia Flash.

taking a break from work, looking for thrills or excitement in action-oriented games, keeping one's mind challenged and active, and making friends through social interaction while playing online games (2). Webster's American Dictionary (28) defines a game as "any form of play or test of skill." Consistent with this defmition, a "leaming game" may be identified as any form of play or test of skill designed to promote or facilitate leaming. As Gros (13, p. 5) observes, "games are transformed when they are used for educational purposes: they are still games but they are used for a specific aim, to leam particular things, and to develop certain strategies and/or abilities." LITERATURE REVIEW

Leaming games have long been used in business, military, and educational organizations. Kroehnert (19) presents more than one hundred games that can be used by trainers in business and educators. Reliant Energy Arkla has used Zodiak: The Game of Business Finance and Strategy to educate employees about the oil supply industry and how to use financial data (5). The Pennsylvania Institute of CPAs created a web-based game called CPA Career Adventures to educate users about the INTRODUCTION accounting profession and to boost the profession's image for recruiting purposes among high school students (1). Ford is In recent years, computer and video games have exploded among the Fortune 500 companies that have created online in popularity. According to the Entertainment Software games at its website, allowing users to challenge each other to a Association, formerly called the Interactive Digital Software "virtual race" on a lunar racetrack (14). Association, sales of computer and video games in the United States rose 8% in 2002 to $6.9 billion, and they are expected to The availability of Web authoring tools such as continue to show strong growth in upcoming years (10). More Macromedia Flash has opened up the doors for more instructors than 221 million computer and video games were sold in 2002, to create web-based leaming games. Flash is a multimedia and that equates to roughly two games per U.S. household (10). program that can be used to create animated and interactive According to Fattah and Paul (11), computer-based games can effects without the need for complex programming, although it be classified into two categories: stored and online. Stored also includes an optional scripting language (6). Flash uses the games are purchased at a store and come packaged for play on metaphor of film animation, and it can be used for animation, consoles such as Sony's Play Station and Microsoft's X-Box. interactivity, graphics, and sound effects through use of its Online, or web-based games, are played via the Intemet and they embedded ActionScHpt. When Flash is used for animation, for represent the fastest growing segment of the game market. The example, developers can place objects in a beginning frame and Entertainment Software Association estimates that the online an ending frame and the program automatically fills in the gaming audience in the U.S. will rise from 58 million users in frames in between. Flash provides a variety of examples under 2002 to 80 million by 2005 (10). At least 50 major Intemet sites its Help option, and there are also a number of Flash tutorials on offer fi-ee online games; for example, Microsoft's zone.com the web developed for educators (6). A plug-in program is offers more than 170 games, most of which are free (2). required to play Flash movies, but version 4 and above of Netscape Communicator and Intemet Explorer incorporate the While the use of computer and video games is often ability to play Flash animations. According to Macromedia, associated with teenagers and young people, games are more than 97% of all web users can play Flash movies (21), and increasingly being played by virtually all segments of society. for the first time user, it takes only about one minute to Online card and board games have attracted older users, and download Flash Player using a 56K modem. those over 50 as well as women now constitute some of the fastest-growing segments of online game users (2, 11). The Macromedia Flash has been applied for a variety of webmotivations of users to play online games include: stress relief. based games and instructional purposes. The Seattle Public Journal of Computer Information Systems Summer 2005 42

Utility used Flash to create the Recycling IQ Game, which tests users' ability to sort thirty-six common refuse items into garbage, mixed recycling, or glass recycling bins; this awardwinning site was used by more than 11,000 residents in its first year (8). The National Geographic Society used Flash to create an interactive experience at its website called Remembering Pearl Harbor (6). Flash has also reportedly been used to design simulation-based leaming modules to train automotive mechanics (17) and the Center for Teaching and Leaming with Technology at the New York Institute of Technology has applied Flash to develop online and CD-ROM based training for faculty development (27). Klassen and Willoughby (16) point out that the use of leaming games is consistent with the philosophy of "active leaming" which many educational authorities have advocated (e.g., 3, 18, 23). Kohn (18) has suggested that in order to promote a deeper understanding of material, students should be engaged in what they are doing. MacKeachie (23) also says that involving students as active participants results in a positive leaming experience. A training manager for an oil and gas company echoes these statements saying, "Games and simulation work because they get adults to leam by doing" (5, p. 19). Web-based leaming games would seemingly have substantial appeal to the "digital generation" of younger students who have grown up playing computer and video games (24). According to Prensky (24), the use of educational games is necessary to connect with the "video game generation" who do not respond to traditional instructional approaches such as lecture. Based on these observations, it is not surprising that Kirk (15) says that instructors at both the college and secondary levels are increasing their use of games in an effort to motivate leamers and teach complex concepts. A comprehensive report released in 2000 from the Commission on Behavioral and Social Sciences and Education of the National Research Council underscores the growing importance of technology in leaming, and it stresses the need for more research to better understand its application. According to this commission: Because many new technologies are interactive, it is now easier to create environments in which students can leam by doing, receive feedback, and continually refine their understanding and build new knowledge... Computer-based technologies hold great promise both for increasing access to knowledge and as a means of promoting leaming... Much remains to be teamed about using technology's potential: to make this happen, leaming research will need to become the constant companion of software development (4). To this end, various prior studies have been conducted under the broad rubric of investigating the effects of using technology to promote leaming. For example, in separate studies, Cappel and Hayen (7) and Douglas and Van Der Vyver (9) evaluated the effectiveness of using e-leaming materials within university delivered courses. McDonald (22) assessed the effects of using multimedia technology to improve training. The use of web authoring tools to create leaming games has emerged as another important facet of utilizing technology to facilitate student leaming. However, a literature review by the authors uncovered no prior studies that have evaluated the effects of using online games for instructional purposes. Accordingly, this study focuses on three basic research questions whose findings will be of significant interest to educators who are considering the use of online leaming games. First, to what extent will students use online leaming games, where their use is voluntary? For, online games will have no Summer 2005

effect on student leaming where they are not used. Second, if students have access to both online leaming games (in question format) and to online test review questions at their textbook publisher's website, which of these will they prefer? That is, can a leaming game in question format add any perceived value to students beyond what is offered online through a publisher's test bank? Third, what are students' perceptions about the online leaming game experience? Since this last issue is broad in focus, various measures were taken including: student satisfaction levels with the game and users' perceptions about various specific aspects of the game such as whether they thought the game aided their leaming. User feedback was also gathered about the leaming game's perceived strengths and weaknesses via open-ended responses. Considered together, these issues are important since the development and use of games requires time. Educators will likely only want to undertake this effort if it produces sufficient benefits to their students. METHODOLOGY

For purposes of this study, a web-based leaming game, called FlashGame, was created in question format using Macromedia® Flash'"''^. This game is an adaptation of the game "Flashtrivia" by Rosenzweig (23). FlashGame was implemented in two mass sections of a core business course. Introduction to Information Systems, at a large, public university in the Midwest. The online game consisted of three components: a webpage (.htm file), a game shell (.swf file) created in Flash and a text (.txt) file that contained questions. The webpage provided an access point to users to open the game using their Intemet browsers. The webpage then used its plug-ins to display the game shell that contained all pre-designed graphics, animations, and interactivity functions. However, the .swf file was only a "hollow" shell until it loaded the questions from the text file. Some important benefits resulted fi-om the logical link between these three components. The game could be accessed virtually anywhere via a computer with an Intemet connection, and instructors could easily change the content of questions in the text file to update a game without modifying the game shell itself Each online game consisted of ten to fourteen questions about material from an assigned chapter in the course textbook. It was made available to students at the instructors' websites at about the same time the chapter was covered in the course. Use of the game was voluntary and students were not given any course credit for it. However, the instructors encouraged students to use FlashGame as a way to review important course material. In the online game, each question was presented one screen at a time after an initial, welcome screen. As a user accessed each question, four answers "flew in" one at a time in random order from the right side and a point counter began decreasing from 1000. Randomization to the responses was used so students could concentrate on the content of the question each time they played the game instead of memorizing the location of the answers. Respondents had up to 45 seconds to answer a question before no points would be eamed. A cumulative point total was displayed to show the number of points eamed up to that point in the game. A cartoon character also appeared to enhance the visual appeal of the screen and to make the game "feel more ftin." The faster a user answered a question correctly, the more points he/she eamed. Since the game was designed for review (not test) purposes, users were permitted to provide additional answers if their initial response(s) were

Journal of Computer Information Systems

43

incorrect. After all questions were completed, the final screen showed the total points eamed out of the number of points possihle. Students were permitted to retake FlashGame an unlimited number of times. To evaluate users' perceptions of the online leaming game, a survey was administered in class at mid-semester after a total of six FlashGames were made available to students. The survey was conducted at this time since hy then students had gained sufficient experience with the online game to he able to evaluate its use. The survey was completed anonymously and students were encouraged to provide their honest feedback. The survey also included open-ended questions about the perceived strengths of FlashGame and recommendations to improve. The characteristics of the participants are summarized in Table I. As indicated, the same was balanced fairly equally hy gender (52% male, 48% female). Most students (71%) were sophomores and business was the most common major (80%). Participants were also queried about their interest level and usage of video games. Most students indicated they were "very interested" or "somewhat interested" in video games (29% and 48%, respectively), while 23% said they were "not interested." More than one-third (36%) said they play video games at least once per week, while the remainder of the responses varied considerably. A total of 179 students responded to this survey out of 207 enrolled in the course. This represents over 86% of the enrolled students; the remainder were ahsent during the administration of the survey. The responses indicated that 19 subjects made no prior use of FlashGame. Since these subjects were unable to answer the subsequent questions about use of this leaming game, their responses were not included in this study. Thus, except where otherwise noted, the results of this study are based on 160 observations of those students who used the online leaming game. RESULTS FlashGame Usage Students' use of the online game is shown in Table 2. As indicated, more than three-fourths of the 179 students (76%) played all or most of the games prior to completing the survey. An additional 13% reported they played less than half while 11% played none. Statistical analyses showed that students' usage of the online game did not vary significantly at the .05 level by any of the demographic factors presented in Table 1 (gender, student classification, major, video game interest level, or video game usage level). The participants were also queried about their use of the test review questions at the publisher's website. Two-thirds of the 179 students reported at least some use of the publisher's website questions (see Table 2). Preference for the Online Game Versus Questions at the Publisher's Website Of those participants who used both FlashGame and the publisher's test review questions (n = 107), 39% preferred FlashGame, 18% favored the publisher's questions, and 43% indicated no preference. Among those students preferring FlashGame, one user said, "The idea of having something else to study from than just the book is great." Another said, "I find them both helpful but FlashGame is more entertaining." Several users who did not prefer FlashGame mentioned they thought the game made them feel "too mshed" and it reduced their concentration. One user stated, "With FlashGame, I am too Summer 2005

worried about my points." Finally, other participants found value in both FlashGame and the publisher's questions and indicated no preference. As one student said, "The more study aids the better." Another pointed out, "I use the publisher's site first to get it all in my head, then FlashGame for fun and review."

TABLE 1 Participant Characteristics Percentage Gender Male Female

52.2% 47.8%

Student classification: Senior Junior Sophomore Freshman

6.9% 21.3% 70.6% 1.3%

Major: Business Non-business Undecided

80.0% 14.4% 5.6%

Interest level in playing video games: Very interested Somewhat interested Not interested

28.8% 48.1% 23.1%

How often playing video games: At least once a week One or a few times per month Less than once a month Never

36.3% 21.9% 24.4% 17.5%

TABLE 2 Usage and Perceptions of FlashGame Versus Questions at the Publisher's Website Percentaee Number of FlashGames Played All Most Less than half None

46.4% 29.6% 13.4% 10.6%

Number of hours spent taking review questions at publisher's site: More than 2 1-2 Less than 1 Zero

9.4% 30.0% 27.5% 33.1%

Overall Preference FlashGame Questions at publisher's wehsite No preference

39.3% 17.7% 43.0%

Journal of Computer Information Systems

44

User Satisfaction and Perceptions of FlashGame Table 3 presents the results for ten items that address users' reactions to the online game experience. All items are based on a seven-point response scale; the higher the reported mean, the more positive the outcome. As indicated in the table, the mean overall satisfaction rating was 5.92. Additional analyses revealed that user satisfaction levels with the online game did not vary significantly across any of the demographic factors considered. All ten items in Table 3 except one had a mean rating of 5.46 or higher. Subjects showed the highest level of agreement to the items that FlashGame helped them prepare for

exams (6.22) and that this game should be used in this course in the future (6.18). The item that showed the greatest level of disagreement was whether the online game provided enough time to answer each question (4.87, SD 1.68). To minimize response bias, three items were stated negatively; these "reversecoded" items are indicated by an "(re)" in Table 3. For example, in the survey, the second last item in Table 3 was worded, "Taking FlashGame was too stressful," while for reporting and analysis purposes, it is stated, "FlashGame was not too stressful." This allows easier comparison across all responses in the table.

TABLE 3 Overall Satisfaction and User's Perceptions of the Web-based Game Mean

SD

5.92

1.01

6.22

0.87

5.58

1.11

4.87

1.68

5.81

1.15

5.46

1.11

6.07

1.16

5.73

1.04

6.08

1.32

Overall satisfaction # Helped me prepare for exams Wasftin Provided enough time to answer each question (re) Helped me understand important course material Was interesting Was worth the time and effort (re) Helped me identify concepts that should be studied in more detail Was not too stressful (re) Should be used in this course in the future #

6.18 1.05 This item is based on the seven-point scale: 7=extremely satisfied, 6=moderately satisfied, 5=slightly satisfied, 4=don't know/no opinion, 3=slightly dissatisfied, 2=moderately satisfied, and l=extremely dissatisfied. The other items are based on the seven-point scale: 7=strongly agree, 6=moderately agree, 5=slightly agree, 4=don't know/no opinion, 3=slightly disagree, 2=moderately disagree, and l=strongly agree.

(re) Indicates a reverse coded question. These items were asked in a "negative" way in the survey, but they are presented in this table in an opposite (positive direction) to facilitate response comparison aeross all items.

Participants were also asked to rate FlashGame on four specific dimensions using the five-point scale: 5 "excellent," 4 "good," 3 "satisfactory," 2 "somewhat unsatisfactory," and 1 "extremely unsatisfactory." The highest rated dimensions were "ease of use" (4.46, SD 0.66) and "instructions" (4.36, SD 0.66). Subjects rated "point scoring" (3.75, SD 0.94) and "feedback about results" (3.61, SD 1.04) somewhat lower. Finally, the survey queried respondents about the perceived difficulty of the questions in FlashGame. Almost 95% of respondents indicated that the questions used in FlashGame were "at about the right level of difficulty." Only 2% considered the questions "too easy," while about 3% said they' were "too difficult." Perceived Benefits of the Online Game

most commonly cited benefits of FlashGame were: (1) it helped students test their understanding of material and identify topics to study in more detail; (2) it clarified expectations about the type of topics or questions that could be on the exam; and (3) it made leaming more fun or interesting. Additional cited strengths focused on specific aspects of the game's content or design. One participant called FlashGame "a great study tool." Other user comments included: "Much more interesting than the standard Q & A"; "It lets us get a feel for the test"; "It is actually a fun way to study for the exams"; and "It helped me learn concepts I missed from reading the chapter." Recommended Improvements

Table 5 presents respondents' suggestions for improving the online game. By far, the most common recommendation. In open-ended responses, participants identified various mentioned by 55 users, was to include more questions in strengthsofthe web-based game. As shown in Table 4, the three FlashGame. Twenty-two subjects said that the point counter Summer 2005 JournalofComputer Information Systems 45

should be slowed down or it should start after the user has first had a chance to read the question (rather than as soon as the question appears). One user commented, "The way the points start to decrease right when a question starts is annoying. It would be nice if there were a 5 to 10 second grace period first." Fourteen respondents said that more questions from FlashGame should appear on tests in the course. Respondents also had various other ideas for improving feedback or other design elements of the game (Table 5). For example, some participants

wanted immediate feedback when they answered a question correctly (rather than having the game go to the next question) or they wanted the correct answer identified when they missed a question. Three subjects pointed out that the final screen should not say how many questions were answered correctly out of the total number of questions, because users were required to provide the correct answer before they could proceed to the next question. Thus, this message always showed a "perfect score" so it was not needed.

TABLE 4 Perceived Benefits of the Online Game

Tests understanding; promotes leaming; helps me prepare for test; identifies topics I need to study in more detail

No. of Users Mentioning* 46

Clarifies expectations; gives examples of topics or questions that could be on test

32

Makes leaming fun and/or interesting

24

Specific FlashGame design aspects: Variety of questions, good content Game environment; scoring, pace, interactivity, feedback Allows additional attempts at answers Ease of use Graphics, animation character Easy, online access Randomized answer order

12 10 10 6 6 4 3

*This indicates the number of participants who provided one or more of the responses shown in each respective category. Some users' responses fit more than one category.

TABLE 5 Recommended Improvements No. of Users Mentioning* 55

Include more questions in the online game Modify the point counter: slow it down or don't start it until after the user has had time to read the question

22

Put more FlashGame questions on the test

14

Improve feedback: a) Identify more clearly what the correct answer is instead of going immediately to the next question b) Identify what the correct answer is when I get the incorrect answer and/or include explanations of correct answer c) Message at end; no. of correct out of no. possible is not meaningful. (Since it always says you got 10/10 when you didn't, people will guess until all answers are eorrect.) Eliminate the timer Eliminate the use of points Allow printing of questions/answers Randomize the order of questions on each retake •This indicates the number of participants who provided one or more of the responses shown in each respective category. Some users' responses fit more than one category.

Summer 2005

Journal of Computer Information Systems

46

DISCUSSION AND CONCLUSIONS As noted earlier, this study focuses on three research questions. An analysis of the results for each of these questions is presented below. Usage ofthe Online Game Since playing the online game was voluntary, an initial concern in the minds of the instmctors was whether students would play the game on their own time. For, if any benefits were to be realized from the online game, it would first have to be used. The results of this study show that at the time of the administration of the survey at mid-semester, more than threefourths of the students played most or all of the FlashGames, while only 11% played none of them. These findings are rather encouraging, and they suggest that with the proper reminders and encouragement from instructors, most students will take advantage of online games that instructors make available to them, even in a core business course that is required of all business majors. Preference for the Online Game or the Questions at the Publisher's Website As noted earlier, FlashGame and the test review questions provided at the publisher's website had a degree of similarity in that they both presented questions about assigned textbook chapters. One difference between them was that the textbook publisher's website included far more questions than the ten to fourteen multiple choice questions provided for each chapter by FlashGame. Another important difference is that FlashGame attempted to add a greater element of "fun" into the experience by incorporating graphics, animation, counters, and timers. The results of this study demonstrate that among those students who expressed a preference, FlashGame was preferred by a more than two to one margin (39% versus 18%). The open-ended responses revealed that the "fun" factor of FlashGame had a significant appeal to some students, while a lesser number of students felt too pressured or irritated by the counters. It should be noted that the largest response among users was "no preference" at 43%, Many of these respondents indicated that they considered both FlashGame and the book publisher's website questions to be useful in the study process. This was a positive outcome in the minds of the instructors because FlashGame was not designed to be a replacement for the questions at the publisher's website, but rather an additional resource for students in their study process. User Perceptions ofthe Online Game Various measures were taken to assess students' overall opinions of the online game. Most notably, some positive findings included that, on a seven-point scale, subjects rated their overall satisfaction with FlashGame (5.92); that the online game helped them prepare for exams (6.22); and that FlashGame should be used in the future in this course (6.18). In addition, the open-ended responses revealed that many students thought FlashGame helped them to understand course material, identify topics they should study in more detail, clarify expectations about the types of test questions they might later see, and it made leaming more fun or interesting. The most commonly cited improvements included to add more questions to the online game and to slow the counter down or make it less distracting. Overall, the results of this study suggest that web-based Summer 2005

leaming games appear to have substantial promise to support information systems instruction. Many students perceived the web-based leaming game as fun and something that aided them in leaming important course material. Based on the results of this study and reflection about the authors' own experience or "lessons leamed" from this effort, some implementation suggestions are offered below to other instructors who may be considering the use of a similar leaming games approach. IMPLEMENTATION ISSUES Careful, Up-front Planning is Essential It is probably fair to say that many instructors are concemed first with teaching and only later with testing. In other words, they do not like to think about the details of a test until shortly before it is administered. A leaming games approach like the one conducted in this study requires instmctors to do considerably more thinking about their test questions "on the front end" of instruction rather than "after the fact," so as to minimize the degree of overlap between the questions in the online game and actual test questions. If this is not done and many of the same questions appear on the exam, the entire approach could degenerate into an exercise of students memorizing questions/answers for the exam without really leaming the material, thereby defeating the purpose ofthe effort. In this study, the instructors used very few FlashGame questions (about 5%-8% of them) on the actual course exams. The Use of Online Games Requires an Investment of Time by Instructors If a leaming game is not made available through another source such as a textbook publisher, then an instmctor can use a product such as Macromedia Flash which allows for the creation of games without the need for complex programming. Instmctors may want to consult the Help option in Flash or published sources such as Rosenzweig (25) to see illustrative examples of possible leaming games. In this study, once the initial design or shell of the leaming game was completed, the major time investment was deciding which questions to use for the game and then formatting them into a text file. For reasons stated above, it was necessary to carefully select questions to minimize the degree of overlap between the questions of the leaming game and those of the tests. Unfortunately, in many cases, instructors will likely be challenged to find enough "good questions" in a publisher's test bank for both the leaming game and an exam, so there will likely be a greater need for instmctors to write additional questions of their own. However, for those instmctors who are willing to make this time commitment, it sends a message to students that you care about their leaming. It demonstrates that the instmctor is willing to do something extra to help them leam and a number of students appreciate that. Of course, instmctors can take other media format such as CD instead ofthe Intemet to deliver teaming games to students. However, online games were chosen for this study as the interactive and multimedia nature of Web-based materials may lead to student enthusiasm and an increased desire to leam (20). In addition, adding online games to faculty websites can enrich web-enhanced and Intemet-based courses which support educational institutions' intentions of exploring the use of the Intemet for teaching (12). Because Macromedia Flash game is already a rich-media product and can be recognized by most Intemet browsers, instmctors do not need to invest much time on designing the website rather than creating hyperlinks for

Journal of Computer Information Systems

47

these online games.

This Study Represents Only One Possible Way to Use and Study Web-based Learning Games to Support Instruction

Manage Students' Expectations about the Learning Game It is very important to establish elear expectations in students about the online game beginning early in the course and to remind them to use it periodically thereafter to promote its use. Presenting a demonstration of the game before students use it is highly desirable to make sure the game's instructions and use are clear. It is also important to dispel any myths or misconceptions students might have about use of the game. As noted earlier, by far the most common suggestion made by students to improve the online game was to include more questions. Some specific comments from users included: "Have more questions to cover all possible exam questions"; and "Take questions that aren't on the test out and put more questions that are on the test in. Make them word for word." Thus, some students will look for "short cuts" in their study process, and they might think that just because they can answer all the questions on the leaming game correctly, they will get 100% on the exam. It is important to stress that the leaming game only covers a sampling of test topics, it is not meant to be comprehensive in addressing all topics a subsequent test could cover. Students should be encouraged to use the online game as an additional input in their study process. It does not replace other aspects of the study process such as reviewing the book and class notes or taking the test review questions at the publisher's website. It is also desirable to select leaming game questions at the same level of difficulty as questions students will encounter on an exam. If leaming game questions are overly easy, students may minimize the amount of study they will do, and if they are overly difficult, students could get discouraged with the use of the game. Gather Feedback from Students about the Use of the Game and Continually Improve It In designing an online leaming game there are many decisions to make. What should screens look like? What feedback will be provided to users? If a game is to be presented in question format, what question format(s) will be used? How many answers will users be allowed to provide to each question? Will re-takes be permitted? These are only a sampling of issues that surface. Instmctors should not become overly concemed if they do not "get everything right" the first time, or even the second. For example, this game was designed so that when users provided a correct answer, it took them to the next question. If an incorrect answer was given, that answer was eliminated and the question was displayed until the user provided the correct answer. In the survey responses, some students commented that they wanted more feedback after providing each einswer. Based on this, the authors have decided that a message "correct" or "incorrect" will be displayed after each answer in future versions of the game. Also, as noted earlier, a few users pointed out that the message on the last screen showing how many questions were answered correctly out of the total possible was not necessary since the game required users to provide a correct answer before proceeding to each question. Thus, this message will be eliminated in future iterations of the game. As with any system, design improvements can always be envisioned later. The important thing is for instructors to gather user feedback about the use of the games used and then to look for opportunities to improve it. Summer 2005

In reality, there are a host of other options. This study involved the voluntary (non-graded) use of online games by students outside the class setting for purposes of course leaming and review. Students could provide additional answers until they got a question correct and they were allowed to play the game multiple times. Altematively, online games could be played inside or outside of class in a variety of different ways and for a variety of different purposes. Leaming games could also be played by individuals or groups. They could be used to generate student interest and/or questions for class discussion or they can assess performance (and be counted as a quiz toward a student's course grade). The design of a leaming game could also take on many forms. For example, sound was not used in the leaming game reported on in this study, because it was expected that most students would play FlashGame in the campus lab where students do not have access to sound (unless they make the extra effort to check out headphones from the lab monitor). However, it is quite easy to add sound effect to a game developed in Flash to increase the "fun" and appeal of game. Because the use of online leamijig games is at an early stage of development, other researchers are^encouraged to investigate their use in a variety of contexts. This research will provide instructors with important insight about how to design and use web-based leaming games most effectively. Finally, different research approaches could be taken to evaluate the use of online games. This study was exploratory in nature. The authors were interested in gaining insight about the relatively unexplored domain of how students perceive the use of web-based leaming games to assess their potential to support education. The authors also wanted specific feedback from students about how to possibly improve the design of the game in future iterations. An altemative approach for future research would be to assess the impact of online leaming game use on student performance, i.e., do students who use online leaming games perform better than those who do not in terms of their overall course grade? With additional research, more will be known about the use of online games so that they can be applied as effectively as possible to support leaming objectives.

REFERENCES 1. 2.

3.

4.

5. 6.

Anonymous. "Attracting Students," The Practical Accountant, 34:12, December 2001, p. 24. Bames, G. "Play Time: Online Games are Booming; And the Growth Isn't Coming from Kids," Wall Street Journal, March 5,2002, p. R6. Bonwell, C.C. and J.A. Eison. Active Learning: Creating Excitement in the Classroom. ASHE-ERIC Higher Education Report No. I. The George Washington University, School of Education and Human Development, Washington, DC, 1991. Bransford, J.D., A.L. Brown, and R.R. Cocking, eds. How People Learn: Brain, Mind, Experience, and School Committee on Developments in the Science of Learning. Commission on Behavioral and Social Sciences and Education of the National Research Council, National Academy Press, 2000, ISBN: 0-309-07036-8. Brotherton, P. "Let the Games Begin," American Gas, 81:3, April 1999, p. 19. Bull, G., G. Bull, and M. Hofer. "Moving Images," Learning and Leading with Technology, 29:5, Febmary

Journal of Computer I nfor .nation Systems

48

7.

8. 9.

10. 11. 12.

13.

14. 15.

16.

17.

2002, pp. 42-45. Cappel, J.J. and R.L. Hayen. "Evaluating E-leaming: A Case Study," Journal of Computer Information Systems, 44:4, Summer 2004, pp. 49-56. City's Recycling IQ Game Wins Swana Public Education Award," Biocycle, 44:12, December 2003, p. 18. Douglas, D.E. and G. Van der Vyver. "Effectiveness of Eleaming Course Materials for Leaming Database Management Systems," Journal of Computer Information Systems, 44:4, Summer 2004, pp. 41 -48. Entertainment Software Association Website. Available at www.theesa.com. Accessed August 12, 2004. Fattah, H. and P. Paul. "Gaming Gets Serious," American Demographics, 24:5, May 2002, pp. 38-43. Garrett, N., T.D. Lundgren, and K.S. Nantz. "Faculty Course Use of the Intemet," Journal of Computer Information Systems, 41:1, 2000, pp. 79-83. Gros, B. "The Impact of Digital Games in Education," First Monday (Peer reviewed online joumal), 8:7, July 2003. Available at www.firstmonday.org/issues/issue8_7/ xyzgros/index.html. Accessed August 4, 2004. Hopper, J. "The (Online) Games People Play," Broadband, 43:1, January 7, 2002, p. 16. Kirk, J.J. "An Unofficial Guide to Web-based Instructional Gaming and Simulation Resources," ERIC Document Reproduction Service ED472675,2001. Klassen, K.J. and K.A. Willoughby. "In-class Simulation Games: Assessing Student Performance," Journal of Information Technology Education, 2, 2003, pp. 1-13. Klyde, M. "Building a Bridge on the Web: Cognitive and Psychomotor Training with Simulations Built in Flash,"

Summer 2005

18. 19. 20.

21. 22.

23.

24. 25.

26.

27.

28.

Journal of Interactive Instructional Development, 15:3, Winter2003, pp. 8-11. Kohn, A. "Students Don't 'Work' - They Leam," Education Week, September 3, 1997, p. 3. Kroehnert, G. 100 Training Games. New York: McGrawHill, 1993. Liebowitz, J. and G.J. Yavergbaum. "Making Leaming Fun: The Use of Web-based Cases," Journal of Computer Information Systems, 39:1, 1998, pp. 14-29. Macromedia website. Available at www.macromedia.com. Accessed August 1, 2004. McDonald, D.S. "Improved Training Methods Through the Use of Multimedia Technology," Journal of Computer Information Systems, 40:2, Winter 1999-2000, pp. 14-25. McKeachie, W. McKeachie's Teaching Tips: Strategies, Research and Theory for College and University Teachers, 11th ed. Boston: Houghton Mifflin, 2002. Prensky, M. Digital Game Based Learning. New York: McGraw-Hill, 2000. ISBM: 0071363440. Rosenzweig. G. Macromedia® Flash'^'*' MX ActionScript for Fun & Games. Quc Publishing, 2002. ISBM 0-78972799-4. Spannaus, T. "Environments for the Twitch Generation: Leaming Games," Lily North Educational Conference, Traverse City, MI, September 12,2003. Uttendorfer, M. "Advanced Instructional Delivery Capability in a Flash," Journal of Interactive Instructional Development, 15:1, Summer 2002, pp. 6-14. Webster's American Dictionary. New York: SMITHMARK Publishers, 1999.

Journal of Computer Information Systems

49