British Journal of Educational Technology (2008) doi:10.1111/j.1467-8535.2008.00852.x
Colloquium
Developing evaluative indicators for educational computer games Eric Zhi Feng Liu and Chun Hung Lin Address for correspondence: Graduate Institute of Learning & Instruction, National Central University, No.300 Jung-da Road, Chung-Li City, Taoyuan, Taiwan 320. Email:
[email protected] and
[email protected]
Introduction In recent years, in order to motivate students to learn, an increasing number of teachers have been attempting to integrate educational computer games into their teaching process (Roblyer, 2006). However, given the vast number of existing educational computer games, the question is whether any of them can improve student learning. If the answer is no (McGinley, 1991), the next question is whether there exist any evaluative indicators that can help teachers or parents to select the games that are suitable for their students or children, respectively. Fileni (1988) believed that games can enable students to develop their skills and improve their learning. Duffield (1990) demonstrated that educational computer games provide greater opportunities for students to put their learning into practice and also motivate them to learn. Previous studies (Alessi & Trollip, 2001; Dempsey, Lucassen, Haynes & Casey, 1996; Roblyer, 2006) have pointed out that games entertain, instruct, change attitudes and enable skill development. The efficacy of student learning can be improved by designing such educational computer games appropriately. The elements of educational computer games include graphics, sounds, human– computer interfaces, gameplay and storylines (Prensky, 2001; Shneiderman & Plaisant, 2004). The graphics comprise still images or picture effects such as 3D objects, textures and full-motion videos. In this case, it is necessary to consider the compatibility between size and style. The sounds include music and sound effects. The sounds employed in an educational computer game could be the background music or verbal reminders of certain consequences. The human–computer interface includes the objects and menu system that users can click on; these are used to control the characters’ actions during the game. The main principle to be taken into consideration while designing a human–computer interface system is its user-friendliness. Gameplay is included to provide entertainment and interactivity while playing an educational computer game. The main objectives of the gameplay are to create interesting interactions and challenges. The storyline is a sequence of scenarios that provide information to the © 2008 The Authors. Journal compilation © 2008 British Educational Communications and Technology Agency. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA.
2
British Journal of Educational Technology
users, which is the main aim of an educational computer game. The purpose of designing a storyline is to set challenges, and goals that users can experience and achieve. Method Content analysis In this research, content analysis was used to analyse existing educational computer games, which were obtained through the three most famous search engines in Taiwan (Yahoo!, Google and Yam [http://www.yam.com]) and from computer game stores. A total of 196 educational computer games were analysed in this research. Delphi technique The Delphi technique is a systematic interactive forecasting method for obtaining forecasts from a variety of independent experts. Questionnaires are administered in two or more rounds to a panel of carefully selected experts. By following specific procedures and steps, the Delphi technique can be used to integrate different expert opinions, and ultimately, to reach a consensus. In this study, the following people were invited to serve as experts in the Delphi survey: six educational technology experts, six educational psychology experts, six game design experts, six elementary school students with at least a year of experience in using educational computer games for learning, six junior high school students with at least a year of experience in using educational computer games for learning, six senior high school students with at least a year of experience in using educational computer games for learning and six school teachers with at least a year of experience in using educational computer games for teaching. In the first round, based on their opinions and the results of the content analysis of the 196 games, the experts listed the necessary evaluative indicators. The questionnaires for the second round were produced based on an analysis of these lists. The final results of the second round included five categories, which comprised a total of 60 items. Similarly, after integrating the results of the second round, the final results of the third round included five categories, with 43 items. In the fourth round, after analysing the results of the third round, the opinions of the experts were found to be consistent, and the Delphi survey was concluded. Findings Characteristics of educational computer games In this study, 196 educational computer games were analysed. The results revealed that there are many different genres of educational computer games; puzzle games comprise the most common genre (Table 1). Further, the multimedia materials used most often are pictures (pictures: 196; sounds: 168; animated items: 120). For games, the most detailed documentation pertains to the software and hardware requirements (n = 143). On the other hand, descriptions of suitable users are not documented in several educational computer games (n = 120). However, most educational computer games can provide suitable feedback (number of games that provide suitable feedback: 188; © 2008 The Authors. Journal compilation © 2008 British Educational Communications and Technology Agency.
Colloquium
3
Table 1: Genres of educational computer games Genre Role playing Puzzle Simulation Action adventure Shooting Sports Strategy
Number of games
Ranking
17 60 40 21 20 24 14
6 1 2 4 5 3 7
provide the status of main characters: 110; record user profiles: 60) and are designed to be used in language learning (language: 84; science: 40; math: 35; history: 22; geography: 15). Evaluative indicators Based on the results of the content analysis and Delphi survey, we obtained 43 indicators and classified them into five categories for evaluating educational computer games: game information, multimedia, interface design and structure, content, and feedback. Game information: • • • •
Are software and hardware requirements documented? Are there any descriptions of suitable users? Is related information documented in the game? Is the learning subject matter documented in the game?
Multimedia: • • • • • • • •
Are the pictures and animations in the game related to the learning content? Are the multimedia materials in accordance with the learning goals? Is the game attractive? Are animations used to increase users’ interest? Are the pictures, sounds, and animations arranged appropriately? Are the pictures clear? Are pictures used to increase users’ interest? Are sound effects used to increase users’ interest?
Interface design and structure: • • • • • • •
Is the order of the contents reasonable? Can a learner obtain a record of the gameplay? Does the game provide individual learning opportunities? Does the game interact closely with the user? Is the gaming screen clear and easy to understand? Is the operation of the game easy to learn? Does the game allow the use of quick keys? © 2008 The Authors. Journal compilation © 2008 British Educational Communications and Technology Agency.
4
British Journal of Educational Technology
• Is the design of the gaming screen attractive? • Is the important information presented clearly? • Does the game menu include a help function? Content: • Is the storyline of the game in accordance with the activities related to the learning content? • Are the descriptions in the game short and easy to understand? • Is the content presented in a conceptual order? • Do users require help to comprehend the difficult learning materials? • Are the challenges in the game related to the learning materials? • Are the learning goals documented in the game? • Does the game context match the learning context? • Is the information in the game accurate? • Is the content of the game interesting? • Does the game use rich media? • Are the game’s resources in accordance with its storyline? • Are the pictures, animations, sounds, and language used in the game suitable for the user? Feedback: • • • • • • •
Is the feedback provided in a timely manner? Are the hints in the game easy to understand? Is the speed of the game controlled by the user? Does the game provide a proper assessment of the user’s skills? Can the game report a user’s current status? Can the learning process be saved during the game? During gameplay, is there any information that updates the user with regard to the current situation? • Can a user browse through the feedback history at any point in time? • Does the game allow a teacher to edit the feedback or the game’s design?
Discussion In the future, it will be important for game designers to develop educational computer games that are both entertaining and educational. Currently, while several such computer games are available, not all of them are suitable for learners. Teachers or schools should select the best and most suitable educational computer games. In this study, the indicators for evaluating educational computer games were developed through a content analysis of existing games and the use of the Delphi technique. In order to obtain different opinions from a wide range of user groups, we recommend that a larger number of user groups be interviewed. This will promote the development of educational computer games and provide a more complete idea of the evaluative indicators required to assess educational computer games. © 2008 The Authors. Journal compilation © 2008 British Educational Communications and Technology Agency.
Colloquium
5
Acknowledgements The authors would like to thank the National Science Council of the Republic of China for financially supporting this research under Contract Nos. NSC 94-2520-S-008-007, NSC 96–2520-S-008-003 and NSC 96-2524-S-008-002. References Alessi, S. M. & Trollip, S. R. (2001). Multimedia for learning: methods and development (3rd ed.). Boston, MA: Allyn & Bacon. Dempsey, J. V., Lucassen, B. A., Haynes, L. L. & Casey, M. S. (1996, April). Instructional applications of computer games. Paper presented at the Annual Meeting of the American Educational Research Association, New York, USA. (ED394500). Duffield, J. A. (1990). Designing computer software for problem-solving instruction. Educational Technology Research & Development, 39, 1, 52–60. Fileni, F. (1988). Educational and cognitive aspects of videogames. In D. Crookall et al (Eds). Simulation-gaming in education and training (pp. 252–260). Oxford: Pergamon Press. McGinley, R. (1991). Start them off with games! The Computing Teacher, 19, 3, 49. Prensky, M. (2001). Digital game-based learning. New York, NY: McGraw-Hill. Roblyer, M. D. (2006). Integrating educational technology into teaching (4th ed.). Upper Saddle River, NJ: Merrill Prentice Hall. Shneiderman, B. & Plaisant, C. (2004). Designing the user interface: strategies for effective human– computer interaction (4th ed.). Boston, MA: Addison Wesley.
© 2008 The Authors. Journal compilation © 2008 British Educational Communications and Technology Agency.