Computers & Education 56 (2011) 127–137
Contents lists available at ScienceDirect
Computers & Education journal homepage: www.elsevier.com/locate/compedu
Effects of constructing versus playing an educational game on student motivation and deep learning strategy use Nienke Vos*, Henny van der Meijden 1, Eddie Denessen 2 Radboud University, Behavioural Science Institute, Montessorilaan 3, P.O. Box 9104, 6500 HE Nijmegen, The Netherlands
a r t i c l e i n f o
a b s t r a c t
Article history: Received 16 February 2010 Received in revised form 12 August 2010 Accepted 12 August 2010
In this study the effects of two different interactive learning tasks, in which simple games were included were described with respect to student motivation and deep strategy use. The research involved 235 students from four elementary schools in The Netherlands. One group of students (N ¼ 128) constructed their own memory ‘drag and drop’ game, whereas the other group (N ¼ 107) played an existing ‘drag and drop’ memory game. Analyses of covariance demonstrated a significant difference between the two conditions both on intrinsic motivation and deep strategy use. The large effect sizes for both motivation and deep strategy use were in favour of the construction condition. The results suggest that constructing a game might be a better way to enhance student motivation and deep learning than playing an existing game. Despite the promising results, the low level of complexity of the games used is a study limitation. Ó 2010 Elsevier Ltd. All rights reserved.
Keywords: Games Elementary education Interactive learning environments Learning strategies Media in education
1. Introduction In the last decades, views on learning and instruction have changed fundamentally. In most contemporary theories of learning, generally referred to as social constructivist learning theories, learning is seen as a process of knowledge construction with an emphasis on active and self-regulated learning (Shuell, 2001). Constructivist learning approaches underline the idea of an active, experiencing student in a situation where knowledge is not transmitted to the student, but constructed through activity or social interaction. There are a few basic assumptions of constructivist learning. First, learning can be seen as an active process of knowledge achievement (Driscoll, 1994). Students are said to construct their knowledge, based on their pre-knowledge and interest. Next to knowledge construction, self-regulation is said to be important in learning. This means students can manage their learning process. According to Perkins (1999), knowledge construction and self-regulation leads to better understanding, remembering and actively use of knowledge. This is also shown by research of Greene and Azevedo (2009), who demonstrated that students who used constructivistic self-regulated learning strategies were more likely to obtain deep, conceptual understanding of complex topics than when they would limit their learning to obtaining declarative knowledge. Constructivist theories of learning have had noticeable impacts on education practice, for example on teacher didactics, which has shifted from direct teaching to the promotion of active learning in classrooms. Additionally constructivist theories ask for different learning environments. According to Driscoll (1994) these constructivist environments are complex, realistic, and meaningful. A complex learning environment will engage and challenge students to construct new knowledge, which they can preferably apply in everyday life, in a realistic setting. This setting can make learning a personal meaningful experience, which contributes to student motivation to learn (Shaffer, Squire, Halverson, & Gee, 2004). This motivation is essential according to Bransford, Brown, & Cooking, (2000) to reach a deeper level of learning. Learning according to the constructivist learning theory invokes a high level of active engagement of students. Opportunities for constructivist learning environments have grown since the increasing role of Information and Communication Technology (ICT) in daily life. ICT offers a great spectrum of interactive learning possibilities (Tam, 2000). For example, television and computers bring some realism into the classroom by watching movies and searching on the Internet for information. The technological developments of the computer have ensured that there are numerous opportunities to offer the curriculum in an authentic, complex and meaningful context.
* Corresponding author. Tel.: þ31 6 42720126/þ31 24 3615819; fax: þ31 24 361621. E-mail addresses:
[email protected] (N. Vos),
[email protected] (H. van der Meijden),
[email protected] (E. Denessen). 1 Tel.: þ31 24 3615763. 2 Tel.: þ31 24 3613080. 0360-1315/$ – see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2010.08.013
128
N. Vos et al. / Computers & Education 56 (2011) 127–137
One of these opportunities is game play. When playing games, learners are emerged in personal learning experiences, which are less accessible in regular educational settings. During game play learners can reflect on their actions and they can draw conclusions, adjust their hypotheses and test them again, if needed (Gee, 2003). Games facilitate a so-called ‘trial-and-error’ approach that has been considered supportive of the development of logical thinking and problem-solving skills. Games can provide experiences across various situated contexts that enable learners to understand complex situations (Shaffer et al., 2004). These above mentioned characteristics of game playing could contribute to knowledge construction (Gee, 2003). Besides that, students are in control of the game (Kirriemuir & McFarlane, 2004), which indicates a certain level of self-regulation. In sum, games very well fit with the social constructivist theory of learning. 1.1. Effects of gaming 1.1.1. Intrinsic motivation Computer games have become very popular, and economically the games industry is one of world’s largest industries. Playing games has become an important activity in the daily life of children. A key concept that frequently emerges in the literature on games is motivation. Games have the potential to enhance motivation for learning, because they stimulate curiosity and interest by presenting learning activities in meaningful contexts in which the learner is in control (Kirriemuir & McFarlane, 2004). Last decade, many empirical studies showed that games enhance student motivation (Burguillo, 2010; Cordova & Lepper, 1996; Lopez-Morteo & Lopez, 2007; Rosas et al., 2003, Tüzun, Yilmaz-Sollu, Karakus, Inal, & Kizilkaya, 2008). For example, Cordova and Lepper studied the effects of introducing a meaningful context in a mathematics game on the intrinsic motivation and learning outcomes of elementary school children. They found that providing concrete contextualizations for games (i.e. solving puzzles) by embedding them in a playful environment enhanced student motivation and learning outcomes significantly. The students became more deeply involved in the activities, they attempted to use more complex operations, and as a result, they learned more. Tüzun et al. (2008) designed a three-dimensional educational game about geography for elementary school children. When comparing the motivation of students who learned in the game-based learning environment to those who learned in a traditional school environment, they found that students demonstrated statistically significant higher levels of intrinsic motivation in the game-based environment. In above mentioned cases, educational games could thus provide a higher level of intrinsic motivation than a traditional school context. 1.1.2. Deep learning and achievement By playing games, children are confronted with problems they must overcome if they want to reach their goals. Through trial-and-error, children learn from their mistakes and their efforts to find a solution for the problem (McFarlane, Sparrowhawk, & Heald, 2002). In this way, they develop problem solving skills and thinking skills. Gee (2003) stated that games are very suited to the development of inquiry skills; children learn by formulating hypotheses, and testing them. Papert (in Beeksma & Van der Hulst, 2005) stated that the game drags the student into a process of deep learning. Marton and Saljö (1976) studied student’s approach to learning, referring to the strategies used by students during a learning task. They discerned two approaches: surface learning and deep learning. Surface learning means that the student tries to memorise the given information by detail (Biggs, Kember, & Leung, 2001). Deep processing or deep learning involves the critical analysis of new ideas, linking them to already known concepts and principles, and leads to understanding and long-term retention of concepts so that they can be used for problem solving in unfamiliar contexts. Deep learning represents approaches that focus on integration, synthesis, and reflection. Students who use deep approaches to learning perform better as well as retaining, integrating and transferring information at higher rates than students using surface approaches to learning (Laird, Shoup, Kuh, & Schwarz, 2008). Deep approaches are usually preferred because students look beyond the sign associated with information (surface approaches) to the more important underlying meaning (Marton & Saljö, 1976). Also, deep approaches can be linked to deep learning outcomes (Abbott, Townsend, Johnston-Wilder, & Reynolds, 2009; Biggs et al., 2001; Ramsden, Beswick, & Bowden, 1989). Students have their own preference learning approach, which can either be more surface, or deep. This preference learning approach is not only personally stated, teaching context also seems to play an important role in students learning approach. The educational context could enhance student deep level of learning approach, which could be positive for students learning outcomes (Biggs et al., 2001; Hamilton & Tee, 2010). Empirical research of Hamilton and Tee suggests that when deep strategies are asked from the context, a student will use a deep strategy. This would then lead to deep learning outcomes. Kim, Park, and Baek (2009) concluded in their research on meta-cognitive strategies in game-based learning, that meta-cognitive strategies, in terms of thinking aloud and modeling had positive effects on social problem solving, which affected achievements in learning and gaming. In their paper the authors concluded game-based learning, when using meta-cognitive strategies can be an effective learning environment for increasing students’ performance. In other words, games could thus enhance students level of deep learning, by appealing to critical thinking skills, problem solving skills, decision making, knowledge transfer and meta-analytic skills (Gee, 2003; Kirriemuir & McFarlane, 2004; Wideman et al., 2007). Effects of games on deep learning have also been investigated by Oyen and Bebko (1996). They found games to enhance children’s interest and engagement towards a learning task. Oyen and Bebko compared a traditional lesson with a game lesson and found the primary school children in the game condition enjoyed the task more than the children in the traditional lesson condition. The children in the game lesson did not want to stop playing the game, and therefore rehearsed the content more than the children in the traditional lesson condition. This could suggest a deeper level of knowledge construction. Several other studies on deep learning in ICT contexts have been conducted (Crisp & Ward, 2008; Greene, Bolick, & Robertson, 2010; Ke & Grabowski, 2007; Kebritchi, Hirumi, & Bai, 2010; Kwon & Cifuentes, 2009; Lopez-Morteo & Lopez, 2007; Owston, Wideman, Ronda, & Brown, 2009; Rosas et al., 2003). Kebritchi et al. (2010) found significant improvement of mathematic achievement of students who worked with a computer game, unlike the students who didn’t work with the computer game. Other empirical research showed teacher trainees learned deeper when using computer assisted assessments (Crisp & Ward, 2008). Similar results were found by Kwon and Cifuentes (2009), who found students to have a deeper understanding of science concepts when working collaboratively with a computer-based concept mapping instrument. Additionally, Greene et al. (2010) found that the use of a hypermedia learning environment in high school caused more declarative and conceptual historical knowledge, as well as historical thinking skills. Learning with computers could thus very well enhance student deep strategy and deep learning outcome.
N. Vos et al. / Computers & Education 56 (2011) 127–137
129
Fig. 1. Constructing a memory drag and drop game on the website www.memoryspelen.nl.
1.2. Game construction Last few years some research on education by designing is conducted. Education by designing can be seen as another way to meet a constructivist learning environment. This form of education is also been described as ‘learning by doing’. During such learning tasks students must understand the things they learn and apply this knowledge executing the learning task. They have to develop skills to structure their thoughts and actions (Barron & Darling-Hammond, 2008). According to Robertson and Howells (2008) making games have the potential to be powerful learning environments. They mention that making a game is a rich task, in which students ‘exercise a wide spectrum of skills’ (p. 562). They also argument making a game is an authentic learning activity in which students are actively engaged. In their exploratory research the effects of role-playing game design, they collected qualitative data from the learning process and products of six 6th grade students in Scotland. Results of their study displayed motivation and enthusiasm for learning, determination to achieve and
130
N. Vos et al. / Computers & Education 56 (2011) 127–137
Fig. 2. Constructing a memory drag and drop game with visual representations of the proverbs on the website www.memoryspelen.nl.
links to learning transfer. Especially the link to learning transfer could be related to a successful constructivist learning environment. Robertson and Howells suggest making a game could indeed be a powerful learning environment in which students are in control of their own learning and thinking. According to Kafai (1995) game construction puts children in control of their own learning and thinking and provokes them to plan and manage the complex process of creating a game. It could be argued whether game construction would meet the constructivist learning environment even more than game playing. In summary, games (play and design) seem to comprise all elements for a learning environment in which students are stimulated to use deep learning strategies and show more intrinsic motivation. It could be questioned whether this is also be true for instructive tasks with a simple game. In the present study we investigated how different interactive tasks in which a game was included affect student intrinsic motivation and deep strategy use. We hypothesized that children in the game construction condition are more intrinsically motivated and use more deep strategies than the children in the play condition. Present research aims to contribute to the theoretical notions with regard to the effects of interactive computer-based tasks on learning. 2. Method 2.1. Research design In this study, we compared two instruction environments with a game with respect to their effect on student motivation and deep learning strategy use. For this comparison, we conducted a quasi-experimental quantitative study. Two groups were distinguished. Students in one group were given the task to construct a game on Dutch proverbs (construction group). The students in the other group had to play an existing game on Dutch proverbs (play group). Before and after completion of the tasks, student motivation and deep learning strategy use were measured.
N. Vos et al. / Computers & Education 56 (2011) 127–137
131
Fig. 3. Playing a memory drag and drop game on the website www.memoryspelen.nl.
2.2. Participants The study was conducted in four elementary schools in the Netherlands. The participants were 235 students from nine classes; 113 fifth grade students and 122 sixth grade students (aged 10–12 years old). Classes were randomly assigned to the two research conditions. Five classes (128 students) were assigned to the ‘construction’ condition, and four classes (107 students) were assigned to the ‘play’ condition. The students in the two conditions showed a higher level of Dutch language skills than the national average, given their fifth grade scores on a national standardized language test (Central Institute for Test Development (CITO)), t(232) ¼ 2.92, p ¼ .004. However, the both conditions didn’t differ significantly in their language skills, t(231) ¼ 1.39, p ¼ .165. 2.3. Tasks and materials Two lessons with a game application were developed and given by the first author. In one lesson students were asked to construct their own game, whereas in the other lesson students played an existing memory game. The goal of both lessons was to master a number of Dutch proverbs. The two lessons were designed using the software of De Digitale School [The digital School]. De Digitale School aims to offer digital learning materials. One of these materials is a learning environment where memory games can be constructed and played. The content of the games construction was not fixed, which means that there are possibilities for input of new contents within this game context (see http://www.memoryspelen.nl). Students in both conditions were given a worksheet. The first part of the worksheets was the same for both conditions. This part included an introduction to Dutch proverbs. The students were asked to seek eight Dutch proverbs and to identify their meaning. The students were free to choose which sources they wanted to use for their search. Available sources were for example Internet and books about Dutch proverbs. They were also free to choose which proverbs they wanted to select. All students succeeded in their search on proverbs and their meanings. After this search, the students were to continue the lesson. The introduction in Dutch proverbs took 20 min. It is from this point onwards that the two lessons differed.
132
N. Vos et al. / Computers & Education 56 (2011) 127–137
Table 1 Presentation of the intrinsic motivation scales and the corrected item total correlations, for pre-test and post-tests.
Competence
Interest
Effort
Pre-test
Corrected item–total correlation
Post-test
Corrected item–total correlation making (construct)
Corrected item–total correlation playing
I think I am good at school I think I do pretty well at school, compared to others I am satisfied with my performance at school I am pretty skilled at school I think I am pretty good at school Reliability (Cronbach’s a)
.73 .62
I think I was good in making/playing this gamea I think I did pretty well in making this game, compared to others I am satisfied with my performance while making the game I was pretty skilled at making this game I think I was pretty good in making this game Reliability (Cronbach’s a)
.68 .49
.70 .70
.50
.46
.70 .67 .81
.79 .73 .86
I think making this game was quite enjoyable I think making this game was interesting I think making this game was fun While I was making the game, I often thought about how much I enjoyed it I think making this game was boring Reliability (Cronbach’s a)
.67 .72 .83 .62
.82 .75 .85 .65
.76 .85
.73 .90
I did my best while I was making the game I tried very hard to do well in making this game It was important to me to do well in making this game I put much effort in making this game Reliability (Cronbach’s a)
.65 .62 .45
.63 .68 .45
.58 .76
.53 .77
.65 .83 .69 .87
I think school is quite enjoyable I think school is very interesting I think school is fun At school I often think about how much I enjoy it I think school is boring Reliability (Cronbach’s a)
.62 .65 .73 .56
I do my best at school I try very hard to do well at school It is important to me to do well at school I put much effort in school Reliability (Cronbach’s a)
.66 .57 .39
.53 .82
.47 .73
a The items presented in this column are the construction condition items. For the playing condition, the word ‘making’ was replaced by ‘playing’, as can be seen in the first item.
In the construction condition students were guided by their worksheet in constructing their own game. The goal was to master Dutch proverbs by constructing a game about proverbs. The format of the game was a ‘drag and drop game’. This is a game where the player has to drag one picture and to drop it next to the picture it relates to. In the case of Dutch proverbs, the player had to drag a picture with the textual representation of a proverb to a picture with the meaning of the proverb (also a textual picture). The students were asked to fill in their selected proverbs and meanings in the game application (see Fig. 1). During the game construction the teacher and two researchers gave feedback on students work, both on content and process. After 120 min, the lesson finished and the game was sent to the webmaster. Those who had finished their game before the end of the lesson received an additional worksheet with instructions for adding visual representations to their game (see Fig. 2). After completing the post-test questionnaires, students were told that their game could be placed on the website of De Digitale School, so the students could play the game their selves and show it to their parents and classmates. Within a few days the students were told by the webmaster of De Digitale School whether their game was approved.1 In the play condition students were guided by their worksheet in playing a game. The goal was to master Dutch proverbs by playing an existing game about Dutch proverbs, constructed by the first author, on a website (http://www.memoryspelen.nl). This game was also a ‘drag and drop game’. The students had to drag a picture with the visual and textual representation of a proverb to the picture with the meaning of the proverb (see Fig. 3). The goal of the game was to drag all eight proverbs to their meanings as quickly as possible, with minimal errors. The students could help each other, but could also compete against each other. During the game construction the teacher and two researchers gave feedback on students work, both on content and process. If a student wished to stop playing the game before the end of the lesson, the students received another worksheet with instructions to play another memory game (a regular memory click-game) about Dutch proverbs. The goal of the game was to match all eight proverbs with their meanings as quickly as possible, with minimal errors. The lesson in this condition took 90 min. 2.4. Instruments To gain insight into the effects of the two different lessons, student intrinsic motivation and deep strategy use were investigated. Learning effect of the two lessons wasn’t measured, because students in the construction condition were free to select their proverbs. This made it difficult to compare the learning outcomes of students from both groups. Besides, according to Biggs et al. (2001) deep strategy use is highly related to deep learning outcomes. By measuring deep strategy use we could thus make some cautious speculations about student learning outcomes. For pre-test purposes, a two parted questionnaire consisting of measures of student intrinsic motivation and deep strategy use was administered. For both conditions, a contextualized post-test questionnaire was constructed to measure the effect of constructing or playing a game on student motivation and to assess the deep strategy the students used during the tasks they completed in the lessons. The posttest questionnaires were conducted directly after constructing/playing the game. 2.4.1. Pre-test: intrinsic motivation inventory Student intrinsic motivation was measured by a selection of 14 items from the Intrinsic Motivation Inventory (IMI) (Ryan & Deci, 2000). This instrument was selected because of its broad coverage of the concept intrinsic motivation. It has earlier been used in several
1
Criteria by the Digital School for approving the games were: games should be finished, and games should not contain any abusive words.
N. Vos et al. / Computers & Education 56 (2011) 127–137
133
Table 2 Presentation of the deep strategy use scale and the corrected item total correlations, for pre-test and post-tests. Pre-test
Corrected item–total correlation
Post-test
Corrected item–total correlation making (construct)
Corrected item–total correlation playing
I find most new topic interesting and want to spend extra time trying to obtain more information about it I ask myself questions about topics discussed in lessons to check whether I understand the topics I spend a lot of my free time finding out more about interesting topics discussed at school I make a point looking at most of the suggested materials
.52
I found making/playing this game interesting and want to spend extra time trying to make/play more gamesa While making this game, I asked myself questions to check whether I understood the proverbs I want to spend free time making these kind of games
.64
.62
.46
.37
.70
.65
.36
.75
.75
I think school is instructive Reliability (Cronbach’s a)
.37 .67
.56 .89
.54 .86
.34 .54
After making this game, I would like to make more or these games I thought making this game was instructive Reliability (Cronbach’s a)
a The items presented in this column are the construction condition items. For the playing condition, the word ‘making’ was replaced by ‘playing’, as can be seen in the first item.
experiments related to intrinsic motivation (Ryan & Deci, 2000). The original questionnaire consists of seven subscales. For this research, three subscales were selected; ‘interest’, ‘perceived competence’, and ‘effort’. According to Ryan and Deci the interest scale refers to intrinsic motivation. Perceived competence is theorised to be a positive predictor of intrinsic motivation. Thereby, effort is a separate variable that is aimed to be relevant in motivation questions. Because of the expected relevance of the ‘perceived competence’ and ‘effort’ scale, these two scales were included in the questionnaire, next to the primarily intrinsic motivation scale ‘interest’. Students were asked to rate to what extent they agreed with statements like ‘I find school very interesting’ (interest), ‘I think I am good at school’ (competence), and ‘It is important for me to do well at school’ (effort). The answers were given on a 5-points Likert scale ranging from ‘strongly disagree’ (1) to ‘strongly agree’ (5). Homogeneity of the subscales was assessed by inspecting the corrected item total correlation. As can be seen in Table 1, the corrected item total correlations for each pre-test scale were satisfactory, as also the internal consistency (Cronbach’s alpha) of all intrinsic motivation scales. Three intrinsic motivation pre-test scales were computed by calculating mean scores of the items for each scale. 2.4.2. Pre-test: deep learning strategy inventory To measure students’ deep learning strategy use, items referring to this concept were selected from the Revised two-factor Study Process Questionnaire (R-SPQ-2F) (Biggs et al., 2001). The items were translated and reformulated carefully to make them suitable for the target group. The items consisted of five items referring to deep learning strategy (e.g., ‘I test myself on important topics until I understand them completely’). Students were to rate the items on a 5-point Likert scale ranging from ‘strongly disagree’ (1) to ‘strongly agree’ (5). In Table 2 the corrected item total correlations of the pre-test are presented. These correlations suggest that every item refers to the same dimension. The internal consistency (Cronbach’s alpha) of the five deep learning strategy use items was .67. One deep strategy use scale was computed by calculating mean scores of the five items. 2.4.3. Correlations pre-test scales Correlations between the four distinguished pre-test scales were calculated to investigate whether the scales related to each other. Pearson’s correlations revealed all scales correlated significantly at .01 level (Table 3). This suggests that intrinsic motivation in form of competence, effort and interest relates to deep strategy use. Students who are intrinsically motivated also show a high degree of deep strategy use. Besides, these results also indicate that the three intrinsic motivation scales are related. 2.4.4. Post-test: intrinsic motivation inventory The post-test measure of motivation consisted of similar items as the pre-test questionnaire. These items were reformulated to assess to what extent the students were intrinsically motivated while constructing or playing the game. For instance, students were asked to rate statements like ‘I found it very interesting to make/play this game’ (interest), ‘I think I was good in making/playing the game’ (competence), and ‘It was important for me to do well at making/playing the game’ (effort). The corrected item total correlations of the three intrinsic motivation scales of both post-test measures are presented in Table 1. Cronbach’s alphas for both construction condition and play condition were satisfactory. 2.4.5. Post-test: learning strategy use inventory To the strategy inventory, similar adjustments were made as to the motivation inventory. Items were reformulated to measure the deep learning strategies that students had used specifically during the construction or playing of the game. For example, students were asked to rate items like ‘When constructing/playing the game, I learned the proverbs by repeating them until I understood them completely’. The Table 3 Pearson correlations between deep strategy use, competence, effort, and interest.
Deep strategy Competence Effort Interest a
Significant at .01 level.
Deep strategy
Competence
Effort
Interest
– .26a .33a .51a
– .55a .39a
– .40a
–
134
N. Vos et al. / Computers & Education 56 (2011) 127–137
Table 4 Descriptive statistics pre-test and post-test scores on intrinsic motivation (competence, interest, and effort) and deep learning strategy. n
Intrinsic motivation Competence Play condition Construction condition Interest Play condition Construction condition Effort Play condition Construction condition
Deep strategy use Play condition Construction condition
Pre-test (at school)
Post-test (During game lesson)
Mean
SD
Mean
SE
Boys Girls Boys Girls
45 60 76 52
3.69 3.53 3.55 3.35
.78 .60 .64. .65
3.33 3.06 3.61 3.91
.10 .09 .09 1.12
Boys Girls Boys Girls
45 60 76 52
3.17 3.18 3.15 3.18
.81 .52 .62 .67
2.74 2.86 3.94 4.16
.14 .09 .11 .10
Boys Girls Boys Girls
45 60 76 52
3.94 3.96 4.00 3.84
.65 .62 .58 .54
3.06 3.13 3.91 3.93
.12 .09 .09 .09
Boys Girls Boys Girls
46 61 74 51
3.02 3.00 3.06 3.06
.60 .63 .66 .61
2.44 2.49 3.21 3.43
.10 .10 .09 .10
Notes. Pre-test scores are observed means, post-test scores are adjusted means; Ranges of the scales were 1–5.
corrected item total correlations of both post-test measures are presented in Table 2. Cronbach’s alpha was .89 for the construction condition and .86 for the play condition. 2.4.6. Correlations between pre-test and post-test scales As can be read, the pre-test questionnaires aimed to measure general intrinsic motivation and deep strategy use, where the post-test questionnaires aimed to measure lesson specific intrinsic motivation and strategy use. To investigate the measurement stability of the questionnaires used, Pearson correlations were calculated. Each post-test motivation scale had a significant correlation with its pre-test scale (interest r ¼ .18, p < .001; effort r ¼ .18, p < .001; competence r ¼ .15, p < .05.). Further, the deep strategy post-test scale correlated with the deep strategy pre-test scale (r ¼ .25, p < .001). The significant correlations suggest the post-test scales to give some insight into student motivation and strategy use. However, it should be noted that the post-test questionnaires referred to only one lesson, whereas the pre-test questionnaires referred to aspects of motivation and strategy use throughout all school experiences. Therefore, it can be expected that posttest measures of motivation and strategy use diverged from general pre-test measures. Nevertheless, it can be concluded that the post-test scales give a consistent view of the student motivation and strategy use in the game conditions. 2.5. Data analysis To examine the differences between the post-test scores on intrinsic motivation and learning strategy for the two conditions, analyses of covariance (ANCOVA) were performed with the pre-test scores as covariates. Effect sizes were computed by using Cohen’s d. These effect sizes were computed by dividing the post-test mean differences between the two groups by the pooled standard deviation for all cells in the between-subject design. Effect sizes of .20 reflected a small or minimal effect, .50 as a medium or moderate effect, and .80 or higher as a large or meaningful effect (Olejnik & Algina, 2000). The set level of significance was .05. 3. Results Adjusted means of the intrinsic motivation variables and learning strategy use for both research groups are presented in Table 4. As can be seen at the pre-test competence scores in Table 3, students rated their selves as pretty competent at school, compared to the scale range
Table 5 Results of analyses of covariance on intrinsic motivation (competence, interest, and effort). Competence during game lesson
Pre-test Condition Gender Error Total Corrected total
Interest during game lesson
Effort during game lesson
SS
df
F
p
SS
df
F
p
SS
df
F
p
3.79 4.61 .01 124.41 2791.06 132.19
1 1 1 229 233 232
6.98 8.49 .02
.009 .004 .898
9.11 87.16 1.60 166.55 3074.80 262.00
3 1 1 228 232 231
12.47 119.33 2.19
.001 .000 .141
5.66 37.99 .25 117.29 3096.40 160.49
1 1 1 229 233 232
11.05 74.18 .48
.001 .000 .49
Notes. Tests for parallel slopes of pre-test on post-test scores within groups were non significant.
N. Vos et al. / Computers & Education 56 (2011) 127–137
135
Table 6 Results of analysis of covariance (ANCOVA) on deep strategy use. Deep strategy use during game lesson
Pre-test Condition Gender Error Total Corrected total
SS
df
F
p
9.97 40.48 1.16 125.27 2149.76 177.12
3 1 1 228 232 231
18.15 73.69 2.12
.000 .000 .147
Notes. Tests for parallel slopes of pre-test on post-test scores within groups were non significant.
1–5. There were no differences between boys and girls in their competence at school, F(1,233) ¼ 3.42, p ¼ .066. Also, no initial differences in competence at school between the play and construction condition were observed, F(1,233) ¼ 2.20, p ¼ .139. It can also be seen that students had a moderate degree of interest at school. There were no differences between boys and girls in their interest at school, F(1,232) ¼ .10, p ¼ .755. Also, no initial differences in interest at school between the play and construction condition were observed, F(1,232) ¼ .02, p ¼ .879. At the pre-test, students rated their level of effort at school as pretty high, compared to the scale range 1–5. Again, no differences between boys and girls (F(1,233) ¼ .90, p ¼ .343) and no differences between the play and construction condition (F(1,233) ¼ .07, p ¼ .786) were found. About deep strategy use at school can be said that students had a moderate level of deep strategy use, compared to the scale range 1–5. No significant differences were observed between boys and girls, F(1,233) ¼ .01, p ¼ .93. Likewise, there were no significant differences between students in the different conditions, F(1,233) ¼ .32, p ¼ .575. The post-test descriptives suggest student motivation (in terms of competence, interest and effort) to be higher in the construction condition compared to the pre-test scores which referred to a wide range of prior school experiences. In contrast, student seemed to be less motivated in the play condition than in during their regular school lessons. This was also the case for deep strategy use: students showed less deep strategy use in the play condition than during their regular school lessons. Three analyses of covariance were conducted to verify whether students in both conditions varied in their intrinsic motivation during the game lesson. The results presented in Table 5 reveal a significant difference between the perceived competence of the students in both conditions, F(1,229) ¼ 8.49, p ¼ .004. The minimum effect size (d ¼ .35) was in favour of the group who constructed a game. This indicates students who constructed the game felt more competent than the students who played the game. There were no significant differences in perceived competence between boys and girls, F(1,229) ¼ .02, p ¼ .898. Similar results were found when comparing student interest in the both lessons. Students in the construction condition were significantly more interested than students in the play condition, F (1,228) ¼ 119.33, p < .001. This was a large effect size (d ¼ 1.09). Again no differences between boys and girls were found, F(1,229) ¼ 2.19, p ¼ .141. This was also the case with student effort during the game lessons. Students in the construction condition rated their effort significantly higher than the play condition, F(1,229) ¼ 74.18 p < .001. This was a large effect size (d ¼ 1.36). There were no differences between boys and girls, F(1,229) ¼ .49, p ¼ .485. Analysis of covariance was also used to investigate whether students differed in their deep strategy use during the game lessons (see Table 6). These analyses revealed a significant difference between the students in the construction and playing condition, F (1, 228) ¼ 73.69, p < .001. The large effect size (d ¼ 1.07) was in favour of the game constructors. Differences in deep strategy use between boys and girls were not significant, F(1,228) ¼ 2.12, p ¼ .147. 4. Conclusion and discussion In the present study we addressed the following question. To what extent do different interactive tasks in the form of two game conditions affect student motivation and deep strategy use? Students in the construction condition constructed their own simple ‘drag and drop’ game on Dutch proverbs, whereas students in the play conditions played an existing simple memory game on Dutch proverbs. The conditions were compared with regard to student intrinsic motivation and the use of deep learning strategies. The results showed a significant difference between the two conditions both on intrinsic motivation and deep strategy use. The large effect sizes for both motivation and deep strategy use were in favour of the construction condition. These findings are in line with the studies of Kafai (1996) and Kiili (2005). Based upon their research we expected the game construction condition to enhance intrinsic motivation and stimulate the use of deep learning strategies more than the game playing condition. Constructing a game seems to be more motivating and stimulates a deep learning approach more than playing a game. According to Gee (2003) this might be due to a greater recruitment of critical thinking skills and self-regulated learning (Gee, 2003). The findings of this study suggest that construction of a game meets the constructivist learning environment more than playing a game. These results might be explained by the nature of the tasks. The construction task might have invoked students’ activity more, and might have been more complex than playing the game. Also the task might have been more authentic or more meaningful than playing the game. In addition, the construction lesson appeals to one major feature of constructivism, namely that students should engage in active learning processes. In general, constructing a game demands more student activity than playing a game, which is to some extent a more passive learning activity. Moreover, the students in the play condition seemed to be less motivated and made less use of a deep strategy than during their regular school lessons. These findings might suggest students were less motivated during the game play than during their regular school lessons. However, it can be questioned whether student lesson specific motivation and deep strategy use can be compared to student general school motivation and strategy use. Therefore we chose to control for general school motivation and strategy use, instead of using these variables for repeated measures. It can thus be concluded that games do not always provide higher intrinsic motivation and deep strategy use, however there seem to be some characteristics of the game construction activity that enhance student motivation and deep strategy use.
136
N. Vos et al. / Computers & Education 56 (2011) 127–137
Garris, Ahlers, and Driskell (2002) executed a literature review and found a number of characteristics of a game to be attractive to children. These characteristics were: fantasy, goals and rules, sensory stimulation, challenges, mystery, and control. Probably the game play on Dutch proverbs did not meet this conditions (e.g., too little fantasy and too few challenges). Another explanation can be found in the level of complexity of the game. Children might have found the game to easy to play. When a game is not in line with the prior knowledge of the student, the chances of flow-experiences are small, which has it implications on the intrinsic motivation and the flow of the students (Csikszentmihalyi, 1991; Ryan & Deci, 2000). The present research offers insight into the use of games in education. Nevertheless, some possible limitations should be mentioned. We did not check whether the game we constructed for the play condition met de conditions of Garris et al. (2002). Furthermore, when observing the students in the play condition, we noted the students got bored when playing the game several times over. Most students really liked the game when they played it one, two, or three times. After the third time of playing, students’ attention seemed to decrease. The game play might have been too easy for most of the students. To enhance the complexity, it might have been better to construct different versions of the game so that the students could play the game on different complexity levels. Furthermore, in the present study, no attention has been paid to learning outcomes. In what condition would children have learned most? And would they indeed have learned more when they use more deep learning strategies, as assumed by Biggs (1993)? To find answers to these questions, further research is needed. Another limitation that should be noted is the length of both conditions. The construction condition lasted 120 min, while the play condition lasted no longer than 90 min. The effects on intrinsic motivation and learning strategy use in favour of the construction condition might have been caused by the extra length of that lesson. In this case, extra time could enhance intrinsic motivation and deep strategy use. On the other hand, when we consider the observational notes of the play condition, a longer lesson may have caused an even larger effect size in favour of the construction condition. A longer game play could have been even less motivating because the game would not have challenged the students more than it did during the 90 min. A technical limitation of the study was the lack of process evaluation. No structured qualitative data (i.e. observations and interviews) were collected to gain insight student learning processes. This could have given more insight into the research findings in a deeper way. Therefore, qualitative data collection could be taken into account in future studies. As mentioned before, the present study sheds some light on the potential that interactive tasks with games included have when applied in an educational setting. The results are clearly in favour of the construction condition. The construction of a game not only has positive effects on intrinsic motivation of students constructing a game, it also stimulates the use of deep learning strategies. Present research findings thus encourage the use of game construction in Dutch education.
References Abbott, I., Townsend, A., Johnston-Wilder, S., & Reynolds, L. (2009). Literature review: deep learning with technology in 14- to 19-year-old learners for Becta. http://research. becta.org.uk/index.php?section¼rh&catcode¼_re_rp_02&rid¼17171 Retrieved August 10th 2010 from. Barron, B., & Darling-Hammond, L. (2008). Teaching for meaningful learning. In L. Darling-Hammond, B. Barron, P. D. Pearson, A. H. Schoenfeld, E. K. Stage, T. D. Zimmerman, G. N. Cervetti, & J. Tilson (Eds.), Powerful learning: What we know about teaching for understanding. San Francisco: Jossey-Bass. Beeksma, J., & Hulst, A.van der (2005). Games – meer dan spelen. ICT Verkenningen voor het onderwijs. Zoetermeer: Kennisnet. http://www.kamervanmorgen.nl/ kamerzittingen/games/publicatie Retrieved December 6th 2009 from. Bransford, J.D., Brown, A.L., & Cocking, R.R. (2000). How People Learn: Brain, Mind, Experience, and School. Retrieved August 11th 2010 from http://siona.udea.edu.co/wjfduitam/ curriculo/doc/How%20people%20learn.pdf. Biggs, J. (1993). What do inventories of student learning processes really measure? A theoretical review and clarification. British Journal of Educational Psychology, 63, 3–19. Biggs, J., Kember, D., & Leung, Y. (2001). The revised two-factor study process questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71, 133–149. Burguillo, J. C. (2010). Using game theory and competition-based learning to stimulate student motivation and performance. Computers & Education, 55, 266–575. Cordova, D., & Lepper, M. (1996). Intrinsic motivation and the process of learning: beneficial effects of contextualizations, personalizations, and choice. Journal of Educational Psychology, 88(4), 715–730. Crisp, V., & Ward, C. (2008). The development of a formative scenario-based computer assisted assessment tool in psychology for teachers: the PePCAA project. Computers & Education, 50, 1209–1526. Csikszentmihalyi, M. (1991). Flow: The psychology of optimal experience. New York: Harper Perennial. Driscoll, M. P. (1994). Psychology of learning for instruction. Needham Heights, MA: Allyn & Bacon. Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: a research and practice model. Simulation & Gaming, 3(4), 441–467. Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York: Palgrave Macmillan. Greene, J. A., & Azevedo, R. (2009). A macro-level analysis of SRL processess and their relations of the acquisition of sophisticated mental models. Contemporary Educational Psychology, 34, 18–29. Greene, J. A., Bolick, C. M., & Robertson, J. (2010). Fostering historical knowledge and thinking skills using hypermedia learning environments: the role of self-regulated learning. Computers & Education, 54, 230–243. Hamilton, J., & Tee, S. (2010). Smart utilization of tertiary instructional modes. Computers & Education, 54(4), 1036–1053. Kafai, Y. B. (1995). Minds in play: Computer game design as a context for children’s learning. Hillsdale, NJ: Lawrence Erlbauw Associates. Kafai, Y. B. (1996). Learning design by making games: children’s development of design strategies in the creation of a complex computational artifact. In Y. Kafai, & M. Resnick (Eds.), Constructionism in practice: Designing, thinking and learning in a digital world (pp. 71–96). Mahwah, NJ: Erlbaum. Ke, F., & Grabowski, B. (2007). Game playing for mathematics learning: cooperative or not? British Journal of Educational Technology, 38(2), 249–259. Kebritchi, M., Hirumi, A., & Bai, H. (2010). The effects of modern mathematics computer games on mathematics achievement and class motivation. Computers & Education, 55, 427–443. Kiili, K. (2005). Content creation challenges and flow experience in educational games: the IT-Emperor. Internet and Higher Education, 8, 183–198. Kim, B., Park, H., & Baek, Y. (2009). Not just fun, but serious strategies: using meta-cognitive strategies in game-based learning. Computers and Education, 52(4), 800–810. Kirriemuir, J., & McFarlane, C. A. (2004). Literature review in games and learning. Bristol: Futurelab. www.futurelab.org.uk/resources/documents/lit_reviews/Games_Review.pdf Retrieved December 7th 2009 from. Kwon, S. Y., & Cifuentes, L. (2009). The comparative effect of individual-constructed vs. collaboratively-constructed computer-based concept maps. Computers & Education, 52, 365–375. Laird, T., Shoup, R., Kuh, G., & Schwarz, M. (2008). The effects of discipline on deep approaches to student learning and college outcomes. Research in Higher Education, 49, 469–494. Lopez-Morteo, G., & Lopez, G. (2007). Computer support for learning mathematics: a learning environment based on recreational learning objects. Computers & Education, 48 (4), 618–641. Marton, F., & Saljö, R. (1976). On qualitative differences in learning I: outcome and process. British Journal of Educational Psychology, 46, 4–11. McFarlane, A., Sparrowhawk, A., & Heald, Y. (2002). Report on the educational use of games. TEEM. http://www.teem.org.uk/publications/teem_gamesined_full.pdf Retrieved December 7th 2009 from.
N. Vos et al. / Computers & Education 56 (2011) 127–137
137
Olejnik, S., & Algina, J. (2000). Measures of effect size for comparative studies: applications, interpretations, and limitations. Contemporary Educational Psychology, 25, 241– 286. Owston, R., Wideman, H., Ronda, N. S., & Brown, C. (2009). Computer game development as a literacy activity. Computers & Education, 53(3), 977–989. Oyen, A., & Bebko, J. (1996). The effects of computer games and lesson context on childrens’s mnemonic strategies. Journal of Experimental Child Psychology, 62, 173–189. Perkins, D. (1999). The many faces of constructivism. Educational Leadership, 5(3), 6–11. Ramsden, P., Beswick, D., & Bowden, J. (1989). Effects of learning skills intervention on first year students’ learning. Human Learning, 5, 151–164. Robertson, J., & Howells, C. (2008). Computer game design: opportunities for successful learning. Computers & Education, 50, 559–578. Rosas, R., Nussbaum, M., Cumsille, P., Marianov, V., Correa, M., Flores, P., et al. (2003). Beyond Nintendo: design and assessment of educational video games for first and second grade students. Computers & Education, 40(1), 71–94. Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well being. American Psychologist, 55, 68–78. Shaffer, D., Squire, K., Halverson, R., & Gee, J. (2004). Video games and the future of learning. http://www.academiccolab.org/resources/gappspaper1.pdf Retrieved January 10th 2010 from. Shuell, T. J. (2001). Learning theories and educational paradigms. In P. B. Baltes (Ed.), International encyclopedia of the social and behavioral sciences (pp. 8613–8620). Oxford: Elsevier. Tam, M. (2000). Constructivism, instructional design, and technology: implications for transforming distance learning. Educational Technology & Society, 3(2). Tüzun, H., Yilmaz-Sollu, M., Karakus, T., Inal, Y., & Kizilkaya, G. (2008). The effects of computer games on primary school student’s achievement and motivation in geography learning. Computers & Education, 52(1), 68–78. Wideman, H. H., Owston, R. D., Brown, C., Kushniruk, A., Ho, F., Pitts, K. C., et al. (2007). Unpacking the potential of educational gaming: a new tool for gaming. Simulation and Gaming, 38(1), 10–30.
