people about hygiene, the spread of infections, antibiotic use and resistance. e-Bug .... methods included before and after student knowledge questionnaires; ...
1Introduction 2Education of children and young people is important in the fight against antibiotic resistance. 3Through education we can raise awareness, increase knowledge and modify behavioural intentions 4around hygiene and antibiotic use in our future generation of antibiotic users. e-Bug, led by Public 5Health England, is an international health education resource that teaches children and young 6people about hygiene, the spread of infections, antibiotic use and resistance. e-Bug includes online 7lesson plans and activities for educators, and educational games for students hosted on the e-Bug 8website. Evaluation of e-Bug activities to use in schools and science shows has been well9documented[1-6]; and the National Institute of Clinical Excellence[7] has suggested that schools may 10use the evidence based e-Bug resources to educate children and young people in an age appropriate 11way about hygiene, prevention of infections, and antibiotic use. 12 13Google Analytics have been used to monitor Web traffic to the e-Bug website since 2010[2]. The e14Bug website had 94,675 visitors from 1 st September 2016 to 31st August 2017, and 100,955 visitors 15the previous academic year. The e-Bug games homepage was the second highest visited page with 1628,610 page views from 1st September 2016 to 31st August 2017; senior games homepage had 10,154 17views during the same time period. 18 19The Internet is a suitable tool for health promotion and Internet health interventions have been 20shown to change health behaviours. “Gamification”, where features of gaming are used in other 21disciplines, has become increasingly popular in recent years aiming to make science and health 22education more available and exciting to the general public. ‘Serious games’ are those games where 23the primary focus is not entertainment but education and learning[8]. A meta-analysis of serious 24games in regards to their effect on cognitive processes and motivation found the games to positively 25effect cognitive processes, including learning and retention compared to traditional educational 26methods, with no difference to motivation[9]. 27 28Evidence suggests gamification and serious games for health and wellbeing is most effective when 29targeting health related behaviours[10]. For instance, positive associations between gamification, 30serious games and school aged knowledge and behaviour have been reported in public health topics 31such as asthma[11], fruit and vegetable consumption[12] and oral hygiene[13-14]. 32 33The e-Bug online educational games (www.e-Bug.eu) have been previously evaluated including a 34study that evaluated three e-Bug games using a mixed methods approach; the three e-Bug games 35showed an improvement in knowledge and focused around the use of antibiotics for bacterial versus 36viral infections, and ensuring the course of antibiotics is completed[15]. 37 Page 1 of 24
38The aim of this present study was to evaluate two of the e-Bug educational games: Body Busters, 39previously evaluated by Hale et al. and then modified with new content[15], and Stop the Spread, a 40new educational game launched in 2016. The study research questions are; what is student baseline 41knowledge around the game learning outcomes, what is student change in knowledge following the 42games, and what are student views on the games to suggest improvements? Figure 1 details the 43style of play and learning outcomes of the two games Both e-Bug games are responsive on all devices 44including computers and tablets. Pilot game testing at three schools was conducted prior to the 45game launch in order to ensure the games worked correctly and the instructions were clear. 46 47Figure 1: The style of play and learning outcomes of Body Busters and Stop the Spread 48 Game Body Busters
Style of game Pac-man style game in which users need to collect all the antibiotics to kill the bacteria. Useful bacteria give the users extra lives, but these bacteria are also killed by the antibiotics. Harder levels now also include viruses, which are not killed by the antibiotics.
Stop the Spread
Users have to prevent the spread of infection through a school, observed through a birds-eye view of the playground. Users have to catch a sneeze (red circles) with a tissue and throw the used tissue into a bin. If the users fail to catch the sneeze, more children in the playground become infected (green circles). The game is over once every child in the playground is infected. The longer the game lasts, the more points the user scores.
Learning outcomes There are different types of microbes (bacteria and viruses) Bacteria can be harmful or useful Antibiotics kill useful and harmful bacteria but do not kill viruses
Infections spread by sneezing Catch sneezes in a tissue Put used tissues in the bin Using tissues stops the spread of infections Vaccinations can help prevent the spread of infection
An additional level includes the theme of vaccination and herd immunity. In this level users vaccinate children in the playground as quickly as they can to slow the spread of infection. 49 Page 2 of 24
50Methods 51Research design 52The study was a mixed method evaluation using quantitative and qualitative methods. Quantitative 53methods included before and after student knowledge questionnaires; qualitative methods included 54student focus groups and open ended questions and responses from the student post-gaming 55evaluation questionnaire. 56 57Sampling and recruitment 58Educational providers, including schools and summer schools, were invited to take part in the study 59through convenience recruitment of educators at educational and science conferences, and then 60through snowball sampling. Sampling aimed to ensure a representation of school aged children 61across three local authorities in the UK including; rural and urban schools, different socioeconomics, 62and selective grammar and non selective state schools (see table 1). Local authorities were: 63Gloucestershire, Buckinghamshire and South Wales. 64 65Ethics 66All researchers who observed the sessions had a Disclosure Barring Check (DBS), through Public 67Health England (PHE), to work with children. This study did not require National Research Ethics 68Service (NRES) approval as it was outside the National Health Service and was classed as a service 69evaluation. Public Health England provided written confirmation approving the service evaluation in 70July 2016. Educational providers gave informed written consent prior to the study taking place; 71students involved and their parents were given the option for students to opt out at any point during 72data collection. Teachers reported that no students opted out of the research. Consent was deemed 73accepted if participants completed the before and after knowledge questionnaires. Questionnaires 74were collected in line with the Data Protection Act 1998 and Caldicott 1999 regulations on handling 75and distributing sensitive participant information. Focus group participants provided verbal informed 76consent for participation in the research, audio recording and the publishing of anonymised quotes. 77
Page 3 of 24
78Data collection and analysis 79Data collection took place between August 2016 and July 2017. 80 81Quantitative data 82Before and after knowledge questionnaires were used to evaluate whether playing the e-Bug games 83had made any change to student knowledge. Questionnaires were based on previously validated 84questionnaires used to evaluate the e-Bug games and activities[2,3,15]; four additional questions (1, 852, 3, 5) were included in the questionnaire to cover additional learning outcomes that could be 86indirectly improved through game play and are hereby referred to as ‘general questions’. See 87appendix A for questionnaires. 88 89Data collection consisted of: (1) Students completed pre game play questionnaire 1 alone without 90consultation, (2) Students played on Body Busters for 5 minutes, (3) Students played on Stop the 91Spread for 5 minutes, (4) Students completed post game play questionnaire 2 alone without 92consultation which also included some additional open ended evaluation questions. The game play 93was for 5 minutes to follow the methodology of previous evaluations[15] and as 5 minutes is the 94estimated amount of time it takes to play one round of the games from testing; it allowed the study 95to measure knowledge change after single game play rather than repeated game play. 96 97A researcher was present in each session to monitor and observe game play, and hand out and 98collect questionnaires from students. Data collection occurred in a convenient room where students 99had their own computer. The rationale for that was to model how the games might be played in a 100real life teaching situation. Figure 2 provides further details on the data collection process. 101 102McNemar’s test was used for each response from the multiple choice question to determine the 103significance in the difference in the proportion of correct answers before and after game playing. 10495% confidence intervals of the odds ratio were estimated to determine the odds of students 105answering correctly. Analysis was performed separately for junior and senior school-aged pupils as 106knowledge change could differ between age groups. All statistical analysis was completed in STATA, 107version 14.2 108 109The post game play questionnaire 2 included an additional seven questions on game enjoyment 110including; two Likert Scale questions (students circled a number scale of 1-10) and five open ended 111questions. Likert Scale responses were inputted into MS Excel and mean enjoyment scores for each 112game were calculated for junior, senior and all students. 113 114Qualitative data 115Five open ended questions on enjoyment were included in the post game play questionnaire 2 to 116provide deeper qualitative data from all students. 117 Page 4 of 24
118Semi structured focus groups were facilitated immediately after the game intervention by VY, CH and 119CE who are all trained qualitative researchers for the e-Bug project, Public Health England. All 120twenty-six focus groups took place in person in a convenient room at the educational establishment. 121Focus groups of size 4-6 students, chosen by the class teacher and represented a mix of student 122abilities, lasted 6 to 20 minutes dependent on participant age. Focus groups were audio recorded, 123transcribed verbatim, and checked for accuracy by CH or CE. Twenty six focus groups were planned 124with all five schools participating; no new themes emerged from the later focus groups and 125researchers agreed that data saturation had been reached. 126 127The topic guide for the focus groups was based on previous e-Bug evaluation topic guides for e-Bug, 128and included additional questions on Stop the Spread learning outcomes. The schedule was piloted 129during the e-Bug game development in three testing sessions in schools in March 2016. 130 131All focus group data and open ended responses on enjoyment were inputted into NVivo 10 132qualitative analysis software. NVivo 10 was used to organise, code and analyse the focus group 133transcripts by CH and open ended evaluation responses by CE. A subset of focus group data (two 134junior and two senior transcripts) was analysed by a second researcher (CE) to ensure reliability. Both 135researchers discussed the data and coding to agree on the emerging themes prior to developing a 136thematic framework. Any discrepancies between researchers were resolved through discussion until 137an agreement was reached. The thematic framework was discussed by the research team. 138 139
Page 5 of 24
140Figure 2: e-Bug game play and data collection process 141 142Stage 1: Before knowledge questionnaire Students enter room and researcher is introduced
Researcher explains the outline of the session
Students complete pre game play questionnaire 1 independently
Stage 2: Body Busters Researcher demonstrates how to access Body Busters on the website
Students play Body Busters for 5 minutes
Stage 3: Stop the Spread Researcher demonstrates how to access Stop the Spread on the website
Students play Stop the Spread for 5 minutes
Stage 4: After knowledge questionnaire Students complete post game play questionnaire 2 independently; including evaluation questions
Stage 5: Focus Groups
Students enter the room
Researcher explains the outline of the qualitative aspect of the study
Focus groups are conducted and recorded
Page 6 of 24
143Results 144Main findings 145The study recruited 473 students aged 7-16 year olds from five educational providers across three 146local authorities in the United Kingdom; 123 junior and 350 senior students (Table 1). 147 148Table 1: Demographics of educational providers Educational
Local authority
Type of education
n = questionnaires
provider
n = focus groups (students)
A
Gloucestershire
Summer school
61
14 (61)
B
Gloucestershire
Grammar
29
1 (6)
C
South Wales
State
100
4 (24)
D
Bedfordshire
State
183
4 (20)
E
Bedfordshire
Grammar
100
3 (15)
473
26 (126)
Total 149
150Before and after knowledge questionnaires were completed by 473 students. Baseline knowledge in 151junior students around vaccinations was high (>60% correct responses) and senior students was 152higher (>80% correct responses) except one question which was 55%. Baseline knowledge around 153antibiotics was low in junior and senior students (
