members of the public [3] and tablet technologies have become a ubiquitous form of ... To develop a compendium for prescribing iPad apps and linked training ...
Developing a Framework for App Evaluation: Empowering Learning and Communication with iPads for Children and Young People with Communication Impairments. Claire Hamshire and Julie Lachkovic Manchester Metropolitan University, Manchester, United Kingdom Abstract: This paper presents an overview of a two-year collaborative partnership project between Manchester Metropolitan University and three specialist schools in Manchester, UK. The purpose of the research was to work with a varied group of children and young people experiencing communication impairments and learning disabilities to gain an insight into their needs and subsequently to develop iPad based resources and an App evaluation framework for staff and carers supporting them. The underpinning rationale of the project was to integrate iPads into the learning environment to provide opportunities for children and young people to further develop their skills and communication through the use of new technologies; where this was likely to enhance current practices. This paper outlines the successes and challenges raised by the project, as well as future developments and potential wider implications for iPad usage for this particular group. Keywords: iPad·app·evaluation framework·communication·learning
1
Introduction and Background
New and emerging technologies play an important role in enabling heath care providers and educationalists to respond to a range of challenges in the future [1,2] and technology and innovation will increasingly influence the way that patients and staff perceive, understand and manage health and ill-health [3]. Staff from across health and social care professional groups increasingly use information and information communication technologies (ICT) for everything they do with patients, carers and members of the public [3] and tablet technologies have become a ubiquitous form of communication technology that have begun to permeate the way individuals learn, work and live [4]. People are adopting these new technologies at a rapid rate and tablet devices with a range of apps will become ever more of a key feature in improving the quality and cost of educating not just the formal workforce, but also students, patients and their carers [3]. eHealth will provide the backbone for the future citizen-centred healthcare environment [1] and as such, tablet devices and associate apps have the potential to transform future practice [3]. When iPads were first launched in 2010, it was initially unclear as to the value of the device for promoting learning and teaching [5]. However, research suggested that tablet technology had a wide variety of applications that have the potential to enrich the learning experience and communication [6]. Tablet devices with associated apps are therefore being increasingly used within health and social care [3], and the National Health Service in the UK endorses a number of apps [7]. Technological interventions have often been used to assist in the learning of children and young people with communication impairments; with some evidence of success in the use of iPads to teach literacy [8,9,10]. In addition, health and teaching professionals consider iPads to be an important mode of technology that facilitates learning, communication and independence [11]. However, the speed of the implementation and uptake of these technologies is a workforce risk, as staff attempt to keep up-to-date with new technologies. There are currently no standardised guidelines relating to the use of tablets and apps in the UK, leading to variance and variety across practice and unresolved assessment and treatment issues; with the potential for patient complaints about disparities across services. Therefore, there was a need to develop a common framework and guidelines for practice, applicable to Health and Education staff who utilise tablet devices and apps to support patients and carers. Speech and Language Therapists were a key group of focus within this project as their role connects both health and education sectors.
2
Methods
This two-year longitudinal study was a collaborative partnership project between Manchester Metropolitan University (MMU) and three specialist schools in Manchester, United Kingdom: a nursery for children aged 3-5 years old; a special needs school for children aged 3 -18 years old and a college for young people with additional needs aged 16-25. Together these schools provided education to children and young people with an age range of 3-25 years and a wide-range of additional learning and communication needs and diagnoses including Autism, Downs Syndrome and acquired head injury. One of the challenges of using iPads with children and young people with communication and learning difficulties to support their learning and communication is searching through the large number of apps that are currently available and evaluating their usefulness in relation to a specific learning or communication need. Further, searches through the app store require exact names of specific apps for results to be generated and then choices between purchase or limited lite and free versions to be made. Therefore, the purpose of this research was to gain an in-depth understanding of the different elements of learning for these children and young people and subsequently develop an App evaluation framework to support future iPad usage and learning experiences in a way that promoted both social and personal as well as academic development. The project team worked with educators, speech and language therapists, children and young people with communication impairments and their families to gain an insight into their needs and subsequently develop a portfolio of resources to support their learning and development.
3
Data collection
Central to the whole project was the perspective of the children and young people, their parents and the school staff and we developed a dynamic, two-way process in a review of practice and evaluation of the technology. As such, the project drew on a phased, practice-driven and solution-focused action research approach [12] to make lasting change and provide a basis for good practice; developing a range of strategies and new innovations that addressed students’ needs in more efficient ways. The aims of the project were: 1. 2. 3. 4.
To identify the specific issues and challenges that impacted on the use of iPads with children and young people with communication and learning difficulties. To design practical suitable solutions that enabled staff and carers to assess the suitability of iPad and app usage for use across a wide spectrum of age, ability and need. To develop training materials and linked training packages for teachers, carers and therapists. To develop a compendium for prescribing iPad apps and linked training packages to provide a framework for the support of children and young people experiencing communication impairments and learning disabilities.
We needed to explore the perspectives of staff, carers and children and young people. In order to capture this data a multi-phase evaluation was undertaken with each distinct phase of the project conducted through a cyclical process of thinking, acting, data gathering and reflection [13]. In the first phase, we gathered information from staff and tutors at the schools by inviting them to participate in focus groups. In the second phase, we wanted to gain further insight into the students’ perspectives as iPad users, as we believed that it was important to give the children and young people a ‘voice’ and involvement in their care and life choices [14]. Therefore, students with complex and multiple needs were invited to participate in an iPad user’s group to test a selection of apps and give feedback on their experiences. In the third phase data from these focus groups was used to construct a framework for the assessment of an individual’s ‘tablet readiness’ and an app evaluation matrix that was pilot-tested with children and young people at each of the schools before full implementation. In the final phase, the research team collated the individual app reviews and developed a compendium of case studies for use across the school sample sites along with an assessment framework.
4
Results
4.1
Phase 1 - Staff perspectives
Data was collected via three staff focus groups, one at each of the three sample sites. The goal in these focus groups was to explore staff perceptions and experiences of using iPads and identify training and support needs. As such, there was no definitive schedule or set of questions; instead, staff were asked to complete an initial activity by writing comments individually on post-it notes to identify: • •
Perceived challenges and barriers to use Level of technological confidence and ability
This feedback was then used to inform the group discussion activities. The data from the focus groups indicated that there was a range of prior experience and knowledge within the staff teams at each site. Whilst all teaching and support staff were interested in developing the use of iPads and apps some were concerned about their own technical skills when using the devices and the overall quality of some of the apps available to meet specific communication and learning needs. A significant challenge that was identified was that staff were unsure how to select appropriate apps for supporting particular learning needs and communication difficulties; particularly those that had limited prior experience of using tablet technology to support learning. Sample written comments from the staff included: ‘I have my own iPad and use a lot of apps already, I’m keen to get started’. ‘I have no idea how to use one but with training I think I would be fine.’ ‘I feel that I would need further training as I feel that it could do so much more.’ ‘Unaware which applications are available or how to use them.’ ‘Not enough knowledge and not understanding how the iPad can be best used.’ When the research team collated the focus group feedback it was clear that a significant number of staff lacked confidence and basic skills to use the iPads therefore we developed a series of resources, linked to training sessions on: • • • •
how to assess individuals’ abilities for iPad usage how to set-up iPads to make them usable for children and young people with particular needs adapting the iPad settings to enable individuals to use them effectively evaluating and choosing iPad apps.
Following this training needs analysis the project team developed a series of both one-to-one and group training sessions and resources for staff to facilitate their skill development. 4.2
Phase 2 - Student’s perspectives
Data from students were collected from two of the three sample sites; as the children at the third site were under five and deemed too young to give in-depth feedback. These student groups consisted of a total of twenty-one students with a range of abilities and communication needs - twelve male participants with Autistic Spectrum Conditions, four male participants with profound and complex learning needs and four female participants with profound and complex learning needs. The students were each provided with iPads and given a range of apps to use and asked to give feedback on a selection of apps. Those that were able to speak gave verbal feedback whilst symbols for: ‘Fantastic’, ‘Not sure’ or ‘Terrible’ were available for the non-verbal students. All of the students indicated that they enjoyed using the iPads for a range of different purposes and the responses indicated that they particularly liked apps that allowed them to create images; record
achievements using photos; sequence activities and use cause and effect apps to plan tasks. All of the apps were rated as ‘fantastic’ by the students using the symbols, sample comments from the students that were able to vocalise their perceptions included: ‘I like it, I think the drawing is good.’ ‘I think it was special being able to use the iPads.’ 4.3
Phase 3 – Development of the iPad assessment framework and App evaluation matrix
The next phase of the project had two specific purposes: • •
the development of an assessment framework to evaluate the needs of individual children and young people; and review whether iPad usage was possible or appropriate. the development of an app evaluation framework so that a standardised review of the apps could be undertaken.
The project team began by working with individual children and young people to undertake an initial assessment to gain a shared understanding of their requirements and desires. The aim of this initial assessment was to work in partnership with families, carers and professionals to identify the most appropriate device, accessories (case, stylus, etc.) and apps that would support both communication and learning considering: • • • • • •
a typical range of postures/ environments the physical movements that an individual can complete their visual processing of images level of support available from family members and carers usual method of communicating and interacting with others the aims and expectations of iPad usage
Although each assessment was specific to the individual, common factors that needed to be considered were: • • • • •
Cognitive abilities Sensory abilities Physical abilities Environment Support
The project team subsequently developed standardised guidelines relating to the use of tablets to ensure consistent assessment including: • •
an assessment framework for tablet management, a guide on accessibility for tablet devices,
Following the development of these assessment criteria and guidelines the project team collaborated with school staff and therapists to identify core skills and curriculum areas over which the use of iPads would be evaluated within each of the three schools: • • •
nursery for children aged 3-5 years old – cause and effect, mark-making, special needs school for children aged 3 -18 years old – mark-making, basic numeracy and literacy college for young people with additional needs aged 16-25 – cause and effect, art, and music
Working in partnership with the school staff and therapists we subsequently identified 100 apps for use across these curriculum areas. This was achieved via a systematic search of the Apple iTunes app store; google searching for content areas e.g. vocabulary, cause and effect as well as parent and practitioner
recommendations. Our focus was on the learning and communication needs of the children and young people and we evaluated the apps’ potential usefulness via a two-phase programme of testing. For apps that were initially identified as potentially useful via the app store descriptions, we developed a preliminary evaluation matrix that included the following questions: • • • • • • •
what specific activity/ability were you trying to encourage with the app? was the app age developmentally appropriate for the child/young person? was the app intuitive/accessible and easy to use? does use of the app support individual needs and goals? can the app be personalised for individual skill levels? how does the app build on existing skills and support learning and/or communication? does the app allow the individual child/young person to do something new?
These preliminary evaluation criteria were informed by a review of the existing school and therapy developmental assessment frameworks used to identify individual needs at the three partnership schools, as well as app evaluation tools sourced via an internet search. To test the usability and appropriateness of these criteria the research team undertook a pilot test with three members of staff and six young people. The reviewers in this pilot test gave both verbal and written feedback and these were subsequently refined into: • • 4.4
guidance on how to evaluate tablet-based apps appropriately and select those that can be used to meet specific needs, fourteen-point evaluation matrix with space for open-responses (see Appendix 1). Phase 4 – Development of the compendium of case studies
Following the development of both the initial assessment tool for individual children and young people and the app evaluation matrix, the final phase of the study was an evaluation of the individual apps based on cumulative feedback from staff, parents and children/young people using the double-sided form evaluation framework (see Appendix 1). Each of the school sample sites worked in partnership with the research team to agree a list of apps that would be tested each school term. These apps were then tested using an ‘age and stage’ approach within each of the three schools; whereby each app was used with children and young people with a range of both chronological and developmental ages as well as a range of specific learning and communication needs. As such each app was tested numerous times, across users and in a variety of different learning environments including class-based settings; community settings and home-settings. This process of app testing allowed the research team to collate the reviews from the school staff, therapists and carers and identify where reviews were widely divergent. In the case of divergent reviews the research team undertook a second stage review to ensure that the assessment and review criteria had been applied consistently. If the criteria had not been applied correctly the reviews were discarded; if they had, then staff were interviewed to further explore their experiences. At the end of each school term testing period, these reviews were subsequently developed into case studies. Each case study detailed a vignette of the use of an app for a specific user including, ‘where’, ‘how’ and the learning outcome achieved. These were uploaded on to an online database that the school staff used to inform future practice. The database was designed to facilitate finding apps via a range of routes, specifically users could search by: •
•
developmental stage o cooperative o exploratory o sensory o sequences o single action learning goals o cause and effect
•
•
o speech sounds o taking turns o vocabulary curriculum areas o early years o communication o mathematics o music o art communication strategy o language skills o communication skills o aided and unaided communication
On completion of the study 104 case studies were developed and made available to each of the schools to support further development of apps for learning and communication.
5
Discussion
This small-scale project at three specialist school settings within the UK provides an insight into how the use of iPads can be systematically evaluated and embedded within curricula to support learning and communication. By working collaboratively with school partners and responding to their concerns and challenges we developed a framework for both the assessment of iPad and app usage as well as a set of staff training materials. Specifically, the aims of the project were to evaluate the issues and challenges on the use of iPads with this client group and design practical solutions that enabled staff and carers to support the learning and communication of children and young people with a wide-range of additional needs. During the initial staff focus groups some staff were positive about the potential of the iPads to support both student learning and communication. However a significant number also expressed concerns and an analysis of the focus group data identified that further training and support were required for staff to support usage of the devices. Through the design of training materials and linked training packages this project developed practical evaluation solutions that enabled staff and carers to assess the suitability of iPad and app usage for use across a wide spectrum of age, ability and need. In common with previous research [15,16], staff and students within this project found the introduction of a new technology into the curriculum to be of benefit. However, this study indicates that there is an on-going challenge for staff in keeping up-to-date with new and emerging technologies and in identifying and promoting the use of tablet devices and apps to facilitate learning and communication. Initial assessment of children and young people and systematic evaluation of apps is essential for appropriate usage of any tablet based technology, and whilst the results of this study are positive we note that the project focused on just three school settings and relied on reflective retrospective reports of the practitioners in those settings. As such this information is not directly generalisable to other settings but does provide an insight into the experiences of iPad usage within this group of children and young people, which may be transferable to schools with similar populations.
6
Conclusions
This paper, based on a dynamic, two-way process in a review of practice and evaluation of the technology, gives an insight into how systematic frameworks for iPad usage can be embedded into curricula to support learning and development. The potential for using iPads to improve learning outcomes for students with disabilities, complex needs and Autism Spectrum Conditions is currently unknown, and in order for iPad technologies to be harnessed as an effective learning tool consideration must be paid to the understanding, skill and confidence of those who support and teach the users. An increase in accessible training and support resources is needed to ensure that use of iPad technologies has an infra-structure to support success and negotiate initial technical challenges and to set appropriate expectations for both learners and adults involved in their care.
Each child/young person is an individual, technology can support individual needs and through the use of iPads and a selection of apps in this study we have enabled children to complete activities more independently and provide opportunities to enhance social and emotional growth. By utilising a combination of interdisciplinary expertise this research led to developments in assistive technology at each of the schools and supported learning, social inclusion and communication for the children and young people involved. Thus, by providing technology expertise and developing a systematic system of evaluating apps in partnership with both school staff and therapists we have turned the initial research into real-life benefits by developing a range of resources and initiatives to create opportunities for children’s skill development and promote social inclusion. The research has subsequently developed a basis for good practice for iPad usage with these children and young people; developing a range of strategies and new innovations that addressed students’ communication and learning needs in more efficient ways. The interdisciplinary expertise arising from the project has also been used to develop a portfolio of educational, therapeutic and rehabilitative resources about tablet technologies, consultancy services and a compendium of apps appropriate for children with complex and multiple learning needs.
Acknowledgments This study was part-funded by both Manchester Metropolitan University and Health Education North West. The authors would like to acknowledge the contribution of each of the three specialist partnership schools and the Manchester Metropolitan University staff and students who contributed to the research project.
Appendix 1: App evaluation matrix
Name of reviewer
………………………………………………………………………………………
Project school
………………………………………………………………………………………………………………………………
Name of app
………………………………………………………………………………………………………………………………
Cost of app
………………………………………………………………………………………………………………………………
Creator of the app
Date
………………
………………………………………………………………………………………………………………
Summary of the app content
Activities the app was used for/how are you using the app?
Age/abilities of student(s)/child(ren) using the app
Does the app allow you to do something you couldn’t do before?
Please tick as appropriate
Content and components of the app
Yes
No
N/A
Authenticity – Does the app promote appropriate skill development? Differentiation – Does the app allow users to alter settings to meet individual needs? Usability – Can students/children use the app independently? Motivation – Are students/children motivated to use the app? Learning styles – Does the app address more than one learning style (visual/auditory/kinesthetic)? Images – Does the style of image add to the student/child’s learning? Sound – Does the music/sound add to the student/child’s learning? Balance – Does the app have a good balance of features to engage a range of users? Instructions – Are the instructions include with the app helpful? Support - Does the app have supporting information/website with additional information? Monitoring – Does the app allow for progress monitoring Data - Can student data be shared with teacher/parent? Accounts – Can more than one user account be set up? Interaction/communication – Does the app encourage the user to interact /communicate with others?
Summary of the app - Using the space below please describe why you would or would not recommend the app
References 1. 2. 3. 4.
5. 6.
7. 8.
9.
10.
11.
12. 13. 14.
15.
16.
Department of Health (DoH): Embedding informatics in clinical education (eICE)[online] available at: http://www.eiceresources.org/ (2014) Haßler B., Major, L., Hennessy, S.: Tablet use in schools: a critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning, 32, (2), pp.139–156. (2016) Health Education England (HEE): Framework 15, Health Education England Strategic Framework 2014-2029. (2014) McNaughton, D., Light, J.: ‘The iPad and Mobile Technology Revolution: Benefits and Challenges for Individuals who require Augmentative and Alternative Communication.’ Augmentative and Alternative Communication, 29 (2), pp.107-116. (2013) Berger, E.: The iPad: Gadget or Medical Godsend? Annals of Emergency Medicine 56(1), pp. 21A-22A. (2010) Mang, C.F., Wardley L.J.: Effective Adoption of Tablets in Post- Secondary Education: Recommendations Based on a Trial of iPads in University Classes. Journal of Information Technology Education: Innovations in Practice Vol 11, pp. 301-317. (2012) NHS England: NHS Choices health apps library [online] available at: http://apps.nhs.uk/ (2013) Spooner, F., Ahlgrim-Delzell, L., Kemp-Inman, A., Wood, L. A.: Using an iPad2® with systematic instruction to teach shared stories for elementary-aged students with autism, Research and Practice for Persons with Severe Disabilities, 39(1), pp.30-46. (2014) Oakley, G., Howitt, C., Garwood, R., Durack, A. R.: Becoming multimodal authors: Preservice teachers' interventions to support young children with autism, Australasian Journal of Early Childhood, 38 (3), pp. 86-96. (2013) Johnson, G. M.: Using Tablet Computers with Elementary School Students with Special Needs: The Practices and Perceptions of Special Education Teachers and Teacher Assistants, Canadian Journal of Learning and Technology, 39 (4), n4. (2013) Shane, H. C., Laubscher, E. H., Schlosser, R. W., Flynn, S., Sorce, J. F., Abramson, J.: ‘Applying technology to visually support language and communication in individuals with autism spectrum disorders.’ Journal of autism and developmental disorders, 42(6), pp.12281235. (2012) Todhunter, C.: Undertaking Action Research: Negotiating the Road Ahead, Social Research Update, Issue 34. (2001) Savin-Baden, M., Howell Major, C.: Qualitative Research: the essential guide to theory and practice, London: Routledge. (2013) Coad, J., Flay, J., Aspinall, M., Bilverstone, B., Coxhead, E., Hones, B.: Evaluating the impact of involving young people in developing children’s services in an acute hospital trust. Journal of Clinical Nursing, 17, pp. 3115-3122. (2008) Ashburner, J., Ziviani, J., Pennington, A.,: The introduction of keyboarding to children with autism spectrum disorders with handwriting difficulties: A help or a hindrance? Australasian Journal of Special Education, 36 (01), pp.32-61 (2012) Neely, L., Rispoli, M., Camargo, S., Davis, H., Boles, M., The effect of instructional use of an iPad® on challenging behavior and academic engagement for two students with autism, Research in Autism Spectrum Disorders, 7(4), pp. 509-516 (2013)
