MARCOPOLO: CONTEXT-SENSITIVE MOBILE ... - RESNA

3 downloads 83657 Views 169KB Size Report
Android). MarcoPolo is more flexible than current AAC devices. It runs on ... configurable, and affordable. ... sensitive applications, runs on smart phones.
MARCOPOLO: CONTEXT-SENSITIVE MOBILE COMMUNICATION SUPPORT Carrie Demmans Epp, Rachelle Campigotto, Alexander Levy, Ron Baecker TAGlab, Department of Computer Science, University of Toronto ABSTRACT People with Autism Spectrum Disorder and aphasia often have impairments that limit their ability to communicate verbally. Consequently, their daily activities, social interactions, and quality of life are affected. We have developed a customizable context-sensitive application, MarcoPolo, that can help these populations overcome challenges by providing vocabulary support both orally and visually. Initial evaluations of MarcoPolo have helped refine the application and have illustrated that contextsensitive mobile applications can support communication. Several additional studies will be run to further evaluate the efficacy of context-sensitive communication support. Two of these will start in 2011 and early results will be presented at FICCDAT. KEYWORDS Autism, aphasia, Augmentative and Alternative Communication (AAC), mobile devices INTRODUCTION Individuals with Autism Spectrum Disorder (ASD) and aphasia have impairments that inhibit verbal communication. Augmentative and Alternative Communication (AAC) support has benefitted both populations [1]. However, few AAC devices have been designed to address the specific needs of the aphasia population and even fewer provide context-sensitive support [2]. For those with aphasia who have partially retained the ability to speak, AAC devices make speaking laborious [3]. MarcoPolo is a context-aware mobile communication application that runs on standard multi-use touch-screen devices that fit in the palm of your hand (e.g., iPhone or Android). MarcoPolo is more flexible than

current AAC devices. It runs on commodity hardware, is remotely customizable, and enables vocabulary navigation using the device’s built-in GPS. We are in the early stages of evaluating MarcoPolo's usefulness as an AAC device. We have performed feasibility and formative evaluations, which have been both informative and encouraging. We will be performing more structured qualitative and quantitative evaluations in early 2011. RELATED WORK The communication support needs of those with aphasia and ASD vary. People with aphasia experience language modality impairments and variations in the severity of their noncommunicative ability [4], whereas the communicative abilities of children with ASD range between having difficulties initiating and maintaining communication to being noncommunicative [5]. The support that is currently available to these populations ranges from large picture cards known as Picture Exchange System (PECS) [6] for those with ASD to technological supports, like DynaVox systems [7], for both populations. Device abandonment is an issue. Even with the many forms of assistive technologies that are available, 35% of AAC devices are abandoned shortly after their introduction for use as communication aids [8]. This is often attributed to the disadvantages of AAC devices, such as the high cost of purchasing them and the difficulties that users have communicating through pre-selected symbols [5]. Additionally, these devices are usually not portable and have a high learning curve that makes their adoption difficult [8]. These factors have prompted the repurposing of everyday technologies for AAC devices because they are portable, configurable, and affordable. Context-sensitive

applications that support communication aim to improve the symbol selection process that limits communication [4] as well as address the known deficiencies of traditional AAC devices.

vocabularies without requiring that the editor has access to the physical device.

Improving AAC is crucial for aphasia research: past studies of communication methods, such as picture boards and communication books, have been successful [9]. However, support for sorting through words and symbols on AAC devices in restricted contexts has had limited success [9]. The symbol selection process can be frustrating, and it can make communication slow. The use of technology, such as iPods, for students with exceptionalities has shown behavior and skill improvements, measured according to their Individualized Education Plans (IEP) [10]. However, few studies have examined how to employ pre-existing devices as assistive technologies. MarcoPolo, Proloquo2Go [11], and PixTalk [6] seek to fill this gap. Like most AAC devices, MarcoPolo stores vocabularies represented as words that are associated with images, but the vocabulary item's verbalization relies on text-to-speech instead of audio recordings. To communicate using MarcoPolo, a person chooses vocabulary items, which may then be read aloud. SYSTEM OVERVIEW Our research laboratory was approached by an aphasic man who wanted a better solution than the many communication aids that he was using. At the time, he always carried a briefcase full of aids, such as maps, a DynaVox, and a notebook in which he had organized an extensive vocabulary (over 8000 entries that consisted of more than 43000 words) that he used to support his communication (Figure 1). The challenges he faced and the limitations of the technologies that he was using inspired the creation of a context-sensitive AAC, called MarcoPolo, that runs on commodity hardware. MarcoPolo's vocabulary is fully customizable and its items can be organized by category or by geographic location. MarcoPolo also allows vocabularies to be edited on the hand-held device or through a web-interface. This accommodates anytime anywhere editing, and it allows both users and caregivers to edit

Figure 1: The aphasic man’s aids. Since MarcoPolo has text-to-speech capabilities, it can speak for the user or remind the user of words; this supports the user’s ability to speak for him or herself. MarcoPolo is also flexible enough to allow for sentence formation or the individual use of words within a sentence that is primarily spoken by the user. MarcoPolo, like many other contextsensitive applications, runs on smart phones. This enables it to be aware of the user's geographic context through GPS. MarcoPolo can then recommend relevant words based on the user's location because the device provides access to this information. This is important since it limits the effort required to sort through vocabulary that is organized in different categories and can be found in many locations. PRELIMINARY TESTING AND RESULTS There are several ways to evaluate contextsensitive systems. We will begin by discussing current methods and how we plan to use them. We will then discuss a feasibility evaluation as well as an ongoing formative evaluation. There is no consistent methodology for evaluating context-sensitive applications for users that have highly variable needs [12], like those with aphasia or autism. Many of the studies that get performed are purely qualitative and limit our ability to compare results for the systems being evaluated. This is partly due to the memory and communication limitations that are often present in the targeted populations. We have chosen a

methodology that is informed by the A-FROM model, because it focuses on the individual and can be combined with quantitative methods. A-FROM measures the quality of the individual’s life by considering the effect that interventions have on communication spheres, life situations, personal identify, and diagnosis severity [13]. We are including system use tracking in this evaluation model to help alleviate the need for users to remember or communicate their activities; the logged data recreates a reliable history of user actions. However, log data cannot reveal user intent. Combining system use data with the qualitative data of the A-FROM model allows for its proper interpretation [14]. In addition to evaluating completed systems, it is important to perform formative evaluations during the development of a system and at various points throughout its life; this ensures that the application continues to meet its users’ needs. However, at some point during the application’s development its features become stable enough to determine if the application is a feasible solution to the problem for which it was designed. To determine if MarcoPolo could feasibly support communication, we attempted to complete what we expected to be a common use case of the application: ordering a tea at a coffee shop. For this task, the evaluator was required to order a tea, prepared to her preferences, using only the context-sensitive application. She did not speak and could only communicate through the device. She ordered and received an extra-large steeped tea with one cream on a weekday morning. There was background noise from people talking and music was playing. The evaluator did not speak and had to increase the volume on her phone and reach it towards the server's ear to have the order filled. The achievement of the evaluator’s goal, getting a tea, supports the use of MarcoPolo in noisy settings, provided those communicating through the device are within arm’s reach. Furthermore, MarcoPolo facilitated communication in a noisy environment while failing to draw unnecessary attention because it was sufficiently quiet.

The man who inspired the creation of MarcoPolo has been participating in a formative evaluation of MarcoPolo for people with aphasia. During the early development of this application, he confirmed that vocabulary items should be associable with images. His feedback led to the creation of re-orderable lists of vocabulary items instead of limited fixed lists of words organized by someone else. He also wanted the ability to add previously created and organized vocabulary items automatically so that he did not need to enter them manually. This functionality is being developed, and it will enable users to share vocabularies that they can later customize to suit their needs. The most interesting thing about this alpha tester is that he has adopted the product. He says that he always takes MarcoPolo with him and that it is as good as or better than the other aids that he uses. Now that we have shown that using MarcoPolo can support communication and is perceived as useful by one of our current users, we can move on to more quantitative evaluations that rely on mixed-methodologies rather than solely relying on qualitative data. Upcoming studies will evaluate MarcoPolo and establish whether it can support communication efficacy for individuals with ASD and aphasia. We will consider the extent to which our context-sensitive application facilitates social interaction and communication for members of these populations. Using qualitative and quantitative methods, we will focus on real-life situations rather than standardized tests to gain insight from those that will benefit directly from MarcoPolo. We will also consider the insights of family members and professionals, such as clinicians and educators, since the best practices for ASD interventions emphasize collaboration and parental involvement [15]. The data collected from these groups will further illuminate the possibility of supporting communication with low-cost everyday technology. FUTURE WORK We have planned two studies for early to mid 2011. The first study will involve students

on the autism spectrum at a Toronto high school where a teacher will conduct a variety of curriculum-based lessons that will use MarcoPolo. Teachers will observe the students and report to the research team by documenting their observations and by participating in focus groups. Additionally, the frequency of application use, vocabulary use, and the addition of vocabulary items will be logged to capture any changes in user behavior within the context-sensitive AAC. The second study will involve individuals with aphasia at two Toronto organizations that serve aphasics. One organization focuses on using technology, while the other encourages using non-technological communication solutions. Participants will be given a device that runs MarcoPolo. We will log their interactions with the application, and we will evaluate their impressions, experiences, and feedback to determine if communication that uses location-specific vocabulary makes daily life easier. Qualitative and quantitative data will be collected from the perspectives of the aphasic users, their family members, and the clinicians involved in their care. We will gather information about their impressions and experiences through online surveys, questionnaires, and focus groups which will then be analyzed to determine areas for future work and development. The upcoming studies of the contextsensitive mobile application, MarcoPolo, should provide evidence that using an AAC on a mobile device is a viable solution. By addressing the shortcomings of AAC devices and obtaining feedback from individuals affected by the success or failure of various devices, we hope to gain a better understanding of what features can benefit these populations in their daily lives. We can then continue to improve tools to support them better.

REFERENCES [1]

[2]

[3]

[4]

[5]

[6]

[7] [8]

[9]

[10]

[11] [12]

[13]

[14]

ACKNOWLEDGEMENTS This research is supported by NSERC and Google Research. We would like to thank Bill Scott, Dr. Elizabeth Rochon, Aakash Sahney, and Kevin Tonon for their efforts.

[15]

G. Thunberg, A. Sandberg, and E. Ashlen, Speechgenerating devices used at home by children with Autism Spectrum Disorder: a preliminary study, Focus on Autism and Other Developmental Disabilities, pp.104-114, 2009. A. Walter, TalksBac a predictive communication device for nonfluent adults with aphasia, Journal of Language & Communication Disorders, pp.45-70, 1998. J. E. McGrenere, Insights from the Aphasia Project: designing technology for and with people who have aphasia, CUU'03 Conference on Universal Usability, Vancouver, pp.112-118, 2003. H. Goodglass, The assessment of aphasia and related disorders, 3rd Edition, Lippincott, Williamns, & Wilkins, Philadelphia, PA, 2001. M. Tentori and G. Hayes, Designing for interaction immediacy to enhance social skills of children with autism, Ubicomp, Copenhagen, Denmark, pp. 51-60, 2010. G. De Leo, C. Gonzales, P. Battagiri, and G. Leroy, A smart-phone application and a companion website for the improvement of communication skills of children with autism: clinical rationale, technical development and preliminary results, Journal of Medical Systems, pp. 1-9, 2009. DynaVox. http://www.dynavoxtech.com/ M. Dawe, Desperately seeking simplicity: how young adults with cognitive disabilites and their families adopt assistive technologies, CHI 2006, Montreal, Canada, pp.1143- 1152, 2006. A. Kraat, Augmentative and Alternative Communication: does it have a future in aphasia rehabilitation?, Aphasiology, pp.321-338, 1990. G. Marks and J. Milne, iPod therefore I can: enhancing the learning of children with intellectual disabilities through emerging technologies, Readings in Education and Technology: ICICTE, Ballarat Australia, pp.165-172, 2008. Proloquo2Go. http://www.proloquo2go.com/ R. Elman, A. Kagan, and N. Simmons-Mackie, Evaluating interventions for living successfully with aphasia what, why and how?, The Aphasia Institute, Toronto, 2007. K. Moffat, L. Findlater, and M. Allen, Generalizability in research with cognitively impaired individuals. Workshop on Designing for People with Cognitive Impairments, CHI 2006, Montreal, Canada, 2006. K. Church and B. Smyth, Understanding the intent behind mobile information needs, 13th international conference on Intelligent User Interfaces - IUI '09, Gran Canaria, Canary Islands, pp.247-256, 2008. K. Atchison, Best practices for designing and delivering effective programs for individuals with Autism Spectrum Disorders, California Department of Education and Development Studies, Los Angeles, 1997.