1
Proposal of Evaluation Methods of Interaction between a Mobile Robot and Children with Autism Spectrum Disorder Christiane M. Goulart† , Javier Castillo∗ , Carlos T. Valado∗ , Eliete Caldeira∗ , Malte Trauernicht‡ , and Teodiano F. Bastos-Filho∗† † Post-Graduate Program in Biotechnology ∗ Post-Graduate Program in Electrical Engineering Federal University of Espirito Santo, Av. Fernando Ferrari, 514, Vitoria, Brazil ‡ Graduate Program in Electrical Engineering, Hanze University of Applied Science, Institute of Engineering, Groningen, The Netherlands Email:
[email protected];
[email protected];
[email protected];
[email protected];
[email protected];
[email protected]
Abstract—Manifestations of cognitive and neurobehavioral disorders such as repetitive movements, impaired communication and social interaction level characterize Autism Spectrum Disorder - ASD, a condition of unknown etiology and still without cure. Currently, many researches integrate robotics to the treatment of ASD with varied functions that allow interaction with children with this disorder. Therefore, evaluation criteria become important to analyze the efficiency the use of robotics in this interaction. The methods here proposed are Goal Attainment Scaling (GAS), System Usability Scale (SUS) and Fitts’ Law. The use of such methods of evaluation allows transpose qualitative assessments into quantitative, making the work more robust and reliable. Index Terms—ASD; Mobile Robotics; GAS; SUS; Fitts’ Law.
I. I NTRODUCTION A. Autism Spectrum Disorder (ASD) Among the characteristics associated with Autism Spectrum Disorder (ASD) stand out social deficit, deficits in communication and in the production and perception of affective expressions [1] [2]. ASD includes Autism, Asperger Syndrome and Pervasive Developmental Disorder Not Otherwise Specified, with variations in severity and nature of symptoms [3], [4], [5]. The pervasive developmental disorder not otherwise specified and autism include conditions that may or not be associated with mental retardation [5]. Autism signs are the difficulty in talking about personal feelings and understand the feelings of others, lack of engagement in interactive games, lack of eye contact and joint attention, communication difficulties and sensitivity to physical contact [5]. On the other hand, Asperger syndrome is characterized by deficits in social interaction and limitation of interests and behaviors, but it is differentiated in its early development by the absence of delay in speech or in perception, cognitive development, self-help skills and in curiosity about the environment [6]. The prevalence of ASD is estimated between 1 and 6 children in 1000, while the prevalence of autism comprises an average of 5.2 per 10,000 children, varying according to the determination of
the case [7].The etiology of ASD is not yet specified, but it is probably related to multifactorial conditions of genetic and nongenetic causes (environmental and biological) [8], [9] Individuals belonging to this spectrum have difficulties in performing independent functions in many important areas of social and occupational life [1]. Currently, there is no cure for this condition, but there are behavioral treatments that can improve the quality of life and independence of individuals with ASD [10], [5]. B. Mobile Robotics Beyond human intervention, not human intervention, as social robotics, has been used as a tool for therapy of social and communication skills of children with ASD [1], [11]. Furthermore, social robotics allows to establish situations of meaningful and sophisticated interaction, using speech, sound, motion and visual indications [10]. Many studies show that the interaction between children with ASD and robots aids the interaction with parents, caretakers and other mediators, for example, when the child shows excitement in contact with the robot and turns to one of the parents, expressing his/her excitement [1], [12]. There are also studies that say that interaction with mobile robots has been satisfactory for children with ASD, because robots are predictable, simple and easy to understand [11]. C. Evaluation Methods Many studies use evaluation scales that provide quantitative values that allow a desired analysis of an object of study. These scales are important because they allow quantitatively assessing subjective or complex phenomena, such as quality of life, disability, efficiency of a computer system, among others [13]. This paper will address the following evaluation scales: GAS, Fitts’ Law and SUS, to analyze activities related to mobile robot.
2
Goal Attainment Scaling (GAS) was developed in 1968 in a study of mental health professionals, involving two steps [14]. The first one was relative to construction of a goal attainment guide for each patient, based on the level of desired or expected performance. The second one was to apply this guide in assessing the performance of the patient at a predetermined time after the intervention. In addition, a formula was introduced to calculate the goal attainment score [14]. GAS has application as an evaluation method in many fields beyond mental health, such as psychotherapy, education, mental impairment and general rehabilitation [14]. Studies of psychosocial interventions for ASD [15] conclude that using GAS holds promise as an ideographic approach for measuring intervention outcomes and also provide recommendations for defining specific quality indicators. Following these recommendations, the work presented in [16] also supports the use of GAS for manualized interventions in ASD studies, however, further validation of GAS findings on direct observation of goal attainment and sensory behavior remains necessary. Fitts’ Law, proposed in 1954, forecasts necessary time for that a movement occurs towards a target area in function of distance and target size. Its popular model is widely adopted in various research areas, including kinematics, human factors, and recently, human-computer interaction [17]. The System Usability Scale (SUS), created in 1986, consists of a questionnaire that can be used as an extremely simple and reliable tool for evaluate how people perceive the usability of a wide variety of products and services on which they work [18]. The measures of usability presents aspects such as efficacy (whether users can successfully achieve their goals), efficiency (how much effort and resources were spent to achieve these goals), and satisfaction (whether the experience in achieving those goals was satisfactory) [18]. Moreover, it has been reported that SUS can be readily applied to evaluate any type of system and is not biased towards gender or by the user interfaces [19], [20]. Field studies with mobile applications for children with ASD have successfully employed SUS to draw quantitative comparisons between annotation tools [21] and for competency training [22]. Thus, the purpose of this paper is to use the aforementioned methods evaluate the interaction between children with ASD and a mobile robot, the robot movement and its usability and efficiency in stimulating social interaction skills of children. II. M ETHODOLOGY A. Mobile Robot The mobile robot model to be used in this study is a PIONNER 3-AT, one of most popular robot in education and research and it has four wheels, four engines and was developed for the exploration of environments or laboratory experimentation [17!]. A playful image was adapted to the PIONNER 3-AT (Figure 1), a display and a sound system in order to attract the child’s attention. A video camera is used
to capture the childs face images. Moreover, the robot also has a laser sensor for automatic detection and location of the child, so that it can establish a minimum safe distance. Thus, the robot will approach and distance itself from the child, eliciting images and sounds and stimulating the beginning of an interaction. B. Goal Attainment Scale Goal Attainment Scale (GAS) is here used to evaluate the success of the interaction between the child and the robot. Each objective stipulated in the interaction will receive a score according to the success in performing it. This score ranges from -2 (task performed much worse than expected) to +2 (task performed much better than expected). The total score of a goal is given by Equation 1 [23]: Pn i=1 gi (1) T = 50 + 10 · √ n − R · n + R · n2 where: gi is the estimated value of the accomplished task (from -2 up to +2) to the i-th task; n is the number of goals; R estimated average intercorrelation for the outcome scores, normally 0.3; T : a standardized score with a mean of 50 and a standard deviation of 10.
Fig. 1: Mobile robot PIONNER 3-AT (left) and its ludic image (right). Table I describes the proposal of levels of attainment for two goals that will be evaluated in the process of interaction between the child with ASD and the robot. A numerical value is assigned for each activity performed, with 0 indicating the expected level of performance and -2 and +2, indicating the least and most favorable results, respectively. T equal to 50 corresponds to the expected level of performance ranging from - 2 to +2; a T greater than 50 reflects performance above the expected level and a T lower than 50 reflects performance below the expected level [14], [23].
3
TABLE I: Goal Attainment Scaling for two goals
item). The items are [26]:
Goals Predict Attainment
Score
Look at the robot
Touch the robot
Most unfavorable outcome
-2
Lookwatwthewrobotwfor lesswthanw30wseconds andwpresentwrepulsion
Downotwtouchwthe robot
Lesswthan expected outcome
-1
Lookwatwthewrobotwfor Touchwthewrobotwfor lesswthanw30wseconds lesswthanw5wseconds andwnowinterest
Expectedwlevel ofwoutcome
0
Lookwatwrobotwfor Touchwthewrobotwfor morewthanw30 morewthanw5 secondswand maintainweyewcontact seconds withwthewscreen
Greaterwthan expected outcome
+1
Mostwfavorable outcomewlikely
+2
1) 2) 3) 4)
Lookwatwthewrobotwfor Touchwthewrobotwfor morewthanw30 morewthanw5 secondswandwgo secondswandwplay towardswit withwit Lookwatwthewrobotwfor Touchwthewrobotwfor morewthanw30 morewthanw5 secondswandwpay secondswandwpay attentionwtowthe attentionwtowthe screen screen
Another method of interaction analysis consists of behavioral assessment of children with ASD, which will take into account the emotional state of the child at the time of the interaction with the robot. For this, it is important the aid of a team of psychologists, pedagogues, teachers and caretakers involved in working with children with ASD. C. Fitts’ Law To assess the precision of the robot motion towards the child, referring to the time it takes to attract his/her attention, Fitts’ Law will be used, whose accuracy depends on both the distance to the target and the size of a target. The time for a motion (MT) is given by Equation 2 [24], [25]: M T = a + b log2 D/W,
(2)
where: a and b: constants, D: the distance from the starting position to the target center, W : the target width. D. System Usability Scale(SUS) To measure and classify the ease of use (usability) of mobile robot as possible interactive tool for children with ASD, the use of a System Usability Scale (SUS) is here suggested. SUS consists of ten items, where even-numbered items worded negatively, and odd-numbered items worded positively. The parents or caretakers will evaluate the use of the proposed system using 5-point scales numbered from 1 (Strongly disagree) to 5 (Strongly agree); the number 3 means the center of the rating scale (if a volunteer fails to respond to an
I think that I would like to use this system frequently. I found the system unnecessarily complex. I thought the system was easy to use. I think that I would need the support of a technical person to be able to use this system. 5) I found the various functions in this system were well integrated. 6) I thought there was too much inconsistency in this system. 7) I would imagine that most people would learn to use this system very quickly. 8) I found the system very cumbersome to use. 9) I felt very confident using the system. 10) I needed to learn a lot of things before I could get going with this system. The contribution for the items’ values ranges from 0 to 4 (being 4 the most positive answer). For odd items (worded positively), subtract 1 from the value given by the user, and for the even items (worded negatively) subtract 5 from the value given by the user. Then, the scores are added and multiplied by 2.5 to get the overall value, ranging from 0 to 100. Normally, in SUS, values above 68 are considered above average, while values below 68 are below average [18]. III. R ESULTS The use of the mobile robot can trigger a positive interaction, i.e., stimulate attention and movement of children with ASD, as well as the ability to interact with the environment and people around them [10]. Therefore, it is important to have efficient methods of assessment to ensure the robot’s utility in interactions with the child. GAS allows quantifying the subjective evaluation of the interaction between the child with ASD and the robot and it has been used in other works with children with special needs [27]. The association of behavioral and emotional analysis of the child, together to GAS, performed by professionals, will unleash a more effective analysis of the process of interaction between child with ASD and robot. Fitts’ law provides efficiently an assessment of the accuracy of the robot motion to that the child’s safety and integrity are preserved at the time of his/her interaction with the robot. Finally, to evaluate the possible use of mobile robot as a pedagogical tool for children with ASD, the SUS attends efficiently. Research results show that the SUS enables a highly robust and versatile evaluation of the usefulness of the system under study [19]. IV. C ONCLUSIONS AND FUTURE WORKS The methods here suggested for evaluating the interaction between mobile robots and children with ASD, the robot’s usefulness and the accuracy of its movement are efficient, making results more robust and reliable. Future works include conducting studies on other forms of interaction with other robot’s modes in order to evaluate its influence and its use as potential therapeutic tool in the cognitive and social evolution of children with ASD.
4
ACKNOWLEDGEMENTS Authors thank CNPq and CAPES for the scholarship. R EFERENCES [1] E. Kim, L. Berkovits, E. Bernier, D. Leyzberg, F. Shic, R. Paul, and B. Scassellati, “Social robots as embedded reinforcers of social behavior in children with autism,” Journal of Autism and Developmental Disorders, vol. 43, no. 5, pp. 1038–1049, 2013. [2] B. Robins, F. Amirabdollahian, Z. Ji, and K. Dautenhahn, “Tactile interaction with a humanoid robot for children with autism: A case study analysis involving user requirements and results of an initial implementation,” 2010, pp. 704–711. [3] Ministry of Health, “Guidelines of the attention for rehabilitation of the person with Autism Spectrum Disorders (ASD).” Ministry of Health, Brazilia, Brazil, 2013. [4] D. Rossignol and R. Frye, “Mitochondrial dysfunction in autism spectrum disorders: A systematic review and meta-analysis,” Molecular Psychiatry, vol. 17, no. 3, pp. 290–314, 2012. [5] B. Scassellati, H. Admoni, and M. Matari, “Robots for use in autism research,” Annual Review of Biomedical Engineering, vol. 14, pp. 275– 294, 2012. [6] A. Klin, “Autism and asperger syndrome: An overview [autismo e sndrome de asperger: Uma viso geral],” Revista Brasileira de Psiquiatria, vol. 28, no. SUPPL. 1, pp. S3–S11, 2006. [7] [7] World Health Organization (WHO), “Weekly epidemiological record,” pp. 17–24, 2003. [8] M. Rutter, “Aetiology of autism: Findings and questions,” Journal of Intellectual Disability Research, vol. 49, no. 4, pp. 231–238, 2005. [9] R. C. Campos, “Neurological aspects of infantile autism,” Pervasive Developmental Disorders: 3rd Millennium, vol. 2, 2005. [10] F. Michaud and A. Clavet, “Robotoy contest - designing mobile robotic toys for autistic children,” 2001. [11] A. Duquette, F. Michaud, and H. Mercier, “Exploring the use of a mobile robot as an imitation agent with children with low-functioning autism,” Autonomous Robots, vol. 24, no. 2, pp. 147–157, 2008. [12] H. Kozima, M. Michalowski, and C. Nakagawa, “Keepon: A playful robot for research, therapy, and entertainment,” International Journal of Social Robotics, vol. 1, no. 1, pp. 3–18, 2009. [13] J. Fermanian, “Validation des chelles d’valuation en mdecine physique et de radaptation: comment apprcier correctement leurs qualits psychomtriques,” Annales de Radaptation et de Mdecine Physique, vol. 48, no. 6, pp. 281 – 287, 2005. [14] K. Ottenbacher and A. Cusick, “Goal attainment scaling as a method of clinical service evaluation.” The American journal of occupational therapy. : official publication of the American Occupational Therapy Association, vol. 44, no. 6, pp. 519–525, 1990. [15] L. Ruble, J. McGrew, and M. Toland, “Goal attainment scaling as an outcome measure in randomized controlled trials of psychosocial interventions in autism,” Journal of Autism and Developmental Disorders, vol. 42, no. 9, pp. 1974–1983, 2012. [Online]. Available: http://dx.doi.org/10.1007/s10803-012-1446-7 [16] R. Schaaf, T. Benevides, Z. Mailloux, P. Faller, J. Hunt, E. Hooydonk, R. Freeman, B. Leiby, J. Sendecki, and D. Kelly, “An intervention for sensory difficulties in children with autism: A randomized trial,” Journal of Autism and Developmental Disorders, pp. 1–14, 2013. [Online]. Available: http://dx.doi.org/10.1007/s10803-013-1983-8 [17] S. MacKenzie, “Fitts’ law as a research and design tool in humancomputer interaction,” Human-Computer Interaction, vol. 7, no. 1, pp. 91–139, 1992. [18] J. Brooke, “Sus: a retrospective,” Journal of Usability Studies, vol. 8, 2013. [19] A. Bangor, P. Kortum, and J. Miller, “An empirical evaluation of the system usability scale,” International Journal of Human-Computer Interaction, vol. 24, no. 6, pp. 574–594, 2008. [20] S. Borsci, S. Federici, and M. Lauriola, “On the dimensionality of the system usability scale: A test of alternative measurement models,” Cognitive Processing, vol. 10, no. 3, pp. 193–197, 2009. [21] P. L. Weiss, E. Gal, S. Eden, M. Zancanaro, and F. Telch, “Usability of a multi-touch tabletop surface to enhance social competence training for children with autism spectrum disorder,” in Proceedings of the Chais conference on instructional technologies research 2011: Learning in the technological era. Raanana: The Open University of Israel, 2011, pp. 71–78.
[22] A. Sano, J. Hernandez, J. Deprey, M. Eckhardt, M. Goodwin, and R. Picard, “Multimodal annotation tool for challenging behaviors in people with autism spectrum disorders,” 2012, pp. 737–740. [23] T. Kiresuk, A. Smith, and J. Cardillo, Goal Attainment Scaling: Applications, Theory, and Measurement. L. Erlbaum Associates, 1994. [Online]. Available: http://books.google.com.br/books?id=XzgJQ27fd1AC [24] P. Fitts and J. Peterson, “Information capacity of discrete motor responses,” Journal of Experimental Psychology, vol. 67, no. 2, pp. 103– 112, 1964. [25] N. Ferrier, “Achieving a fitts law relationship for visual guided reaching,” 1998, pp. 903–910. [26] J. Lewis and J. Sauro, “The factor structure of the system usability scale,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5619 LNCS, pp. 94–103, 2009. [27] A. Cook, B. Bentz, N. Harbottle, C. Lynch, and B. Miller, “Schoolbased use of a robotic arm system by children with disabilities,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 13, no. 4, pp. 452–460, 2005.
