A Virtual Reality Platform for Assessment and Rehabilitation of Neglect using a Kinect Pietro CIPRESSO 1, Silvia SERINO 1-2, Elisa PEDROLI 1, Andrea GAGGIOLI 1-2 , Giuseppe RIVA 1-2 1
Applied Technology for Neuro-Psychology Lab, Istituto Auxologico Italiano, Milan, Italy,
[email protected] 2 Psychology Department, Catholic University of Milan, Italy,
[email protected] Abstract. Virtual reality can be an effective neuropsychological tool for assessment and rehabilitation of Neglet. We developed a 3D Virtual Reality platform for the assessment and rehabilitation of cognitive deficits, in particular for neglect. Within the Virtual Environments, it is possible to interact with virtual objects and execute specific exercise for Neglet, using a Kinect.
Introduction The need of a new, complete and simply virtual reality (VR) system for assessment and rehabilitation pushed us to create a new set of virtual environments (VE). The innovations of using virtual environments were several: first of all, the possibility to re-create ecological setting, but by testing in a laboratory. Scenarios that are too dangerous or expensive in the real world can be used in VR with relative easiness and cost efficiency. Classic assessment and rehabilitation of neglect. Unilateral Spatial Neglect (USN) is normally assessed with paper-and-pencil tests. In particular, cancellation tests is the most used for neglect assessment: in this task patients should cross out target objects, sometimes embedded in distracters. In addition we can use as tests: object copying tests, line bisection, description of complex figures and reading of words and phrases.In rehabilitation the therapists use spatial exploration task, reading, research tasks and other "passive methods" like prism adaptation, vestibular stimulation and others. Virtual assessment and rehabilitation of neglect Several protocols have been developed for assessment and rehabilitation of neglect. Some studies “translated” paper-and-pencil test in a 3D Virtual environment. In other studies researchers developed a new environment where patients had to perform tasks like track: a virtual ball moving in random directions, grab a target sushi with virtual chopstick and place it in a designated location and using a virtual wheelchair. For rehabilitation different scenarios were created: street crossing application for training patients with USN to cross the street or grasping a virtual object located within a virtual environment using a virtual hand. Often, these programs are integrated with device like Kinect. Interfaces development for input/output hardware devices for applications in neurorehabilitation Recently, there has been a progressive diffusion on home gaming market of advanced game technologies (such as Microsoft Kinect and Nintendo Wii) that allowed the use of a large series of interactive devices at a low cost [2-3]. Microsoft Kinect, beyond a low cost (due to the fact of being targeted on consumer market), makes available Software Development Kit (sdk) to integrate third-part software. NeuroVirtual 3D platform aims to use such commercial devices to integrate new interactive
functionalities in NeuroVR platform [4], to be used and tested in neurorehabilitation framework. This strategy is to obtain a double result: On one hand, to add new interactive features to the VR platform to allow an higher range of contents and neurorehabilitative exercises; on the other hand, to take advantage from these technologies to offer solutions based on low cost and high availability. NeuroVirtual 3D for Neglet We developed two main environments, a VR- home for the assessment and a VR-city for the rehabilitation. VRHome is a two-storey house including a garden. On the ground floor there are living room and kitchen, on the first floor there is a bedroom, a bathroom and a studio. In this environment patients have several ecological task divided into different rooms: for example, in the living room participants should distribute cards for a game and serve tea; or also, in a bedroom must look for clothes in the closets or looking for an object on a desk. The VRcity is made up of 3x3 streets with shops, a square and different points of interest. Patients must look for stores, message boards, trash bins and people in a square. Acknowledgements This study was partially supported by the research project “NeuroVirtual 3D”, funded by Regione Piemonte (Grant No. FA 211-432C- 2012). References [1] Tsirlin, I., Dupierrix, E., Chokron, S., Coquillart S., Ohlmann, S. “Uses of Virtual Reality for Diagnosis, Rehabilitation and Study of Unilateral Spatial Neglect: Review and Analysis” CYBERPSYCHOLOGY & BEHAVIOR Volume 12, Number 2, 2009 [2] D. Giakoumis, A. Drosou, P. Cipresso, D. Tzovaras, G. Hassapis, A. Gaggioli, and G. Riva, Using activityrelated behavioural features towards more effective automatic stress detection, PLoS One 7 (2012), e43571. [3] J. Lee, C. Chao, A.L. Thomaz, and A.F. Bobick, Adaptive integration of multiple cues for contingency detection, in: Human Behavior Understanding, Springer, 2011, pp. 62-71. [4] G. Riva, A. Gaggioli, A. Grassi, S. Raspelli, P. Cipresso, F. Pallavicini, C. Vigna, A. Gagliati, S. Gasco, and G. Donvito, NeuroVR 2--a free virtual reality platform for the assessment and treatment in behavioral health care, Stud Health Technol Inform 163 (2011), 493495.
Illustrations
#5. Device-NeuroVirtual 3D interaction interface
#6. Kinect-NeuroVirtual 3D. interaction interface #1. An overview of the Virtual Environment "Kitchen"
#2. An overview of the Virtual Environment "Studio"
#3. An overview of the Virtual Environment "Bed room"
#4. An overview of the Virtual Environment "City (square)"
#7. Video signal processing from Kinect and Avatar structures
