Towards Playful Monitoring of Executive Functions: Deficits in Inhibition Control as Indicator for Cognitive Impairment in First Stages of Alzheimer Lucas Paletta1, Martin Pszeida1, Mariella Panagl2 1 JOANNEUM RESEARCH Forschungsgesellschaft mbH DIGITAL - Institute for Information and Communication Technologies Steyrergasse 17, 8010 Graz, Austria {lucas.paletta, martin.pszeida}@joanneum.at 2 Sozialverein Deutschlandsberg Unterer Platz 7b, 8530 Deutschlandsberg, Austria
[email protected]
Abstract. Meaningful treatment of dementia today consists of multi-component interventions, such as, cognitive and also physical, sensomotoric stimulation. A serious game was developed for multimodal training performed by clients and caregivers using easily configurable services on a Tablet PC. A key problem in developing substantial knowledge about dementia and impacting factors is lack of data about mental processes evolving over time. For this purpose, eye tracking data were captured from non-obtrusive sensing during games to enable daily monitoring of dementia profiles. An anti-saccade measuring paradigm was used to detect attention inhibition problems that typically occur in executive function related neurodegenerative diseases, such as, in Alzheimer. In a 6 month study with 12 users a classifier was developed that enables to discriminate dementia stages from extracted eye movement features received from training at home. The playful training and its diagnostic toolbox offer affordances for entertaining users, measuring and analyzing mental process parameters, and enabling people with dementia to stay longer at home to slow down the progress of disease. Keywords: Alzheimer intervention system Eye movement features diagnostics Executive functions
1
Playful
Introduction
Dementia is a broad category of neurocognitive disorders that cause a long term and often gradual decrease in the ability to think and remember that is great enough to affect a person's daily functioning. Adequate, sufficient and economically feasible care is currently one of the greatest technological and social challenges [1]. A key problem in developing knowledge about dementia and its impacting factors is the lack of data about the mental processes and the psychophysiological dependencies as they
evolve over time. The individual trajectories of dementia are often suspected to be rather specific, however, longitudinal quantitative studies about dementia are rare. There is no cure for dementia [2], however, cognitive and behavioral interventions may be appropriate, educating and providing emotional support to the caregiver is important. Physical exercise programs are beneficial by activities of daily living and potentially improve outcomes [3]. Cognitive and also physical, sensomotoric stimulation is decisive for a meaningful treatment of dementia, however, lack of exercise is one of the major risk factors for the dementia development [4]. Therefore, multi-component interventions are important, even being accompanied by community settings [5,6]. A key problem in developing knowledge about dementia and impacting factors is lack of data about mental processes evolving over time. Cognitive and behavioral interventions, emotional support by caregivers and physical exercise programs are beneficial to activities of daily living [3]. However, lack of exercise is a major risk factor in dementia development [4].
Fig. 1. Playful multimodal training using a tablet PC with eye tracking device for executive functions diagnostics. The presented playful training presents an integrated theratainment solution for the growing market of care, rehab and diagnostics. In a serious game performed by persons with dementia (PwD), mobile eye tracking was applied for non-obtrusive sensing and daily monitoring of dementia profiles. An anti-saccade measuring paradigm was used for eye movements captured during playing the Tablet PC serious game. It is known to detect impulse control problems as they occur in executive function related neurodegenerative diseases [7]. The results gained from the experiments demonstrate that the serious game attentional diagnostic toolbox offers affordances for entertaining and analysis of behavioral parameters for longitudinal studies.
2 2.1
Playful Training and Mental Health Related Work
Current technical assistance for PwD exclusively focus on applications with cognitive training and omit important training aspects that are beneficial in multi-component interventions. Playful cognitive stimulation for elderly (Lumosity, Cogmed, Cognifit) and with particular consideration of people with dementia (eMotiva, Onto D’mentia, memofit) has been largely exploited. European projects, such as, AALJP M3W, CCE, ROSETTA, GAMEUP CAREBOX and ALFA have investigated means of monitoring and estimating the status of mental processes, motivating users for cognitive and physical activities in various ways. [8] provides a thorough overview on a plethora of serious games for dementia (SG4D) for physical, cognitive, psychosocial [9] and social-emotional health functions with various therapeutic achievements. [8] proposed a taxonomy on SG4D, concluding that assessment games are still highly underrepresented in the domain.
(a)
(b) Fig. 2. Playful multimodal training using the serious game amicasa with a tablet PC with eye tracking device for executive functions diagnostics.
One of very rare studies on play experiences of people with Alzheimer’s Disease (AD; [11]) emphasizes that games should be tailor-made getting suitable for different personalities with AD. Serious games, adapted to people with dementia, may constitute an important tool to maintain autonomy. Obviously, current products lack this flexibility that most obviously is required to adjust to appropriate modes and from this enable truly long lasting motivation and to enable to maintain training at their homes. Executive dysfunction is characteristic in Alzheimer's disease [12], in particular, referring to inhibition abilities and the capacity to co-ordinate simultaneously storage and processing of information. [13,14] found clear evidence for emotion-induced positive enhancement in executive control. The project PLAYTIME2 aims at optimal exploitation of the positive impact of emotion and motivation on executive performance, psychosocial contexts and persistent behavior change through the engagement of people with dementia, to make efficient use of the monitoring of consequences in daily life over long periods of time. The key objective is to increase quality of life of dementia patients but also of caregivers, getting capable to stay active at home for a longer time. 2.2
Playful Training with amicasa
A substantial innovation in the playful training suite of the product amicasa3 is represented via the integrated, multimodal training unit concept. Tasks with the purpose to stimulate cognitive processes are not separated from the tasks that excite physical activities but are in a functional context for the end user. The global task with cognitive, auditive, visual, social and sensomotoric aspects is within the focus and integrated through the serious game framework, i.e., the motivation to gain rewarding and motivating game points through repeated training. amicasa is an interaction platform with an already very high number of task units (200) that is continuously increasing through additional themes that incorporate an exciting database for knowledge and exercises. The game character of the training engages the player and motivates for “knowledge acquisition and physical activities”. The following multimodal unit categories are currently already in the pool of units: Training of memory and remembrance Visual memory Completing gaps in texts Interactive associations (pictures, form, color, content) Search games (visual comparison) Physical activities Playful cognitive test Figure 2 depicts a typical training situation where (a) a formal caregiver assists the PwD in the game, and where (b) a memory game is played in a multi-user configuration, for example, in a residential care-home for the elderly.
2 3
http://www.aal-playtime.eu/ http://www.amicasa.com/
3 3.1
Diagnostics Using Eye Movement Features Impairment of Inhibitory Functionality
Progressive neurological diseases, such as, Alzheimer, Parkinson, Huntington or Wilson, are well known for the decrease in eye movement behavior [15,16]. The characteristics of the impairment support clinicians to localise brain lesions as well as to determine diagnostics about the trajectory of the diseases [15]. Dysfunctionality in the continuous tracking of stimuli was already associated with Alzheimer dementia by [17]. [7] has identified the important indication that Alzheimer patients are characterised with a significant impairment of their inhibitory functionality of eye movements, due to neurodegeneration of fronal and prefrontal lobes which are responsible for inhibitory effects [17]. In early stages of Alzheimer disease, the anti-saccade task is known to identify Alzheimer. This task requires from the test person a voluntary turning away from an actual stimulus and analyses the eye movement behavior further [18]. Figure 3a depicts a test person during the task with the amicasa serious game. Figure 3b depicts the case of position tracking of a test person’s correct anti-saccade behavior (red) for a stimulus to the right (blue), and Figure 3c shows incorrect behavior by following the position of the stimulus instead of turning away.
(a)
(b)
(c) Fig. 3. Cognitive control test for Alzheimer impairment of inhibitory functionality (a)
in which a voluntary turn away from a stimulus is required [19]. (b) Tracking of a test person’s anti-saccade correct behavior (red) for stimulus (blue), (c) Incorrect behavior, i.e., the person follows the stimulus.
Table 1. Features of the anti-saccadic test on inhibitory function of voluntary eye movement control (PDf=person dementia free, PwD = person with dementia), 1-month study (Sec. 4.1)
feature
PDf
PwD
MMSE [1,30]
30.0 0
25.8 4.6
CDT [1,7]
7.0 0
5.5 1.3
Errors [%]
7.7
43.2 20.0
Success [%]
30.7 25
10.8
Anti-sacc. [%]
46.3 20
24.9
Pro-sacc. [%]
19.2
44.4 16.3
correlation ()
-0.632, p=0.09
0.574, p=1.30
The anti-saccade test error is characterised by a large correlation with the MMSE test [19]. However, a direct classification of Alzheimer patients in early stages is not straight forward because the results were extracted from a statistic group comparisons and the individual trajectory can be rather different [19,20]. Further investigations on the ‘Visual Paired Comparison’ (VPC) Test with evaluation of eye movements in a visuo-spatial cognitive control task [21] indicate that the behavior of persons with mild cognitive impairment is significantly different from dementia-free persons. [22] demonstrated that a classifier can be extracted from existing patient data using machine learning techniques with a classification rate of 87%.
3.2 Pervasive Measurement Approach Our approach to the evaluation of the anti-saccadic test procedure is characterized through a pervasive measurement paradigm. The PwD is performing the training and serious game units at home, not in a laboratory environment. Consequently, the input data have to be filtered in order to gain the maximum quality for further processing and evaluation. Usually, a measurement frequency of about 5 Hz was applied as threshold to sort out meaningful data from noise prone data. Various eye movement features were extracted from the data. Areas of interest (AOI) were designed with respect to pro-saccadic and anti-saccadic behavior. Errors were determined from the violation of the anti-saccade condition, i.e., turning attention on the opposite site of the visual stimulus.
4
Experimental Results
The amicasa training suite was configured on a Microsoft Surface Tablet with USBconnected Tobii EyeX mobile tracking technology, providing a 60 Hz sampling of gaze towards the display of the Tablet PC after a 5-point calibration procedure.
Fig. 4. MMSE measured in a 6-months period (Sec. 4.2; 1=month 1, 2=month 3, 3= month 6) for the intervention group using the amicasa training (blue) and for the control group without amicasa training (green).
(a) CDT
(b) MMSE Fig. 5. In a 6-months period study (Sec. 4.2) with 12 participants the error in the anti-saccadic - i.e., generating a kind of pro-saccadic, (‘PSA’) - feature was significantly predictive for the CDT, discriminating PwD (blue; M, S) from PDf (red; M, S).
There were two studies performed to evaluate the pervasive measurement technology.
4.1
1-Month Study
In the first study, 8 participants being clinically classified with Alzheimer disease were using the mobile tracking unit within a period of 4 weeks. After a quality filtering test and sorting out data with less than 5 Hz 60 training unit sessions with the antisaccadic test were applied for data analysis. Table 1 demonstrates that 4 persons with mild dementia performed the study starting with a MMSE of M=25.8, S=4.6 and a Clock Drawing Test of M= 5.5, S=1.3 while the other 4 persons where starting with MMSE = 30 and CDT=7 (dementia-free). The error feature showed a very good Pearson correlation =-0.632 with the MMSE ranking which was reflected by a very discriminative level of the eye movement feature
for PwD (M=43.2, S=20.0) and PDf (M=7.7). Furthermore, both anti-saccadic and pro-saccadic features demonstrated discriminative levels between PwD and PDf conditions in order to enable classification for Alzheimer disease.
4.2
6-Months Study
An overall intervention long-term study was performed over a 6 months period. Figure 4 demonstrates the benefit of using the amicasa training suite, comparing the performance of an intervention group (N=25, predominantly females) that used the amicasa training suite including MAS training in comparison with a control group (N=25, predominantly females) without any specific training. The MMSE was measured in a 6-months period (month 1, 3, 6) for the intervention group that was using the amicasa training (blue) and for the control group without amicasa (green). While the control group performed in a typical Alzheimer decline, the group using amicasa did not decline (M values) substantially. 12 out of 25 participants that were equipped with a Tablet PC (see above) and applied the amicasa training suite used as well the mobile eye tracking units for a period of 6 months. After a data filtering with a removal of any unit training data with a mean data logging frequency larger than 4.5 Hz, and a removal of sessions where center positions were not fixated after a stimulus presentation, eventually 422 training unit sessions remained for analyses. Monthly measurements were aggregated into average measurement values. Eye movement features were extracted and it turned out that the pro-saccadic feature was predictive with respect to CDT and MMSE. The error in the anti-saccadic - i.e., generating a kind of pro-saccadic (‘PSA’) - feature was significantly predictive with M=45.7 (S=12.5) for PwD (CDT=5). Furthermore, the error like pro-saccadic feature PSA was as well predictive with M=43.1 (S=12.4) for PwD (MMSE=26). The long-term study demonstrates the discriminative power of eye movement features in reference to classification of Alzheimer status. Consequently, a support vector machine (SVM) network was tuned with 5-fold cross validation on all available data and achieved a classification accuracy of 75% for MMSE (>26) and 100% for CT (>5).
5
Conclusions and Future Work
We conclude that eye movement features can be successfully applied to provide indicators for Alzheimer diagnostics, considering two independent studies that both showed the discriminative power to classify into dementia and non-dementia participants exclusively from gaze data. Future work will involve larger number of participants in field trials to get more robust and statistically significant estimators for Alzheimer classification. Furthermore, multiple eye movement features will be used for estimation and classification. In addition, multimodal sensing should even lead into better estimates, for example, by incorporating features from movement studies as planned in the PLAYTIME project.
Acknowledgments. The research leading to these results has received funding from the project PLAYTIME of the AAL Programme of the European Union, by the Austrian BMVIT/FFG (No. 857334), ZonMW, the VLAIO, and the Austrian BMVIT/FFG, by project AMIGO.
References 1. Sütterlin, Sabine; Hoßmann, Iris; Klingholz, Reiner; Entwicklung, Berlin-Institut für Bevölkerung und (2011): Demenz-Report, Berlin. 2. Burns, A., and Iliffe, S (2009). Dementia, BMJ (Clinical research ed.) 338. 3. Forbes, Dorothy; et al. (2013a): Exercise programs for people with dementia. In: Cochrane Database Syst Rev 12/CD006489 4. Norton, S.; et al. (2014): Potential for primary prevention of Alzheimer’s disease: an analysis of population-based data. In: Lancet Neurol 2014/13. 5. Graessel, E. et al. (2011): Non-pharmacological, multicomponent group therapy in patients with degenerative dementia: a 12-month randomizied, controlled trial. In: BMC Med 9/129. 6. Korczak, D. et al. (2013): Wirksamkeit von Ergotherapie bei mittlerer bis schwerer Demenz. HTA-Bericht 129, Köln. 7. Crawford, T.J., Higham, S., Renvoize, T., Patel, J., Dale, M., Suriya, A. et al. (2005). Inhibitory control of saccadic eye movements and cognitive impairment in Alzheimer’s Disease. Biological Psychiatry, 57, 1052–1060. 8. McCallum, Simon; Boletsis, Costas. "Dementia Games: A Literature Review of DementiaRelated Serious Games". Serious Games Development and Applications - Lecture Notes in Computer Science 2013; Springer. 9. Kasl-Godley J1, Gatz M. (2000) Psychosocial interventions for individuals with dementia: an integration of theory, therapy, and understanding dementia. 10.J. Cerejeira,1, L. Lagarto, and E. B. Mukaetova-Ladinska, Behavioral and Psychological Symptoms of Dementia, Front Neurol. 2012; 3: 73. 11.Anderiesen, H., et al. (2015). Play experiences for people with Alzheimer’s disease. Int. J. of Design, 9(2), 155-165. 12.Collette, Fabienne et al. (1999). Phonological loop and central executive functioning in Alzheimer's disease. Neuropsychologia. 37. 905-18. 13.P Kanske, SA Kotz. Human brain mapping 32 (2), 198-208, 2011. 14.M. A. Kuskowski, Eye movements in progressive cerebral neurological disease, in: Neuropsychology of Eye Movements, Hillsdale, N.J., C.W. Johnston and F.J. Pirozzolo, editors edition, 1988, p. 146-176. 15.O. B. White, J. A. Saint-Cyr, R. D. Tomlinson, J. A. Sharpe, Ocular motor deficits in Parkinson’s disease, Brain 106 (1983) 571 –587. 16.W. A. Fletcher, J. A. Sharpe, Smooth pursuit dysfunction in Alzheimer’s disease, Neurology 38 (1988). 17.L. D. Kaufman, et al. Antisaccades: A probe into the dorsolateral prefrontal cortex in Alzheimer’s disease. a critical review, .Journal of Alzheimer’s Disease. 18.Simpkins, L. K., Antisaccades: A Probe Into the Dorsolateral Prefrontal Cortex in Alzheimer’s Disease, Master’s Thesis, University of Toronto, 2008. 19.Shafiq-Antonacci, R., Maruff, P., Masters, C., & Currie, J. (2003). Spectrum of saccade system function in Alzheimer disease. Arch Neurol, 60(9), 1272-1278. 20.Crutcher MD, et al. Eye tracking during a visual paired comparison task as a predictor of early dementia. Amer. J. Alzh. Disease & Other Dem. 2009. 21.Lagun, D., Manzanares, C., Zola, S.M., Buffalo, E.A., Agichtein, E. (2011). Detecting cognitive impairments by eye movement analysis using automatic classification algorithms, Journal of Neuroscience Methods, pp. 196-203.
