1
Focus Theme – Editorial
Methodologies, Models and Algorithms for Patients Rehabilitation REHAB
H. M. Fardoun; A. S. Mashat Faculty of Computing and Information Technology, King Abdulaziz University (KAU), Jeddah, Saudi Arabia
Keywords Virtual rehabilitation, multi-user systems for user interaction, virtual humans, communications aids, training tools for rehabilitation, rehabilitation robotics
Summary Introduction: This editorial is part of the Focus Theme of Methods of Information in Medicine on “Methodologies, Models and Algorithms for Patients Rehabilitation”. Objective: The objective of this focus theme is to present current solutions by means of technologies and human factors related to the use of Information and Communication Technologies (ICT) for improving patient rehabilitation.
Correspondence to: Habib M. Fardoun Information Systems Department Faculty of Computing and information Technology King Abdulaziz University (KAU) Jeddah Kingdom of Saudi Arabia E-mail:
[email protected]
1. Introduction Data and information are essential to meet our fundamental needs to decide, move around and partake in social, financial and cultural exercises [1]. Information and Communication Technology (ICT) is obviously an empowering innovation and we should not dismiss this. Obtaining better health and wellbeing through ICT helps to make current research and innovation turn the future of health into the present [2]. What have the best brains in rehabilitation research come up with to improve health and wellbeing with the help of ICT? This is a common question, which is always correct to ask. ICT is in continuous © Schattauer 2016
Methods: The focus theme examines distinctive measurements of strengthening methodologies, models and algorithms for disabled people in terms of rehabilitation and health care, and to explore the extent to which ICT is a useful tool in this process. Results: The focus theme records a set of solutions for ICT systems developed to improve the rehabilitation process of disabled people and to help them in carrying out their daily life. Conclusions: The development and subsequent setting up of computers for the patients’ rehabilitation process is of continuous interest and growth.
Methods Inf Med 2016; 55: ■–■ http://dx.doi.org/10.3414/ME14-11-0001 epub ahead of print: ■■■
use to improve openings and, at same time, to develop a number of possibilities to improve rehabilitation environments and health care solutions through the development of innovative frameworks that use ICT in the preparation and training of people with diverse sorts of disabilities, through the use of Virtual Reality (VR) as a necessary piece of a rehabilitation programme [3, 4]. With the approach of VR, new trusts are developing for disabled people. In spite of the colossal difficulties, earnest endeavours are being attempted to actualize the utilization of VR to counter the snags which are identified with incapacity, offering significant new opportunities for disabled indiMethods Inf Med 1/2016
REHAB
2
Editorial
viduals who use assistive technology for their daily activities in all aspects of their social life on more equal terms than ever before. In addition, VR has been increasingly used in physical rehabilitation to provide individualized treatments, to facilitate learning, increase motivation, and provide patients with pleasant and enjoyable exercises in the home. This can be done, for example, by applying the use of the Nintendo Wii Balance Board, low-cost visual biofeedback systems, and the implementation of programs for improving the patient’s balance [5]. The focus theme contains a set of research articles that focus on technologies and human factors related to the use of ICT for improving patient rehabilitation. It contains research papers in conjunction with case studies, perspectives on rehabilitation and rehabilitation in practice.
2. Selected Papers Seven papers were selected, after a peer review process, as the best papers presented at the REHAB-Workshop [6] and were included in the focus theme “Methodologies, Models and Algorithms for Patients Rehabilitation”.
2.1 Self-assessment of Rheumatoid Arthritis Disease Activity Using a Smartphone Application: Development and 3-month Feasibility Study Nishiguchi et al. presented a study that investigated the feasibility of a self-assessment system for rheumatoid arthritis (RA) using a smartphone application [7]. They measured the daily disease activity over nine RA patients for a period of three months, using the Disease Activity Score (DAS28) predictive model [8]. Subsequently, the authors carried out a questionnaire survey to investigate participants’ opinions of the disease activity monitoring system, showing favourable responses to the list of questions. The authors were able to demonstrate that the predictive model was able to play an important role for patients with RA in Methods Inf Med 1/2016
self-diagnosis and self-management of their disease activity because it offers a set of benefits, like: • The application does not include invasive measurements like blood testing, but daily measurements (joint symptoms and gait parameters) that can be made using a smartphone. • The disease activity can be represented objectively by the gait measurements in the predictive model. • The medical staff and patients may be able to share information regarding the patient’s condition at home in real time using the functions of the smartphone. As a consequence, patients can receive timely advice from their medical providers and seek interventions before acute exacerbation of symptoms. The authors showed that the proposed study indicated that the applied predictive model can longitudinally predict DAS28 and may be an acceptable and useful tool for the assessment of RA disease activity for both patients and healthcare providers.
2.2 Interaction Detection with Depth Sensing and Body Tracking Cameras in Physical Rehabilitation The research work of Omelina et al. [9] proposed a camera-based method for identifying the patient and detecting interactions between the patient and the therapist during therapy. In order to do this, the authors proposed the use of motion capture systems combined with serious games (SG) for physical rehabilitation as a measuring tool or to provide feedback to the patient and the therapist [10 –13]. The authors recognized some limitation of such systems, like the occurrence of more than two people in front of the camera who were interacting with a player(s) lead to problems, which were not clearly separated. The presence of more than two people lead to an unstable user selection for skeletal tracking, especially when they were interacting. At the same time, the authors proposed two approaches to handle these problems: • Creating constraints – creating a set of rules on how to use the SG system in order to avoid situations that may cause problems, e.g., people should not stand
•
closer than a specific distance. This solution significantly decreases the usability and ergonomics of the system since it limits the ways of using the systems and users need to learn the additional rules on how to use it. Improving system intelligence – additional processing that can identify and track the right person and can identify whether the people in the scene are interacting or not, e.g., the system can notify people about problematic poses. This may increase the computational complexity, but it does not decrease the usability and ergonomics.
Merging both solutions, the authors proposed a simple, fast and robust approach for detecting human interactions in the RGB-D (colour + depth) image stream and increased system intelligence, in order to improve the serious gaming experience in therapeutic practice, using a method based on local binary patterns (LBP). As a result, the proposed work shows state-of-the-art performance of real-time face recognition using low-resolution images that are sufficient to use in adaptive systems. The proposed approach for detecting interactions showed 91.9% overall recognition accuracy, which is sufficient for applications in the context of serious games. In the present study, the authors focused on the detection of interactions during the gameplay within physical therapy. The proposed research work clearly shows how it helps to discriminate between active and passive motion. It showed that during an interaction, the precision of the capturing devices decreased and, thus, a good detection for these interactions can help to filter out erroneous measurements.
2.3 Eye Gaze Correlates of Motor Impairment in VR Observation of Motor Actions Júlio et al. [14] proposed the use of eyetracking technology to quantify the recovery process, by exploiting the neural correlates of eye gaze trajectories in the observation and execution of goal directed actions in a virtual reality display. The authors attempted to show through this research that mirror neurons may serve rec© Schattauer 2016
Editorial
2.4 Quantitative Evaluation of Performance during Robotassisted Treatment The work of Peri et al. [19] describes a parameter that can be easily derived from data saved by the robot and that quantifies the subjects’ performance. For this, the authors conducted training sessions, with fourteen in-patients between 8 and 16 years, who were affected by cerebral palsy, using paediatric Armeo Spring. All the information produced during the training sessions was automatically recorded and included in a comprehensive performance parameter to calculate the overall performance. As a result, the authors discovered that the effect of the training measured by the performance parameter was different for each training session but, at the same time, an overall improving trend was obtained. This work proposes a parameter as a useful tool to follow subjects’ performance during the training and, thus, to provide meaningful information with minimal additional effort for both patients and clinicians. The results confirmed that variations in terms of this parameter are coherent © Schattauer 2016
with functional clinical improvements, thus giving information on functional recovery. The authors believe that this parameter may be used, not only to plan a rehabilitative therapy, but also to customize and update the strategy patient by patient.
2.5 Virtual Reality to Assess and Treat Lower Extremity Disorders in Post-stroke Patients The research work of Luque-Moreno et al. [20] identified the support of a virtual reality system in the kinematic assessment and physiotherapy approach to gait disorders in individuals with stroke, by adapting a Virtual Reality Rehabilitation System for use on the lower limb of hemiplegic patients. The authors described how many research works were widely presented [21], and how virtual reality rehabilitation, which includes the use of computer-based programs that simulate real-life events [22], was deficit in gait limited functionality in individuals who suffered a stroke, but at the same time, not all of them were able to evaluate analytical kinematic movements of the joints [23]. The authors applied this intervention for two participants, 58 and 49 years old, consecutively, and the results obtained by using a VR-based system with the software of the Virtual Reality Rehabilitation System (VRRS), coupled to a motion tracking in the assessment and treatment of motor arm deficiency after stroke, were satisfactory [24 –26]. The given results of the applied case study showed that the proposed Virtual Reality Rehabilitation System can be very useful to assess bodily harm, because of the possibility that it brings to evaluate specific movements of the lower limbs; for instance, to determine the degree of functional disability of a patient and to compare it with data from healthy subjects. But also, it would be very interesting to correlate the data with that obtained by other motion analysis systems used for gait assessment, such as the Davis protocol. In addition, it would be necessary to develop specific sensors for certain important parameters to get a better assessment of the balance and the lower extremity function [22], such as the measurement of the centre of pressure during standing. These measures would
improve the evaluation of technical aids, like exoskeletons, to assist patients during motion.
2.6 Evaluation of Cognitive Functions through the Systemic Lisbon Battery: Normative Data In their research, Gamito et al. identified normative data for several instrumental activities of daily life in a virtual reality scenario [27]. This research work presents a standardized cognitive assessment measure, to define an indicative normative for performance on daily-life tasks measuring cognitive functionality, displayed and performed on a virtual reality platform [28, 29], which provides cognitive stimulation for patients with cognitive impairments. During this process, the authors also analysed the effects of socio-demographic variables and video-game experiences on cognitive performance in these tasks [30], in order to identify variables that could have an additional negative effect on performance independently of cognitive impairment. For the test of this methodology, 59 participants were recruited from the general population and asked to participate in a study designed to evaluate cognitive abilities. The virtual reality platform consisted of a small town scenario populated with non-playable characters. As a result, the authors found no effects of gender and also found no effects of education. This is intriguing, because education is a usual confounder of performance in traditional neuropsychological tests. However, this result may also be due to the low variability of education in our sample. However, they did find generalized effects of video-game experience, which is unsurprising, as well as effects of age. As technology evolves, the authors believe that they will be able to develop ever more realistic scenarios, for they expect that the effect of the video-game experience will be reduced. On the other hand, the independent effect of age could be simply an effect of general familiarity with digital media, or, alternatively, an effect of general performance speed, which requires further research. The results of Gamito et al., suggest that the tool used can be a useful to assess cogMethods Inf Med 1/2016
REHAB
ognition and imitation of goal-directed actions, and that these neurons are crucial to social interactions, providing the basis for understanding the actions of others through the connection between action and perception [15 –18]. In the current presented research work, the authors took advantage of the assumed shared neural mechanisms in action observation and execution to explore their potential in rehabilitation. The authors proposed a novel technology that assesses eye gaze behaviour in a virtual reality (VR) observation task and demonstrated its use in healthy subjects, as well as in stroke patients, suggesting important implications for diagnostic and rehabilitation purposes. However, the main limitation of this approach is the widespread neural basis of the motor action observation loop and the currently limited understanding of the specific role of the mirror neuron system in it. Thus, lesions of the mirror neuron system may affect multiple aspects of motor execution and observation.
3
REHAB
4
Editorial
nitive functioning during the execution of activities of daily living. However, a larger study comparing normal with clinical samples (i.e., with acquired cognitive impairment) and evaluating comparative performance between the results of other traditional neuropsychological tests is still needed.
2.7 ACL Reconstruction Decision Support through Personalized Simulation of the Lachman Test and Custom Activities The research work of Stanev et al. [31] presents the development of a realistic knee model, as a whole of a full 3D multi-body model, comprised of 10 ligaments, along with a tibiofemoral contact. Currently, there are three alternative modelling approaches of interest: finite element analysis [32], multi-rigid body analysis, or their combination [33]. While the research of Stanev et al. considered patient specific and pre-surgical objective evaluations, aiming to provide a feasibility study on how simulated virtual physiological humans could help towards the aforementioned grand challenge. In the scope of this study, different parameters are presented for patient specific parameterization, while the model can simulate different tasks. The proposed scheme demonstrates how virtual physiological human simulations can be used in a pre-surgical step for optimal planning of several parameters related to the surgical procedure, and how the effect of the chosen parameters on the motor behaviour of the knee can be estimated through the proposed simulation scheme, thus leading to a powerful clinical decision support. Even if the potential use of such a framework is evident, there are several issues that still need to be considered and a clinical validation of the simulation results is still necessary. Moreover, it is still important to allow interaction of the existing knee models with soft tissues (i.e., menisci).
3. Conclusions The development and subsequent setting up of computers for the patients’ rehabiliMethods Inf Med 1/2016
tation process is of continuous interest and growth. It has already been demonstrated that using ICT involves a greater sense of empowerment and an improvement in the quality of life of disabled patients, allowing the development of new methods to evaluate behavioural and functional deficits more effectively. This focus theme discussed seven research works that presented varied solutions for the use of ICT for the patients’ rehabilitation process, showing how the use of ICT has resolved or improved their daily life. Nishiguchi et al. [7] investigated the feasibility of a self-assessment system for rheumatoid arthritis patients using a Smartphone application. Alternatively, the paper of Omelina et al. [9] presented a camera-based method for identifying the patient and detecting interactions between the patient and the therapist during therapy, which helps to discriminate between active and passive motion of the patient, as well as to estimate the accuracy of the skeletal data. Alves et al. [14] suggested that neural motor circuits are involved, at multiple levels, in the observation of motor actions displayed in a virtual reality environment in a study with healthy participants and stroke patients. Peri et al. [19] proposed a parameter to assess the performance of subjects during upper limb robotic training, exploiting data automatically recorded by the robot, with no additional effort for patients and clinicians. Subsequently, Luque-Moreno et al. [20] proposed the identification of the support of a virtual reality system in the kinematic assessment and physiotherapy approach to gait disorders in individuals who had suffered strokes. Later, Gamito et al. [27] identified normative data for several instrumental activities of daily life tasks in a virtual reality scenario that are available on the Systemic Lisbon Battery. Finally, the last paper by Stanev et al. [31], proposed a scheme which provides a feasibility study on how virtual physiological human simulations can be used in a pre-surgical step for optimal planning of several parameters related to the surgical procedure and could help towards the aforementioned grand challenge. These studies prove the effectiveness of employing ICT in the patients’ rehabili-
tation processes and it is important to chart the field and to open the dialogue between the different professionals working within it (e.g., researchers, clinicians and engineers). Acknowledgments
We want to thank the editors of Methods of Information in Medicine for supporting us by publishing this focus theme.
References 1. Dudeck MA, Horan TC, Peterson KD, AllenBridson K, Morrell G, Anttila A, et al. National Healthcare Safety Network report, data summary for 2011, device-associated module. Am J Infect Control 2013; 41 (4): 286 –300. 2. Haux R, Hein A, Kolb G, Künemund H, Eichelberg M, Appell JE, et al. Information and communication technologies for promoting and sustaining quality of life, health and self-sufficiency in ageing societies – outcomes of the Lower Saxony Research Network Design of Environments for Ageing (GAL). Inform Health Soc Care 2014; 39 (3 – 4): 166 –187. 3. Fardoun HM, Mashat AS, Lange B. New Methodologies for Patients Rehabilitation. Methods Inf Med 2015; 54 (2): 111–113. 4. Laver K, George S, Thomas S, Deutsch JE, Crotty M. Cochrane review: virtual reality for stroke rehabilitation. European journal of physical and rehabilitation medicine 2002; 48 (3): 523 – 530. 5. Bobillier Chaumon ME, Michel C, Tarpin Bernard F, Croisile B. Can ICT improve the quality of life of elderly adults living in residential home care units? From actual impacts to hidden artefacts. Behaviour & Information Technology 2014; 33 (6): 574 –590. 6. Fardoun HM. ICTs for improving Patient Rehabilitation Research Techniques Workshop (REHAB) in conjunction with PervasiveHealth Conference, Oldenburg, Germany, May 20, 2014. Link: http://www.rehab-workshop.org (Last view: 10/11/2015). 7. Nishiguchi S, Ito H, Yamada M, Yoshitomi H, Furu M, Ito T, et al. Self-assessment of Rheumatoid Arthritis Disease Activity Using a Smartphone Application. Development and 3-month Feasibility Study. Methods Inf Med 2016; 55 (1): ■–■. 8. Prevoo ML, van ’t Hof MA, Kuper HH, van Leeuwen MA, van de Putte LB, van Riel PL. Modified disease activity scores that include twentyeight-joint counts. Development and validation in a prospective longitudinal study of patients with rheumatoid arthritis. Arthritis Rheum 1995; 38 (1): 44 – 48. 9. Omelina L, Jansen B, Bonnechère B, Oravec M, Pavlovicova J, Van Sint Jan S. Interaction detection with depth sensing and body tracking cameras in physical rehabilitation. Methods Inf Med 2016; 55 (1): ■–■.
© Schattauer 2016
10. Chang CY, Lange B, Zhang M, Koenig S, Requejo P, Somboon N, et al. Towards pervasive physical rehabilitation using Microsoft Kinect. 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth); 2012. pp 159–162. 11. Chang YJ, Chen SF, Huang JD. A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Res Dev Disabil 2011; 32 (6): 2566 –2570. 12. Clark RA, Pua YH, Bryant AL, Hunt MA. Validity of the Microsoft Kinect for providing lateral trunk lean feedback during gait retraining. Gait Posture [Internet]. 2013; 38 (4): 1064 –1066. Available from: http://www.sciencedirect.com/science/ article/pii/S0966636213001884 13. Van den Audenaeren L, Vanden Abeele V, Geurts L, Husson J, Annema J-H, Desmet S. Boneshaker. Proceedings of the 2012 ACM annual conference extended abstracts on Human Factors in Computing Systems Extended Abstracts – CHI EA ’12 [Internet]. New York, New York, USA: ACM Press; 2012 [cited Oct 23, 2014 ]. p. 2447. Available from: http://dl.acm.org/citation.cfm?id=2212776. 2223817 14. Alves J, Vourvopoulos A, Bernardino A, Bermúdez i Badia S. Eye Gaze Correlates of Motor Impairment in VR Observation of Motor Actions. Methods Inf Med. 2016; 55 (1): ■–■. 15. Buccino G, Binkofski F, Riggio L. The mirror neuron system and action recognition. Brain and language 2004; 89 (2); 370 –376. 16. Oztop E, Arbib MA. Schema design and implementation of the grasp-related mirror neuron system. Biological cybernetics 2002; 87 (2); 116 –140.
© Schattauer 2016
17. Bonaiuto J, Rosta E, Arbib MA. Extending the mirror neuron system model, I. Biological cybernetics 2007; 96 (1): 9 –38. 18. Oztop E, Kawato M, Arbib MA. Mirror neurons: functions, mechanisms and models. Neuroscience letters 2013; 540: 43 –55. 19. Peri E, Biffi E, Maghini C, Servodio Iammarrone F, Gagliardi C, Germiniasi C, et al. Quantitative evaluation of performance during robot-assisted treatment. Methods Inf Med 2016; 55 (1): ■–■. 20. Luque-Moreno C, Oliva-Pascual-Vaca A, Kiper P, Rodriguez-Blanco C, Agostini M, Turolla A. Virtual reality to assess and treat lower extremity disorders in post-stroke patients. Methods Inf Med 2016; 55 (1): ■–■. 21. Riva G. Applications of virtual environments in medicine. Methods Inf Med 2003; 42: 524 –534. 22. Laver K, George S, Thomas S, et al. Cochrane review: virtual reality for stroke rehabilitation. Eur J Phys Rehabil Med 2012; 48: 1– 2. 23. Mirelman A, Patritti BL, Bonato P, et al. Effects of virtual reality training on gait biomechanics of individuals post-stroke. Gait Posture 2010; 31: 433– 437. 24. Turolla A, Dam M, Ventura L, et al. Virtual reality for the rehabilitation of the upper limb motor function after stroke: a prospective controlled trial. J Neuroeng Rehabil 2013; 10: 85. 25. Piron L, Turolla A, Agostini M, et al. Assessment and treatment of the upper limb by means of virtual reality in post-stroke patients. Stud Health Technol Inform 2009; 145: 55– 62. 26. Kiper P, Agostini M, Luque-Moreno C, Tonin P, Turolla A. Reinforced feedback in virtual environment for rehabilitation of upper extremity dys-
27.
28.
29.
30.
31.
32. 33.
function after stroke: preliminary data from a randomized controlled trial. Biomed Res Int 2014; 2014: 752128. Gamito P, Oliveira J, Brito R, Lopes P, Rodelo L, Pinto L, Morais D. Evaluation of cognitive functions through the Systemic Lisbon Battery: normative data. Methods Inf Med 2016; 55 (1): ■–■. Chaytor N, Schmitter-Edgecombe M. The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills. Neuropsychological Review 2003; 13 (4): 181–197. Spooner D, Pachana N. Ecological validity in neuropsychological assessment: A case for greater consideration in research with neurologically intact populations. Archives of Clinical Neuropsychology 2006; 21: 327– 337. Parsons T. Neuropsychological Assessment Using Virtual Environments: Enhanced Assessment Technology for Improved Ecological Validity. In: Braham S, Jain LC (eds). Advanced Computational Intelligence Paradigms in Healthcare 6; 271–289. Berlin Heidelberg: Springer-Verlag; 2011. Stanev D, Moustakas K, Gliatis J, Koutsojannis C. ACL Reconstruction Decision Support Through Personalized Simulation of the Lachman Test and Custom Activities. Methods Inf Med 2016; 55 (1): ■–■. Weiss J, Gardiner J. Computational Modeling of Ligament Mechanics. Critical Reviews™ in Biomedical Engineering 2001; 29 (3): 303–371. Bei Y, Fregly BJ. Multibody dynamic simulation of knee contact mechanics. Medical Engineering & Physics 2004; 26 (9): 777–789.
Methods Inf Med 1/2016
5
REHAB
Editorial
