Lessons Learnt from Deploying an End-User Development Platform for Physical Rehabilitation 1
Daniel Tetteroo1, Paul Vreugdenhil1, Ivor Grisel1, Marc Michielsen2, Els Kuppens3, Diana Vanmulken4, Panos Markopoulos1
2 3 4 Eindhoven University of Revalidatiecampus St. MS-Reva Overpelt Adelante Centre of Expertise Technology Ursula Overpelt, Belgium in Rehabilitation Eindhoven, The Netherlands Herk-de-Stad, Belgium
[email protected] Hoensbroek, The Netherlands {d.tetteroo, p.e.vreugdenhil,
[email protected] [email protected], p.markopoulos}@tue.nl zorggroep.nl
ABSTRACT
Clinical researchers in rehabilitation technology have often called for exercise customization to address patient specific needs. Where such customization transcends simple parameter setting, the need for End-User Development (EUD) arises. EUD in this field can potentially tap on the expertise of highly skilled workers, but presents serious challenges regarding acceptance by end users and the feasibility of embedding EUD in their professional practice. This paper describes the deployment and adoption process of TagTrainer, a physical rehabilitation technology that supports EUD. TagTrainer was deployed in four rehabilitation clinics and was used by 24 rehabilitation therapists. We analyze how they engaged in EUD activities and we discuss decisions that we took in the design and deployment of TagTrainer. Based on these case studies, we present guidelines for the deployment of EUD systems. Author Keywords
End-user development; physical rehabilitation; field study; socio-technical environment; meta-design ACM Classification Keywords
H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous; J.3 Life and medical sciences: Health INTRODUCTION
End-user development (EUD) is a socio-technical paradigm focused on enabling end-users of interactive systems to engage in the modification, extension, and even creation of interactive artifacts [20]. The aspiration for this field is that EUD shall bridge the divide between design-time and use-time by extending the evolution of interactive systems beyond the initial software development, allowing continuous adaptation of these systems to evolving and Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected]. CHI 2015, April 18 - 23 2015, Seoul, Republic of Korea Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM 978-1-4503-3145-6/15/04…$15.00 http://dx.doi.org/10.1145/2702123.2702504
context specific end-user needs [21]. Although the evolution of end-users’ needs is a phenomenon occurring in almost any domain, certain domains have been identified as especially suitable for the application of EUD practices [10]. One of these domains is healthcare, were the greatly varying needs of individual patients present a design problem of a scale unmanageable with a traditional technology design process. The specific domain that we address in this paper is that of physical rehabilitation after stroke, multiple sclerosis (MS), and spinal-cord injury (SCI). Persons who have been diagnosed with stroke, MS or SCI often suffer from cognitive and/or motoric function loss. In order to regain some of the lost functionality, they usually engage in a physical rehabilitation program as soon as possible. Previous research points at the importance of patient customized exercises and sufficient exercise variability in physical rehabilitation [7,32]. Patients might have their own, personal goals for rehabilitation (e.g., relearning to eat with knife and fork, relearning to play the guitar, etc.) and will vary in the extent to which their cognitive and motoric functions are affected. Customization can be supported at multiple levels. To start with, therapists typically do not rely on a singular device or exercise approach for a patient, but combine one or more of these to address the patient’s condition and training needs. Increasingly over the last decade, a task-oriented training approach is adopted in which patients train on specific daily skills of their choice, that can provide them benefits in terms of their ability to function independently and to integrate socially [31]. For technologies that are integrated in such a training program, the level of challenge should ideally be adapted to the patients’ needs [1], or even adjusted automatically as patients progress through a game [18]. Such parameter adaptations pertaining to range of movements, or the speed and accuracy required, do not fully address the pragmatic concerns of patients relating to the very content and nature of exercises. Being able to change the content, purpose, and difficulty of exercises to match patient interests requires a higher level of customizability that can better be thought of as EUD. Designing EUD solutions for professional use represents several challenges. An important one for the given domain is
to enable novice programming [30]: in the domain of rehabilitation, the vast majority of professionals do not have any affinity with programming. To overcome this challenge, the usability of systems has to be ensured, so that programming presents a low threshold and allows them to easily map concepts from the domain of their professional expertise to that of the programmable environment. Beyond making programming easier, there are further challenges regarding the feasibility of adopting EUD as part of their professional practice, pertaining to motivation, peer and organizational support, culture, and the effect on the productivity of health professionals. While earlier research has identified the importance of such a broader perspective (e.g., [11]), there is as yet little empirical research to document the issues at play, to show the feasibility of EUD as a professional practice in a clinical setting, and to guide future design efforts. Therefore, we present a multi-site deployment study of TagTrainer: an EUD system to support upper extremities training for patients suffering from stroke, SCI or MS. TagTrainer (see [29]) supports arm-hand rehabilitation through the manipulation of physical objects. It allows therapists to author exercise content, and tailor exercises to the needs of their individual patients. The overarching aim of the four deployment studies presented in this paper was to identify the factors that influence therapists’ decisions to engage in EUD practices. In the next sections we first summarize related work on introducing EUD in different professional contexts. We then present TagTrainer, and detail its deployment in four clinical settings. Finally, we discuss the overall lessons learnt and provide pointers for future EUD deployments. RELATED WORK
Many researchers have studied how end-users can be enabled to engage in EUD (e.g., [4,19]), and have studied EUD practices as they occur (e.g., [15], [27]). Blackwell introduced the Attention Investment model [3], that describes from a cognitive psychology perspective the suitability of EUD practices. He claims that the decision to engage in EUD depends on the result of a cost-benefit analysis involving costs for doing a task manually, EUD investment costs, EUD payoff and risks associated to the EUD process. While this model is unspecific regarding how to quantify attention investment and so far little empirical evidence has been presented to support it, it presents a useful starting point for the design and evaluation of EUD systems. Further, others such as Mehandjiev (e.g., [22]) have developed substantial knowledge on the organizational aspects of EUD. Meta-design [11] looks at EUD from a socio-technical perspective, putting the end-user developer on a spectrum with on one end the end-user, and on the other end the meta-designer of an extensible socio-technical system. Rather than treating the end-user developer as a particular species of its own, in meta-design theory users may assume and switch between multiple roles on this
Figure 1: A TagTrainer system with three TagTile boards connected to a laptop running the TTPI software.
spectrum, depending on the context. In this theory, end-user development may become a shared effort amongst users that occupy different positions on the developer-spectrum. Although the before mentioned research provides us with valuable insights on factors that influence the EUD process, they mostly study EUD in a lab setting, or in situations in which EUD is already practiced, and therefore do not help understand what is required for EUD technology to be adopted in the first place. There are just a few studies that report on the deployment of EUD environments and study their adoption. For example, [6] studied the adoption of a system that allows caretakers to create prompting scripts to enhance the independence of cognitively disabled persons. Amongst others, they confirm the importance of a holistic view on EUD – thus considering technology as well as context. Another study reports on the deployment of a platform that allows business users to create custom widgets [27]. The authors conclude that, for the case of small and medium enterprises, a properly led deployment can lead to a self-sustainable EUD environment were users take on different roles, from ‘widget consumers’ and ‘widget creators’, to ‘service creators’ that are similar to what Fischer et al. call ‘meta-designers’ [11]. TAGTRAINER
TagTrainer is a tangible, extensible, interactive training platform for arm-hand rehabilitation therapy [29], consisting of: 1) TagTile boards: interactive boards (24cm x 24cm) that can give visual and audio feedback, and are able to detect RFID-tagged objects (see Figure 1). The boards provide feedback through full-color LEDs, as well as the possibility to playback audio. 2) The TagTrainer Patient Interface (TTPI), which manages personalized exercise programs (i.e. series of exercises) and provides feedback to patients about their progress. 3) The TagTrainer Exercise Creator (TEC): a visual programming tool that supports authoring and modification of exercises to be executed on the board (see Figure 2). 4) An extensible and customizable collection of objects with RFID-tags attached to them.
Figure 2: The TagTrainer Exercise Creator (TEC) software. The center area (A) shows the workspace with the exercise, featuring a timeline with actions associated with the objects (‘measuring cup’ and ‘cup’) involved in the exercise. Properties of the selected action (in this case ‘lift object’) such as position on the board are displayed to the right (B). Finally, additional actions can be dragged from the library (C) into the workspace to extend an exercise. Note that beyond actions involving manipulations on the board, other actions such as giving instructions (D) and pausing (E) can be used.
TagTrainer was developed in response to a request for patient-tailorable rehabilitation technology by rehabilitation clinics. Although TagTrainer is by default provided with a base set of about 150 exercises, the TEC allows therapists to create custom exercises that match the skill level and rehabilitation goals of individual patients. Exercises for arm-hand training involve physical objects; the therapist creates a simple model of an exercise using TEC, as a sequence of object movements on target areas on the board and repetitions thereof. The board recognizes movements such as placing, moving, removing or rotating objects on the board. In principle, any number of objects can be used within an exercise and an unlimited amount of actions can be added to each of the objects. In addition to the object-related actions, instructions, pauses and sound can be added to an exercise (see Figure 2–D, E). TagTrainer is designed to present a low-threshold for new users while still accommodating for the creation of relevant Location
and engaging training content. Following a usability evaluation of an early version of the system [17], complex control structures are not supported and only sequential, stepwise execution of actions is possible using a timeline based scripting - similar to what can be found in animation software. For example, actions scheduled to occur in step 2 of an exercise can only happen after the first step has been completed. Repetitions of (parts of) an exercise can be programmed to increase training intensity. Users are further aided by an automated validation tool that notifies them about errors or inconsistencies in their creations. Errors by patients in the execution of an exercise (e.g. using an incorrect object) are handled automatically through default routines (such as repeating the instructions for the current step) and need not be addressed by the creator of an exercise. METHODS
A field study consisting of four cases was carried out to evaluate whether and how therapists can act as end-user
Length Participants
Pathology
Notes
Case 1 Hoensbroek, Netherlands Case 2 Herk-de-Stad, Belgium
3w
SCI, stroke
Case 3 Overpelt, Belgium Case 4 Hoensbroek, Netherlands
5w
Research team had participated in the design and development of TagTrainer [29]. Aim: to assess EUD practices in a different health system and organizational context. We focused on the rehabilitation of stroke patients. Aim: to study a different patient population and organizational context. Follow up to case 1. Aim: to assess the spontaneous and unguided use of the system, post initial deployment.
8w
Stroke Aged 25-32 MS, stroke Aged 24-54
8w
SCI, stroke Aged 26-44
Table 1: Characteristics of the four case studies ( / = physiotherapist, / = occupational therapist).
Case 1 Case 2 Case 3 Case 4 Week (relative)
S+T S+T S+T S+T
S+T
S+T+CE S+T
S+T+CE
S+T
S+T S+T
1
2
3
4
5
6
7
S+T+CE 8
Figure 3: Administration of self-efficacy (S), UTAUT (T) and credibility/expectancy (CE) questionnaires.
developers of training content, in the context of their daily work. The case studies lasted for three to eight weeks, and took place in The Netherlands and Belgium, see Table 1. An important motivation for the fourth study was that the simultaneous introduction of a new technological artefact and EUD practices in the first three cases made it hard to distinguish to what extent the observed practices related to the introduction of EUD, versus the use of a new training technology. Therefore, a follow-up study was performed at the clinic of the first case study, to see whether prolonged use of TagTrainer would cause a change in EUD practices. Participating professionals
Therapists working in the clinics could participate on a voluntary basis after agreement with their management. In total 24 health professionals participated, see Table 1. Participating therapists had no previous experience with TagTrainer, except for two participants in Case 4, who had also participated in Case 1. The participants’ computer experience was limited to using general office software, email, internet and online social networks. Materials
In the first three cases, which studied initial deployment, two systems per clinic were deployed consisting of one TagTile board connected to a laptop and an initial set of about 150 exercises that were ready to be used. For case 1 this was in two different treatment rooms, while in the 2nd and 3rd case the boards were available in a space in between treatment rooms. The exercise set was iteratively improved between deployments. Importantly, beyond the initial set of exercises, the therapists in the latter three cases were also provided with the (unaltered) exercises created by therapists in earlier studies. For the 4th case, upon request of the therapists, three boards were combined into a single system, providing the therapists with greater flexibility in the design of therapy exercises. This system was placed on a table with wheels such that it could be moved between the two treatment rooms in which the participating therapists worked. Procedures
The procedures for the four studies varied slightly, gradually reducing the involvement of the non-clinical researchers. In the first three case studies, therapists received at the start of the study a two-hour long introduction to the TagTrainer system that included simple hands-on practice. One week after the start of these studies, therapists received additional training for the creation and modification of therapy exercises. Therapists were free to decide whether and when they would use TagTrainer during their therapy sessions.
In the first case study the designer and developer of TagTrainer was on site, holding a scheduled weekly information session, observing the use of TagTrainer and conducting fixes and iterative design of the system as this was necessitated in practice. In the 2nd and 3rd case study a researcher was present one to three times weekly for trouble shooting, observations, and situated interviews. In the fourth case, a researcher was only present for initial training, data collection, and remote technological assistance. Measures
In their stepwise guidance to facilitate successful implementation of technology in therapy, [17] argue that after the initial phases, during which awareness and insight about the new technology are spread amongst therapists, a phase of acceptance follows. During this phase, therapists’ attitude, motivation, and willingness to change are crucial factors in the success of the implementation process. Since we were interested in factors that influence therapists’ decisions to engage in EUD practices, the focus of the research efforts has been on measuring factors that predict technology acceptance and on collecting in-depth information about the practices, events, and opinions that underlie the therapists’ decisions. Logging – We logged the use of TagTrainer (storing time of use, exercises used and the therapist using TagTrainer) and the creation and modification of exercises, including the time of creation/modification, the content created and the author. Self-efficacy – The degree to which therapists felt able to create exercises with the program was measured using a questionnaire based on [2], which measures self-efficacy on thirteen scales, ranging from 0 to 100. We took measurements at three different moments, see Figure 3. Technology acceptance – Technology acceptance refers to the intent of individuals to use a particular technology, and to a number of determinant factors. These were measured using UTAUT questionnaire [33] was administered three times during the studies, see Figure 3. Credibility/expectancy – A credibility/expectancy questionnaire [9] was administered at the end of each study that measures the degree to which therapists believe that TagTrainer is credible as a therapy aid and that they expect it will help patients recover. Observations – A researcher was present for at least one day a week at every clinic to observe TagTrainer use by participating therapists. Field notes and photographic records were kept as a reference.
Semi-structured interviews were conducted with individual therapists, as well as in groups. These interviews discussed therapists’ general experience of using TagTrainer, as well as their experiences with modifying and creating exercises. Depending on their availability and level of involvement in the study, up to four interviews per therapist were conducted. RESULTS Quantitative results
Table 2 shows the most important statistics on TagTrainer usage and exercise creation / modification. Table 3 shows the results from the questionnaires. The UTAUT scores for effort expectancy, facilitating conditions and behavioral intent were slightly above average for all cases. Social influence and performance expectancy were slightly negative in all cases. Overall, little change can be observed over the course of each case. Only during the second case, behavioral intent dropped significantly as was shown by a Wilkoxon signed-rank test (Z=-2.041, p
