Understanding Decision Requirements for

Proceedings of the Human Factors and Ergonomics Society 59th Annual Meeting - 2015

160

Understanding Decision Requirements for Selection of Assistive Technology Yao Ding1, Amrish Chourasia1, Denis Anson2, Tony Atkins3, Gregg Vanderheiden1 1

Trace R&D Center, University of Wisconsin-Madison 2 Misericordia University 3 Raising the Floor - International

Abstract. Individual consumers and clinicians have difficulty selecting computer access solutions and keeping up with the advances in access technologies. To design a decision support tool for consumers and clinicians, a thorough understanding of the decision-making process is important. A cognitive tasks analysis was conducted to reveal the underlying cognitive processes and decision requirements, and identify potential decision aids. The analysis used semi-structured interviews for consumers, and clinical observations followed by retrospective interviews for clinicians. Based on the identified decision needs and information needs, design implications are provided. Keywords. decision support system, assistive technology, computer access, cognitive task analysis, decision-centered design

Copyright 2015 Human Factors and Ergonomics Society. DOI 10.1177/1541931215591033

INTRODUCTION Access to information and communications technology (ICT) has become essential for everyone to equally participate in almost all aspects of society. Many people need assistive technology (AT) to achieve full access due to disability, aging, literacy, or computer literacy. Examples of AT for computer access—or computer access technology or access technology/products, interchangeably used in this article—include textto-speech that reads screen content for people who are blind or have low vision, word prediction and completion that makes text input easier for people with learning or physical disabilities, and many others. Most people in need of access solutions cannot get evaluations from professionals such as speechlanguage pathologists or occupational therapists mostly due to lack of funding and lack of adequately trained providers (World Health Organization, 2011). For those where funding is available, most professionals that focus on access technology evaluations are having difficulty keeping up with new advances in this field (Anson & Ding, 2014). Recent years have seen an increasing use of computerized decision support, which has shown great potential to aid decision-making for health care practitioners and populations as diverse as people with disabilities (Chaudhry et al., 2006; Ding & Vanderheiden, 2014; Garg et al., 2005). Several decision aids have been developed specifically for selection of computer access products. The Computer/Electronic Accommodations Program (Department of Defense, 1990) provides an online, self-rated structured questionnaire, asking the user to rate level of functionality in vision, hearing, cognition, communication, and motor skills. Based on reported job requirements, the program makes recommendations on AT solutions. The Computer Access Selector (Stapleton, Garrett, & Seeger, 1997) allows clinicians to use device-based criteria chosen from a series of prompts to identify an appropriate assistive device from a list of known devices. The two systems require some AT knowledge to use, and may not be

directly understandable and usable by individual consumers or non-specialized clinicians. In addition, these systems are stand-alone programs that do not necessarily connect to continuously updated information sources and may not be able to provide useful and reliable information. An open-source, joint collaborative tool (the Shopping/Alerting Aid) is under development as part of the GPII Unified Listing, which uses data federated from multiple databases that cover assistive technologies or access features in mainstream products (EASTIN, AbleData, GARI, etc.) (Anson & Ding, 2014). The new tool will help people with disabilities find out access technology that fits their needs best, and assist practitioners in clinical selections and keeping up with technology advances (Vanderheiden & Treviranus, 2011). Designing a decision aid requires a thorough understanding of its users, tasks to support, and the environment in which it will be used. This paper presents the use of cognitive task analysis and decision-centered design (Hoffman, Crandall, & Shadbolt, 1998; Militello & Klein, 2013) among individuals with disabilities and clinicians to understand the process of AT selection, elicit professional knowledge, identify knowledge gaps and desired information, and discuss potential decision support opportunities. Based on the identified information and decision needs, implications for design are provided. METHOD Consumer Interviews Participants. Six participants were recruited from different states in the U.S. One participant was physically impaired, one was blind and hard of hearing, one was deaf-blind, two were blind, and one had no disability. All the participants with disabilities had experience in selecting AT for themselves without consulting professionals. The participant without disability is a coordinator at a University Disability Resource

Downloaded from pro.sagepub.com at UNIV OF WISCONSIN-MADISON on June 29, 2016


Center and provides academic accommodations for students with access barriers. Procedure. After participants gave informed consent, semi-structured interviews were conducted with each participant in-person, over the phone, or via email, whichever was preferred. Each in-person and phone interview lasted about one hour. Email interviews took several days to weeks, allowing participants to compose response at their own pace. All audio interviews were recorded and transcribed for analysis. Interview questions were centered on strategies that participants used in searching and choosing access solutions, information needs, and expectations for the support tool. Core questions include: 1) What was the source of information about AT? 2) How did you research on the choices? What information was helpful? What were the challenges? 3) Why not see a clinician? 4) What affected your selection? Clinical Observations and Retrospective Interviews Clinical observations were also performed at a local clinic to gather information from a clinical perspective. The clinic provides augmentative and alternative communication (AAC) solutions for people with communication difficulties, and also provides access solutions for computer users. Participants. Evaluations of four clients were observed. Five clinician participants were interviewed (one evaluation was performed by two clinicians). Four clinicians are speechlanguage pathologists and one is an occupational therapist specializing in computer access. Procedure. Clinicians were asked to perform their routine evaluations. After informed consent of both clinicians and clients, the researcher observed and video recorded the evaluation session through a one-way mirror from the observation room. Each session lasted about 1.5 hours. For some clients, more than one visit on different days was needed to reach a final recommendation. Observation of each client’s session(s) was followed by a retrospective interview with the clinician(s) involved in the evaluation. Clinicians were asked to dissect the whole process of selecting AT for the client, detailing information and knowledge used in each step. Major topics included: 1) difficulties in selecting AT 2) what support could be provided, in what way 3) how to learn about new products, keeping up with advances in the AT field The interview questions were cued by replaying video clips of the observation when needed. Each interview lasted about one hour and was audio recorded. Data Analysis Interview and observation recordings were transcribed and coded with NVivo using an AT selection model proposed by Scherer et al. (2007). Themes not covered by this model were identified and integrated to form a new model. All responses across participants were consolidated by theme. Based on the elicited knowledge and reasoning and reported

161

challenges, we identified information needs and decision requirements, which were then transitioned into design concepts and system features. RESULTS Individual Consumers Information sources. Participants obtain AT information from various sources. Among the sources below, 1) to 7) are associated with passive information-seeking behaviors, and 8) to 10) are active information-seeking behaviors. Participants less often search actively, either online or visiting a manufacturer. A participant described this passive searching behavior as “… just took advantage of the opportunities that came my way”. A possible explanation is the lack of an entrance with full access to AT information, as a participant answered why he did not do any research on his first AT, “…because of not being able to do research with out having a working screen reader, I jumped on this”. 1) disability resource centers 2) AT expos 3) friends with similar needs 4) family members 5) user groups, online communities, mailing lists, newsgroups 6) clinicians 7) social media (Twitter, IRC channels) 8) online searching (Google, AbleData, manufacturers’ catalogs) 9) software manuals, training materials, demos 10) visiting facilities to compare devices side by side (hardware) Difficulties in searching. To understand what hinders active searching, participants were asked to describe challenges in seeking AT information they have faced. Most participants had difficulty finding out what might be available that might meet their access needs, which directly leads to hardship in finding the right language for an effective search. A participant found it challenging … to be able to describe the product accurately so that you get at least a right neighborhood to look … I didn't know what to call the thing I use to press the keys. I looked under “adaptive grips”, “adaptive aids”. It turns out this was actually called the “typing aid”, “handicapped or disabled typing aid”. It was almost too simple. I just didn't expect to find it under “typing aid”. Another participant said “I have to probably scratch my head a little longer over that one” when he was asked to find keywords to google his need of mounting a laptop to wheelchair. Among other things, lack of knowledge in the AT area accounts for a large part of the difficulty in searching – either they cannot find a proper term, or the term is so broad (e.g., computer access for blind people) that the search renders too much information to sort through. Selection strategy. Once the participants knew that a category of products might meet their access needs, they used



different strategies to choose one among many alternatives to make a purchase decision. The most often mentioned is “visualizing how a product works”, which could be achieved by reading user reviews and if possible, through hands-on experience. Most participants found access technology foreign to them, or had heard of some but only conceptually knew how to use them. This unfamiliarity is further complicated by the wide range of user needs due to different types, degrees, and combinations of disabilities. Becoming informed about a product requires a comprehensive account of it for a person without much background in AT. The participants reported they needed 1) thorough and accurate descriptions, accompanied with pictures 2) detailed, unbiased user reviews, comparisons 3) demos and tutorials, audio or video demos preferred by some 4) trial versions for extended period of time The participants valued user reviews a fair amount. “Comparing models or different products doing the same thing was useful as well as testimonials of people I felt I knew what they were talking about”, said a participant. But quality of available reviews is not satisfactory for most participants. They noted that: 1) there are no or few side-by-side reviews and/or comparisons written by unbiased professionals 2) reviews may be objective but from users with different needs 3) sales hype is distracting; reviews are not factually based Demos were reported to be beneficial for “learning the basics”, but limited in experimenting with a product thoroughly. Some participants preferred trial versions with relatively extended periods – “the trial software that would allow for one month versus 40 minutes was incredibly helpful … I want to test a range of things I do within a month, not within 40 minutes.” Reasons for not seeking professional help. Participants were asked why they decided not to see a clinician or AT specialist, and instead, chose access products by themselves or with the help of family, friends, or peers from disability communities. Their response confirmed our observation that clinicians have a tendency of overgeneralizing and making snap decisions. One participant stated It's not that I have anything against clinicians. But honestly, for instance, when I went to the Commission in preparation for getting a braille display, I knew a lot more about it than my counselor did. He did say that we could set up an appointment with the technology person, but that person was pretty much used to using and recommending one kind of braille display – the BrailleNote Apex. He wasn't really familiar with the other ones.

162

2) couldn’t get help because the providers are so backlogged 3) think consultants are offering one-size-fits-all approach 4) like to be DIY by nature 5) feel sufficient and confident by reading online, trying a demo, asking a friend, etc. Factors affecting decisions. Participants also listed main factors that affected their purchase decisions. Some participants made quick choices without much research, but with “a slightly rushed excitement”, or “intuition saying ‘go for it’”. Other than that, multiple aspects of a product are considered: 1) Availability/convenience: whether a product is easy to get or needs a lengthy process of waiting 2) Price, funding, or insurance plans: become a factor when there are significant differences in functionality; open source or shareware preferred, if improving much and quickly 3) Functionality: listed by two participants as the most important factor 4) Usability: frequently mentioned, to be specific, “user friendliness”, “usable”, and “easy to use”. Although participants reported that learning takes a lot of time and effort, “easy to learn” or “learnability” were not mentioned. 5) Community support: whether support about a product is available from disability communities or a product’s user community. One participant that switched from Windows to Ubuntu noted that “the extremely helpful community of Vinux (a set of access solutions for blind users on Ubuntu) users made the transition easy and the learning process a pleasure”. 6) Settings: whether it is compatible with preferred operating systems (Windows, Apple, Linux, etc.), compatible with other access products, and whether it is used at home or at work 7) Trial: whether sufficient time is given to experiment with the product 8) Popularity, user reviews and opinions: the overall trends in user opinions on a company are also considered. Design Implications for Consumer Version The interviews revealed decision requirements and information needs for researching and making purchase choices. Some needs can be addressed by providing the desired information or adding more capabilities. Table 1 lists the decision and information needs and corresponding design implications.

Because of the tech-savvy nature of the sample, some participants felt confident in their choices and enjoyed the effort in doing their own research. Reasons for not seeing a clinician or AT specialist included: 1) no such service where they live Downloaded from pro.sagepub.com at UNIV OF WISCONSIN-MADISON on June 29, 2016


Table 1 Consumers' Decision/Information Needs and Design Implications

Decision / Information Needs Search with layperson’s language for access needs and AT products/categories No accessible entrance to the system, need to establish a basic input/output channel without an AT Too much information In-depth user reviews

Objective descriptions Trials and Demos

Prices and funding resources Community support Settings

Design Implications Fuzzy searching (the capability of translating between plain language and AT terms) Q&A type of assessment of access needs Provide basic access features in default (text-to-speech, large font size, high contrast, etc.) and allow the user to turn off unneeded ones; Provide a means to call for assistance when interaction cannot be achieved Provide need vs. feature matrix for easy filtering Provide “save for later” function to allow reviewing saved inquiries or products later Provide consolidated user reviews from multiple trusted sources Provide side-by-side comparisons for multiple products Provide a mechanism for users to easily rate reviews Use impartial information sources that avoid sales hype Allow users to rate the usefulness of product descriptions Provide availability of products, trials and demos in local resource centers and “trying out” programs Offer “try-before-you-buy” by rendering full trials without having to install Provide pricing and funding information based on the user’s health plan Provide links to user communities, such as forums, email lists, social media accounts, etc. Provide compatibility information

Clinicians Decision-Making Process of AT selection. A typical AT selection process at the participating clinic consists of: 1) Introduction. The clinician briefly introduces the clinic, clarifies the scope of services, and gives examples of client needs and possible solutions. 2) Setting goals. The clinician asks about the client’s goals and expectations, discusses with the client and if needed, adjust the goals. 3) Assessment of access needs. The clinician then assesses the client’s functional abilities and limitations, including sensory, motor, cognitive, and speech and language status. 4) Preliminary selections. Based on the client’s abilities and access needs, the clinician is able to narrow down to a few choices. 5) Short-term use/placement. The clinician sets up a product or a set of products, asks the client to try, and observes.

163

6) Short-term evaluation. If there are multiple choices, the clinician and client evaluate each alternative by its usage, price, training needs, user’s preference, etc. 7) Long-term use/placement. When a final choice is reached, the clinician provides more training and customization if needed, to make sure it will be used effectively in the long term. 8) Long-term evaluation. The client’s abilities or access needs may change over time, so re-evaluation is needed. Factors affecting recommendations. These factors are found to affect clinician’s selection strategies and client’s decisions. 1) Insurance and funding. Clinicians spent a fair amount time on explaining what is covered by healthcare plans, what is not; what funding resources are available, and how much will be paid out-ofpocket for each choice. Funding agencies also play a major role in shaping the evaluations. A clinician pointed out, … no reason to write a 20 page report for a 90-minute evaluation … I could summarize that in probably a page and a half … but it won't get approved … a lot of what we do before, during, and after evaluations is [to] figure out how to meet the insurance stipulations. 2) Projected needs. As mentioned above, a client’s needs and abilities may change over time. Clinicians must take into account potential changes in abilities and needs when recommending a solution for longterm use. A clinician remarked, “… once insurance funds something, that's it for five years, and so we need to make sure that it's also going to meet their needs in five years …” 3) Performance on trials. This was reported to be the only way of evaluating and comparing choices – “Just really through trialing them, and just see how they respond.” The importance of trial revealed another issue, that practitioners only recommend products that their clinic or school has, even in cases where they are not the optimal choices. 4) Size and portability 5) Availability of family and caregiver support 6) Clients’ preferences Keeping up with new AT. When asked what knowledge was used to match the client’s needs to products, a clinician said “that's just where it comes in handy to be knowledgeable about what's out there. There's no formula”. However, becoming familiar with everything in the market was deemed as the most difficult part. How do the clinicians keep up with new technologies that become available? First, they do research on their own time. Second, AT vendors and companies send or visit to show new products to the clinic, and/or loan to the clinic. Third, when clinicians are aware of what is out there, even if they do not have it, they would request it if needed.



164

Design Implications for Clinician Version The observations and retrospective interviews revealed clinicians’ underlying cognitive processes and knowledge used in recommending access solutions. For non-specialized clinicians, this selection process should be supported providing desired information and decision cues. Table 2 lists clinicians’ decision and information needs and corresponding design implications. When clinicians use the system for their own research, they can be deemed as individual consumers and thus the design implications in Table 1 also apply.

ACKNOWLEDGEMENT The contents of this paper were developed with funding from the National Institute on Disability, Independent Living, and Rehabilitation Research, Administration for Community Living (ACL), Dept of Health & Human Services (HHS), grant # H133E080022. However the contents do not necessarily represent the policy of the Department of Education, and you should not assume endorsement by the Federal Government.

Table 2 Clinicians' Decision/Information Needs and Design Implications

Decision/Information Needs Insurance and funding Projected needs

Keep up with new technology Matching access needs to solutions

Trials and Demos

REFERENCES

Design Implications Provide information on possible insurance or funding coverage Based on a client’s profile, provide possible access needs to consider that reflect potential changes in functional abilities Alert clinicians to new products that become available Provide need-product mapping for recommendation support, and productneed-product mapping to help clinicians explore new products with similar functionalities For clinics that house few products, provide trials

DISCUSSION A cognitive task analysis was performed to understand the decision process of selecting AT. Decision requirements and information needs by consumers and clinicians were identified and implications for system design were discussed. The results will be used to generate system specifications and guide system design. These findings may also inform design of other information systems for people with disabilities. Some findings need further research attention. For example, the consumer’s version requires an entrance point that is accessible without having to use any AT. We need to examine what access features should be included to maximize initial usage without overwhelming the user. Features like fuzzy search and potential needs require accumulated usage and user profiles. As the number of system users grows, future research may focus on algorithms on identification of needs patterns and suggestions to consider. When the system enters testing phase, its effectiveness will be evaluated in real-world settings by consumers and clinicians. Data collection is still ongoing. Clinical observations will be extended to other parts of the country to include a more diverse sample of professionals such as clinicians who are not expert in computer access but recommend it when needed. Consumer interviews may also investigate those who attempted to find products for computer access but failed and explore the reasons.

Anson, D., & Ding, Y. (2014). Federated Databases and Supported Decision Making. In C. Stephanidis & M. Antona (Eds.), Universal Access in Human-Computer Interaction. Design for All and Accessibility Practice (Vol. 8561, pp. 337-347). Crete, Greece: Springer. Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., … Shekelle, P. G. (2006). Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine, 144(10), 742–752. Department of Defense. (1990). Computer/Electronic Accommodations Program (CAP). Retrieved March 8, 2012, from http://www.cap.mil/ Ding, Y., & Vanderheiden, G. C. (2014). Computerized Decision Support for People with Disabilities: Review and Outlook. Journal on Technology & Persons with Disabilities, 2, 134– 143. Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., … Haynes, R. B. (2005). Effects of computerized clinical decision support systems on practitioner performance and patient outcomes. JAMA: The Journal of the American Medical Association, 293(10), 1223– 1238. Hoffman, R. R., Crandall, B., & Shadbolt, N. (1998). Use of the critical decision method to elicit expert knowledge: A case study in the methodology of cognitive task analysis. Human Factors: The Journal of the Human Factors and Ergonomics Society, 40(2), 254–276. Militello, L. G., & Klein, G. (2013). Decision-centered design. The Oxford Handbook of Cognitive Engineering, 261–271. Scherer, M., Jutai, J., Fuhrer, M., Demers, L., & Deruyter, F. (2007). A framework for modelling the selection of assistive technology devices (ATDs). Disability & Rehabilitation: Assistive Technology, 2(1), 1–8. Stapleton, D., Garrett, R., & Seeger, B. (1997). VOCASelect and Computer Access Selector: Software tools to assist in choosing assistive technology. In Let’s Tango - Partnering People and Technology (Vol. 17, pp. 351–356). Pittsburgh, Pennsylvania: RESNA Press. Vanderheiden, G., & Treviranus, J. (2011). Creating a global public inclusive infrastructure (pp. 517–526). Springer. World Health Organization. (2011). World report on disability. Geneva, Switzerland: World Health Organization.
