accessibility of Web sites and applications that are meant for the general public, created with public funds, or in- tended to be used in a professional or training ...
APPLIED RESEARCH
SUMMARY • Argues that user-focused testing is essential to find usability problems of users who are elderly or have disabilities • Offers practical guideiines and tips for thinkingaloud studies with those users
Conducting Usability Studies with Users Who Are Elderly or Have Disabilities THEA VAN DER GEEST
INTRODUaiON
I
n the past few years, I have been involved in a number of Web evaluation studies whose participants were users who are disabled or elderly. Many of these users have embraced the Web as a way to get easier access to information, services, contacts with others, and entertainment. But very often, Web sites are creating barriers for them, at the same time as they are reducing barriers of access. We can safely assume that Web sites are made by designers who have no intention to exclude groups of users from using the site. Our studie.s, however, have proved that good intentions are not enough to create Web content that is accessible and also usable for people with various kinds of physical, sensory, or cognitive disabilities. Web designers need to familiarize themselves with accessibility guidelines and apply them properly, but that goal is seldom achieved. Even when the guidelines are applied, a site s ease of use for people with disabilities needs to be confirmed in a usability test with actual users who have disabilities. This article describes how to conduct user-focused tests with test participants who are elderly or who have disabilities. Using guidelines In many countries, laws or recommendations require the accessibility of Web sites and applications that are meant for the general public, created with public funds, or intended to be used in a professional or training context. Most of these laws demand compliance with the Web Content Accessibility Guidelines (WCAG), priority level 1 (World Wide Web Consortium 1999) or equivalent sets of guidelines. Expert review of the Web sites, rather than usability studies with users with disabilities, is the sole
method used for determining whether Web sites meet the legal accessibility requirements. The WCAG and other accessibility guidelines are based on the assumption that users with disabilities will access Web content either by adapting their browsers (for example, enlarging font size) or by using assistive technologies (for example, a screen reader that "reads out loud" the content of the site and displays the text in Braille). When the design principles in the guidelines are ignored, people with disabilities may not be able to access the content of the Web site, or they may be able to do so only with great difficulty. Accessibility guidelines ensure that users with disabilities can actually get to the content of the site. In the working draft for an updated version of the WCAG guidelines, the four basic principles of accessibility are stated as follows (World Wide Web Consortium 2004). 1. Content must be perceivable. For example, someone with hearing deficiencies must find a visual representation of the audio information, such as captions or a transcript for the sound track of a movie. 2i Interface elements must be operable. For example, people who do not have much strength in their hands or fingers must have the opportunity to open menus as long as needed by disabling any time-out options. 3 . Controls and content must be understandable. For example, terms and phrases that are in a language other than the primary language of the document are explained. In the source code of the pages, these words are
Manuscript received 25 April 2005; revised 25 Augu.st 2005; accepted 27 August 2005. V'olimif 33, Number I. Febman 2006 • TechnifalCONMUACATION
23
APPUED RESEARCH Conducting Usabiiity Studies with Users Who Are Eideriy or Have Disabilities
fan der Geest
also marked with a language attribute, which enables screen reader software to pronounce them properly. 4. Content must be robust enough to work with current and future technologies. For example, the content must be accessible with a particular screen reader, even if that program is not yet adapted to the newest Web browser technologies.
setting 1. 2. 3. 4.
up and conducting the test: Decisions you have to make beforehand Preparations for the actual test The test session Reporting the test findings
Conducting usability tests with disabled users Principles 1, 2, and -4 locus on giving all users access to the content, while principle 3 also takes the usability of the content into account. However, our studies and those of others have shown that even when .sites comply with the guideiines, they are not necessarily effective, easy to use, and pleasant to u.se for people with disabilities (Disabilities Rights Commission 2004; Gahrmann 2004; te Velthuis 2005; Theofanos and Redish 2003, 2005; van der Geest, 2004). To assess whether sites cause usability problems for users with disabilities, one must conduct user-focu.sed tests, with those u.sers as test participants. This article contains gLiiclelines and hints tor conducting usability tests with users with disabilities: physical, sensory, or cognitive limitations that influence their use of Web sites and content. The guideiines are based on a protocol for user-focused Web evaluation developed for the Dutch certification agency for accessible information and communication technologies, the Dutch national Accessibility Bartimeus Founchition (van der Geest 2003)- The protocol is based on current best practices of usability testing, as described for example in Barnum (2002), Krug (2000). Paciello (2000), Rosson and Carroll (2002), Rubin (199'4), and the Society for Technical Communication Web site. In addition, the guidelines presented here reflect our experiences in testing Web site usability applying the test protocol with various groups of users who are disabled or elderly.
The number of people with disabilities in the general population is considerable and will increase in the next years. The following figures might help you convince your client or boss that it is worth designing the site "for all" and testing it with elderly and disabled people. About 15% of the working-age (16-64) pt)pLilation of the European Union is hampered in their daily activities by physical or mental health problems, illness, or disability. In the various countries, about 5% of the population is registered as handicapped or receives benefits on the basis of their disability. Older age often coincides with (more) physical and cognitive limitations, such as becoming hard of hearing, seeing colors less well, or having (mtirc) difficulty mcmcjrizing new material. By age 65, the percentage of people with illnesses and disabilities rises to 30%: by age 80, to about 50"/(i. The percentages for U.S. inhabitants are very similar (European FoLindation 2003; Lippincott 2004; United Nations Statistics Division 2005).
ISSUES TO CONSIDER BEFORE THE TEST Issue 1: Deciding to test
In all of tiiir projects, the thinking-aloud method in combination with observation was used as one of the means to collect data. In thinking-aloud sessions, test participants execute a number of tasks with a Web site while verbalizing their actions and thoughts. The method is relatively easy to apply and yields a wealth of information on what users like or dislike about sites, and when, where, and why they experience problems in using the site. The guidelines in this article are based on the assumption that the reader is planning to apply this particular method. It is also assumed that the test is executed to find problems with the site and then improve it, and not for benchmarking or effect measurement of the final site. We approach usability' testing with users as a necessary step in the process ot creating accessible, usable, and attractive sites f(ir all users, including those with special needs. The guideiines are presented following four stages in 2 4 Tcchnic:ilCOMMjraCA'T1ON • \'olLmi
