Evaluation of Guidelines for Designing Accessible Web Content

INTERACT'99 Workshop."Making Designers Aware of Existing Guidelines for Accessibility

Position paper for the IFIP TC.13 INTERACT'99 Workshop: MakingDesignersAware of Existing Guidelinesfor Accessibility(31 August 1999)

Evaluation of Guidelines for Designing Accessible Web Content Chetz Colwell

& Helen Petrie

Sensory Disabilities Research Unit, Department of Psychology, University of Hertfordshire, Hatfield, ALl0 9AB, UK e-mail: [email protected], h.l.oetrief~herts.ac.uk Abstract This study evaluates the Web Content Accessibility (WCA) Guidelines, developed by the Web Accessibility Initiative (WAI). The first aim of this study was to investigate whether the Guidelines can be used effectively by Web page authors. To this end students of HTML (HyperText Mark-up Language) used the Guidelines to adapt Web pages. The second aim of the study was to examine whether pages developed using the Guidelines are accessible to visually impaired people. In order to achieve this visually impaired Web users assessed the accessibility of the pages developed by the Web page authors. The study found some difficulties in the use of the Guidelines that were similar to those identified by previous research. There were unexpected results regarding the accessibility of certain HTML elements.

Introduction The WCA Guidelines [6] are available on the Web and have been developed by the WAI, which is a program of the of the World Wide Web Consortium (W3C). Two other online documents are associated with the Guidelines, a Techniques document that contains examples of HTML code, and a Checklist, which can be used to evaluate Web pages. There are hypertext links between the three documents and to other documents on the Web, such as the HTML 4.0 specification.

Evaluation of guidelines Four aspects of the WCA Guidelines were identified which required evaluation to discover whether the Guidelines can be used effectively and efficiently. The current study investigated whether: (i) the organisation and presentation allows the discovery of the required advice; (ii) the content provides sufficient and relevant advice; (iii) the relevant advice can be identified, and applied appropriately; and whether (iv) the interface to the guidelines is usable. The current study is set within the context of previous investigations of the use of guidelines. Such research has shown that there are limitations in their use, for example, in their selection [4]; in their practical application [3], and in their interpretation [1]. This study has found that these limitations also apply to the WCA Guidelines.

1 Chetz Colwell is a PhD student supported by the Economic and Social Research Council, UK, Grant No: S00429637041. 11

INTERACT'99 Workshop ."Making Designers Aware of Existing Guidelinesfor Accessibility

Experiment 1: adaptation of pages The use of the Guidelines was investigated using a group of students who were learning HTMLo The 12 participants all had experience of creating Web pages, which ranged from having made a few personal pages to having been employed to create larger sites. Prior to the experiment the participants were asked to read the Guidelines in order to familiarise themselves with the document. The task for each participant was to adapt an existing Web page, making it accessible by following the advice contained in the Guidelines. The following HTML elements were adapted by the participants: images, imagemaps, forms, frames, and tables. Each participant was given an HTML file that contained the element(s) they were to work with. Participants attended the laboratory where they were observed while performing the task. The experimental method used was an adaptation of the 'think aloud protocol' [2]. This involved the participant telling the researcher what they were doing while they were performing the task, and describing any problems they encountered. Video recordings were made of the dialogue between the researcher and the participants, and of the computer screen at which the participants worked. The observations and the recordings provided useful insights into the experiences of participants, the problems they encountered, and the methods they employed.

Results From both observations of participants and their reports several aspects of the Guidelines were identified to which improvements could be made: their general structure and tone; navigation within and between the documents; the content and presentation of examples; and additional information to be provided. These are similar problems to those identified by previous research. For example, many participants had difficulties in locating the information they required within the Guidelines. They also experienced other navigational problems, for example they were not always clear whether they were reading the Guidelines or the Techniques documents. An important implication of these navigational problems was that some participants missed important information regarding the implementation of the advice and general accessibility issues.

Experiment 2: evaluation of pages The pages developed in the first experiment were collated and made available on a Web site [5] which was evaluated by 20 visually impaired Web users. The evaluation was performed remotely, i.e. the participants did not attend the laboratory, but accessed the site from home or work, at a time convenient to them. Participants were asked to view several examples of each HTML element, such as images, frames, tables etc. They were then asked to state which example they found to be more accessible, and why. in addition, the participants were asked to interrogate each table in order to find the answer to a specific question. Participants submitted their answers to the questionnaire via email. Participants chose which elements they wished to evaluate, so not all the elements were evaluated by all the participants. The participants used a wide range of browsers and screenreaders to evaluate the Web pages. The browsers included mainstream browsers such as Internet Explorer and Lynx as well as specially designed browsers, such as WebSpeak. The participants had an average of 2.7 years' experience of using the Web, ranging from a few months to 10 years.

]2

INTERACT'99Workshop: Making Designers Aware of Existing Guidelinesfor Accessibility

Results The most interesting finding from this second experiment concerned the images. Although the page authors had given the images alternative text, this was not available to six of the fifteen visually impaired participants who evaluated the images. The accessibility seems to be related to the browser used and the participants' experience of using the Web, rather than to the content of the alternative text. There is an apparent assumption in the Guidelines that assistive technology will automatically present alternative text to the visually impaired user. However, this finding suggests that this may not be a reasonable assumption. The page authors involved in experiment 1 implemented various elements and attributes suggested by the Guidelines. In the tables IDs and Headers were used, in the forms Fieldset, Legend and Label were used, and for the frames, Noframes were used. Although the page authors followed the examples provided in the Techniques document, they each implemented the advice in a different way. The browsers used by the visually impaired participants do not yet support some of these elements. The participants were therefore unable to comment on them. Despite the lack of browser support for the accessibility techniques for tables, all but three of the participants were able to access them. This is in surprising contrast to the inaccessibility of the images. The page authors also made other design decisions, such as changing the layout of the tables and of the frames. In some ways these decisions had a greater impact on the accessibility than the use of some elements intended to improve accessibility. This is of concern, because the use of such elements should have improved accessibility, and because the page authors spent time learning about and implementing these 'special' elements.

Conclusions Experiment 1 evaluated the use of the Guidelines in the adaptation of Web pages. A number of problems with the Guidelines were identified, some of which were similar to those found in previous research. For example, navigational difficulties meant that important information was missed. The findings of experiment 1 were reported to the WAI WCA Guidelines Working Group. The suggestions were discussed and some changes were subsequently made to the Guidelines. In experiment 2 visually impaired Web users evaluated the accessibility of the pages developed in experiment 1. There were some unexpected results for example, tables seemed to be more accessible than expected and the alternative text for images less accessible. Some design decisions made by the page authors that were not based on the Guidelines seemed to increase accessibility more than some of the accessibility improvements, few of which were supported by the participants' browsers. The findings of experiment 2 will also be reported to the WCA Guidelines Working Group for discussion.

References [1] de Souza, F. & Bevan, N. (1990) Use of Guidelines in menu interface design: evaluation of a draft standard. In D. Diaper et. al. (Ed.), Proceedings of the Human Computer Interaction conference (INTERACT '90), (pp. 435-440). Elsevier. [2] Dix, A., Finlay, J., Abowd, G. & Beale, R. (1993) Human-Computer Interaction. Hemel Hempstead: Prentice Hall. [3] Mosier, J. N. & Smith, S. L. (1986) Application of guidelines for designing user interface software. Behaviour and information technology, 5(1), 39-46. [4] Newman, W. M. & Lamming, M. G. (1995) Interactive System Design. Wokingham: Addison Wesley. [5] Test site used in experiment 2 (June, 1999) http:tlwww.psy.herts.ac.uk/sdrultestsite [6] Web Content Accessibility Guidelines 1.0 (May, 1999) http:I/www.w3.orglTRIWAI-WEBCONTENTI 13