© The Human Communication and Interaction Research Group The original paper that should be used for all citations can be found on www.hci.uniovi.es. Cite as: Schrepp, M. & Fischer, P. (2007). GOMS models to evaluate the efficiency of keyboard navigation in web units. Eminds – International Journal of Human Computer Interaction Vol. I, No. 2, 33-46.
GOMS Models to Evaluate the Efficiency of Keyboard Navigation in Web Units Martin Schrepp1, Patrick Fischer2 1
SAP AG Dietmar-Hopp-Allee 16 69190 Walldorf Germany
[email protected] 2
University of Mannheim Germany
[email protected]
Abstract One of the basic accessibility requirements for web units is that they must be usable by keyboard. But it is not sufficient that a user can perform all actions using the keyboard. In addition a user must be enabled to perform these actions in an acceptable time. We present a criterion which can be used to describe what an acceptable amount of keyboard support means. This criterion is based on a simple comparison of the performance a user can reach with mouse and keyboard navigation. The actual performance estimations for a web unit are based on GOMS models. These GOMS models allow a quantitative comparison of the efficiency of keyboard and mouse navigation. This comparison enables us to decide if the amount of keyboard support for a web unit is sufficient or if there is an unacceptable disadvantage for keyboard users.
1
Introduction
One of the main accessibility requirements for web pages is that they can be handled by a keyboard. A visitor to the page must be able to reach all interactive elements on the page (for example, links or fields in online forms) by keyboard actions. In addition he or she must be able to perform all possible actions on the page (for example, follow a link, fill out a field, press a button) using the keyboard. This basic requirement is, for example, covered by the WCAG 1.0 [1] Guideline 9 Design for device independence, the WCAG 2.0 Draft [2] Guideline 2.1 Make all functionality operable via a keyboard interface or Section 508 [3] Checkpoint §1194.21a When software is designed to run on a system that has a keyboard, product functions shall be executable from a keyboard where the function itself or performing a function can be discerned textually. Interactive elements of web pages are usually accessible by keyboard over the TAB-chain. Thus, all interactive elements of the page are organized into a virtual chain. The user can set the cursor focus to the next respectively previous element on this virtual chain by pressing the tabulator key (short TAB) respectively the key combination SHIFT + TAB. This standard method to make web pages accessible over keyboard is problematic if a web page contains a huge number of interactive elements, since the user has to press TAB very often to reach the required element (see [4] or [5]). For example, the entry pages of many popular web sites, like online journals or news pages, contain often several hundred links. In such cases it is necessary to add accelerators to provide a reasonable level of keyboard support, for example access keys. To guarantee a certain degree of usability for users who navigate using the keyboard it is not sufficient that all active page elements are available over keyboard. In addition it must be guaranteed that keyboard
users are able to navigate in a web page with sufficient performance. The design concept of Design for All1 tries to address this problem (see, for example, [6], [7]). The basic idea of this concept is to create user interface designs which as many people as possible can use either directly or with the help of assistive technology. Web sites designed under that concept must guarantee a certain degree of usability for all user groups, including disabled users. This is especially important for the design of intranets and web applications [4]. Users of such applications use them often on a daily basis. They must therefore be able to work efficiently. It is, for example, not acceptable that a task, which can be solved using the mouse in 1 minute, requires 10 minutes if the user is restricted to use the keyboard. If we fail to provide an efficient keyboard support we exclude disabled persons from working with the intranet or web application, even if it is accessible accordingly to the common accessibility guidelines. The accessibility of intranets or web applications is important for the workplace integration of disabled people (see, for example, [8], [9]). Computerized workplaces offer handicapped persons often a good chance for a job which is not in conflict with their disability. Web applications in particular have a high potential concerning workplace integration, since they allow their users to access all relevant information over the web. Handicapped users can profit from such web applications, since they allow them to work fully or partly from home. But this requires that the design of web applications takes the special needs of disabled users into account. We use in the following the term web unit2 to refer to web pages, pages in intranets and web applications. The basic problem we address in this article applies equally to all types of web units. The basic question we want to answer in this paper is how we can decide if keyboard support for a web unit is sufficient. We have to decide if users are able to navigate in a web unit using the keyboard with sufficient performance or if the design of the web unit puts an unacceptable burden to keyboard users. A natural criterion to decide if keyboard navigation is sufficiently efficient is to compare it to mouse navigation. Let k be the time required by an experienced user to finish a task using the keyboard and m be the time necessary to finish the same task using mouse and keyboard. We can define that keyboard support is sufficient if k < cm. The value c is a constant which depends on the usage scenario. To define reasonable values for c we have to take usage scenario specific factors into account. For example, how frequent a user performs the task per day, how important the fast completion of the task is for the user, if the working environment of the user limits the time he or she has to complete the task, etc. Thus, it is not possible to define a scenario independent method to obtain reasonable values for c. For example, for a news page on the web obviously a higher value is acceptable for c than for an intranet of a company or a web application used for professional work. An adequate keyboard support is not only important for disabled persons. The needs of disabled persons concerning keyboard support are similar to the needs of expert users. Such expert users often prefer to handle an application completely over the keyboard (see, for example, [10], [11]), since working with the keyboard is much faster than working with the mouse. Thus, improving the accessibility of a software product by providing an efficient keyboard support will also improve the usability of the product [8].
2
GOMS models
A common problem in user interface design is to compare different design alternatives concerning their efficiency. Such a comparison can be done with user tests, but such tests usually require a huge effort for recruiting users and to conduct the test. A cheap alternative are quantitative methods for the analysis of user interface designs. One of the best known techniques for quantitative user interface analysis is the GOMS model (see [12], [13], [14]). The acronym GOMS stands for Goals, Operators, Methods and Selection rules which are the basic building blocks of the analysis. A GOMS model allows to predict how long an experienced user needs to perform a given task in a given interface design. We describe in the following the simplest form of the GOMS model, which is often referred to as keystroke level model. Please note that there are several more sophisticated variants of the GOMS model
This concept is also often referred as Universal Design or Barrier free Design. The term web unit is defined in the WCAG 2.0 Draft [2] as a collection of information, consisting of one or more resources, intended to be rendered together, and identified by a single Uniform Resource Identifier (such as URLs). 1 2
available, for example the Critical-Path Method GOMS [16] or the Natural GOMS Language (NGOMSL) [14]. For an overview of the different GOMS models see [13]. The central idea of the GOMS model is that the time necessary to perform a certain task is the sum of the times of the elementary actions (pressing a button, moving the mouse cursor to a certain place at the screen, clicking on a link, etc.) required to finish the task. Different users will need different times for these elementary actions. But for a comparative analysis of screen designs it is sufficient to use a set of typical or average times for the elementary actions. These average times are determined in laboratory experiments (see, for example, [13], [15]). Examples for such average times used in a GOMS analysis are: – time it takes to tap a key on the keyboard: 0.2 seconds – time it takes to point with the mouse to a screen position: 1.1 seconds Our goal is to compare the efficiency of mouse and keyboard navigation in web sites. Given the arguments above it is evident that a GOMS analysis is a tool well-suited for this purpose.
3
GOMS models for web unit navigation
To be able to compare mouse and keyboard navigation in web units we have to formulate adequate GOMS models for these two navigation methods. In general, we assume that the task of a user is to detect and follow a link on a given web unit. In principle this task can be split into three sub-tasks: 1. locate the target link on the web unit, 2. select the target link by moving the mouse respectively keyboard focus to the link, 3. confirm that the selected link is the target link and follow this link. The first sub-task is independent of the used navigation method, i.e. the time required for this sub-task should be the same for users who navigate using the mouse and for user who navigate using the keyboard. The time required for the other two sub-tasks obviously depends on the navigation method, i.e. different times for mouse and keyboard navigation must be expected.
3.1
A GOMS model for mouse navigation
If the user navigates using the mouse the three sub-tasks of the navigation task can be described as: 1. locate the target link on the page, 2. moving the mouse focus to the target link, 3. click on the target link. Let t1 be the time required to locate the target link, t2 be the time required to move the mouse focus to the target link, and t3 be the time required to click on the target link. Thus, the total time to perform this task can be computed as t = t 1 + t 2 + t 3. The time t3 results in fact from two components. The first component is the cognitive decision time required to verify if the link to which the mouse points is the correct target. The second component is the manual execution time required for the mouse click. A good estimate for t2 (1.1 seconds) is already published (see, [13] or [15]). Estimates for t1 and t3 will be derived later in this paper.
The time to position the mouse pointer to a target obviously depends on the distance of the pointer to the target and on the size of the target (this is called Fitt’s law, see [19] or [20]). The estimate of 1.1 seconds is an average over a representative sample of different distances and target sizes.
3.2
A GOMS model for keyboard navigation
If a user navigates using the keyboard and if we assume that the target link is the n’th link in the TAB-chain, then the navigation task can be described as: 1. locate the target link on the page, 2. press n-times the TAB key to move the cursor focus to the link, 3. follow the target link by pressing the ENTER key. An estimation for the time necessary to press a key on the keyboard (0.2 seconds) is already available [15]. But this estimation can not be used in our case. This results from the fact that the cognitive task of the user requires that he or she controls the current cursor position while pressing the TAB key repeatedly. We can assume that the process which controls the cursor position interferes with the process of pressing the TAB key. Thus, we can expect that the time required to press the TAB key is higher than the above mentioned estimate. In addition the user may loose the current cursor position completely and needs to detect it again. We assume in our GOMS model for keyboard navigation that the total time to complete the navigation task is given by: t(n) = t1 + nt4 + npt5 + t6. Here t1 is again the time required to locate the target link, t4 is the time required to press the TAB key, p is the probability to loose the cursor focus during navigation, t5 is the time required to locate the lost cursor focus again, and t6 is the time required to follow the target link by pressing the ENTER key. Obviously t(n) depends on the position of the target link in the TAB-chain, i.e. on the number n. Again we have to assume that t6 consists of two components. The first component is the cognitive decision time required to decide if the link which currently has the cursor focus is the correct target link. The second component is the manual execution time required to press the ENTER key. The GOMS model for keyboard navigation described above is an enhancement of a much simpler GOMS model for keyboard navigation already described in [5]. The main enhancements are that this model takes the time to locate the target into account and considers the situation that the subject can loose the focus during navigation. We will see later that these two factors account for a huge amount of the total time required to solve the navigation task. Our GOMS model is adequate to describe the keyboard interaction of sighted users. It can not be used to describe the interaction of blind users. For blind users we have to consider the fact that they scan the page linearly over their screen reader or Braille display. Thus, the process to locate the target link and to follow this link is different for blind users. GOMS models which describe the interaction of blind users with web units are described in [17].
4
Estimation of the model parameters
We performed a study to estimate the parameters of our GOMS models. The task of the participants was to navigate to a link on a web page using only the keyboard. The mouse was not available for the participants during the task. It was not necessary to repeat the same study with users working with the mouse, since the required parameter estimations for our GOMS model for mouse navigation either exists or can be derived from the keyboard study.
4.1
Participants
10 persons participated in the study. Their age ranged from 25 to 43 years. All participants were experienced computer users. They were not paid for their participation in the study.
4.2
Material
The time required detecting a target link on a page and the time to find a lost focus again on the page obviously depends on the number of links on the page. In addition the chance to loose the focus during keyboard navigation depends on the number of TAB presses required to reach the target, the complexity of the page layout, and on the visibility of the current cursor position in the page. To get reliable measures for the parameters of our GOMS model it is thus necessary to use a variety of web pages which show a realistic difference concerning these factors. We picked 15 common German web pages for the study. This sample of pages contained, for example, pages from web shops, information pages of companies or public organizations, web pages of television channels, etc. The number of links on the selected pages varied between 38 and 200 (mean 89, standard deviation 46.63). For each page a target link for the navigation task was identified. The number of TAB presses necessary to reach the target link varied between 14 and 119 (mean 57.28, standard deviation 34.42).
4.3
Procedure
Each participant was welcomed and placed in front of a monitor. The instruction for the participant was visible on the monitor in form of an HTML page. The participant was instructed to read this instruction carefully and to clarify open questions with the experimenter before he or she starts the experiment. The participant was instructed that he or she has the task to navigate in web pages using only the keyboard. Navigation over the TAB key respectively the key combination SHIFT + TAB was explained in detail. Each participant had to solve the navigation task for 10 of the selected web pages. The pages which are assigned to the participant and their sequence were determined randomly per participant. The instruction page contained direct links to the 10 pages assigned to the participant. These links were embedded in a short text which named the target link on the test page. The participant could start the experiment directly from the instruction page by following these links. When the participant followed such a link, the corresponding web page was loaded and the participant could start to solve the navigation task for this page. Measurement was done in the background by a tool which was installed locally on the test computer. This tool captured all keyboard events of the participant and the corresponding system time. Thus, it was possible to calculate the time necessary for each keyboard action by a difference of the captured system times of two successive keyboard actions. After the participant has clicked on the target link in a test page, he or she was instructed to close the browser window. The participant was then automatically redirected to the instruction page. He or she could then start the next page in the sequence again over a link in the instruction page. Figure 1 shows the screen flow in the study.
4.4
Data analysis
Each participant had to solve the navigation task for 10 of the selected web pages. The first two pages processed by a participant were considered as trial pages. These trial pages should allow the participants to get familiar with the TAB navigation. They are therefore not included in the analysis. The other 8 pages are used for an estimation of the parameters of the GOMS model. Since each of the 10 participants solved the navigation task for 8 different web pages (the 2 trial pages are excluded) we got 80 different observations. Five of these observations must be excluded from the analysis because of technical
Figure 1: Screen flow in the study. problems with the used web pages during the experiment. Thus, the estimation of the parameters for the GOMS model is based on 75 different observations. The participants were instructed to avoid pressing keys until they have located the target link on the page. Thus, the parameter t1 can simply be estimated as the average time difference between the loading of the page and the first press of the TAB key. Since the manual execution time for pressing the key is still included in this estimation we additionally subtracted 0.2 seconds (usual estimation for the time required to press a key on the keyboard). The time t6 to execute a link over the keyboard can be estimated directly as the average time between the last press on the TAB key (which sets the focus on the target link) and the press on the ENTER key. Since the manual execution times to press a key on the keyboard and to press a mouse button are usually identical (0.2 seconds) ([13] or [15]), we can use the estimate for t6 also as an estimate for t3. We calculated per participant all differences between the captured TAB events. These differences were ordered accordingly to their size and analysed using the Nalimov test. This statistical test is designed to detect outliers in the data. A value is called an outlier here, if the hypothesis that it belongs to the population can be rejected with a risk of error smaller than a defined probability α (we used α = 0.1 in our study). The detected outliers were identified as focus losses. Their average is thus the estimate for t5 and their frequency is the estimate for p. The average of all other times is the estimate for t4. The detected number of outliers per subject differs between 2 and 22 in our study (mean 14, standard deviation 7.12).
Figure 2: Example of the measured times of the keyboard events for one participant in one of the navigation tasks. Figure 2 shows the observed time differences between successive keyboard actions for one of the test participants and one web page. We can clearly see three time intervals which are much longer as the other times. The first of these time intervals represents the time required to detect the target link, the second a focus loss, and the third the time required to select the target link.
4.5
Results
We got the following estimates for the parameters over all subjects: – t1: 9.6 seconds (time to detect target link in the page) – t4: 0.27 seconds (time for pressing the TAB key) – t5: 2.61 seconds (time to detect a lost cursor focus again) – t6: 1.13 seconds (time to select target link) – p: 3.18% (probability of focus losses) In addition we estimated these parameters for each individual participant of the study. The estimated parameter values per participant showed that there are significant interindividual differences. The observed standard deviations of the estimated parameter values between participants are 4.75 for t1, 0.06 for t4, 1.7 for t5, 0.46 for t6, and 1.33 % for p. As expected the estimate for the time to press the TAB key (0.27 sec.) is higher than the usual estimate for taping on a key while typing a text (0.2 sec.). Thus, our hypothesis that the cognitive process for the control of the cursor focus interferes with the process of pressing TAB repeatedly seems to be confirmed. As we can see focus losses occur relatively seldom (in only 3.18% of the cases). But if we calculate their influence on the task completion time we see that they are responsible for 20.71% of the total time subjects needed to solve the navigation task. This shows directly how important it is to minimize the probability of focus losses by an adequate design of a web page. For example, by a natural flow of the TAB chain or by clearly indicating the cursor focus. In addition the most time consuming part of the navigation task is the initial location of the target. This accounts for 33.87% of the total time subjects needed to solve the navigation task. Please note that the calculated times for the parameters of the GOMS model are valid for non-disabled highly trained computer users. Since disabled users who are not able to use a mouse will be much slower in hitting a key their disadvantage in performance compared to non-handicapped users can be in fact much
higher (some authors estimate that the average time a disabled user needs to press a key is higher than 0.6 seconds [18]).
5
Applications
The estimated times for the parameters of a GOMS model vary between different user groups or even individuals. Thus, a keystroke level GOMS model can not be used to calculate absolute timings with any degree of certainty. But by using average or typical values it is possible to obtain the correct ranking of the performance of different interface designs. Assume that we have a web unit which is designed for a number of different tasks. We can pick now a set of representative tasks and use the GOMS models to predict the expected times for users who solve the tasks using the mouse and keyboard respectively only the keyboard. A comparison of the predicted times can then be used to decide if keyboard support for the web unit is sufficient. Assume, for example, that we have a web page with 100 links which has not implemented any shortcuts or access keys. Thus, keyboard navigation must be done over the TAB chain. Assume in addition that users of this web page select all links with the same probability. Accordingly to our GOMS model for mouse navigation the average time to detect and follow a link on this page using the mouse will be 11.83 seconds. According to our GOMS model for keyboard navigation the time required for this navigation task using the keyboard is 28.56 seconds. If we want, for example, to achieve that the performance of keyboard navigation should not exceed two times the performance of mouse navigation we can conclude that we must implement additional keyboard support for this page (see, for example, [4] or [5] for a description of different methods to enhance keyboard support in web pages). Assume, for example, that it is possible to group the 100 links into 5 groups with 20 links. Assume further that 5 page internal navigation links are added to the top of the page. The i’th of these page internal navigation links sets the cursor focus to the first link in group i. Given these additional accelerators the average time to detect and follow a link on the page decreases accordingly to our GOMS model to 14.49 seconds.
6
Conclusions
The goal of this paper was to describe an inexpensive method to compare mouse and keyboard navigation in web units concerning their efficiency. We have shown that a GOMS analysis is a suitable tool to perform such a comparison. Therefore, we formulated GOMS models for keyboard and mouse navigation in web units. A comparison of the predicted GOMS times for mouse and keyboard navigation can be used to decide if the level of keyboard support for a web unit is sufficient or if it must be improved. Depending on the underlying problems this can require a redesign of the web unit or the introduction of accelerators [5] for keyboard handling, for example access keys. Thus, our method can be used to decide if the design of a web unit puts an unacceptable burden to keyboard users. What is acceptable depends clearly on the usage scenario of a web unit. For a web page of an online journal, which is visited once a week by an average reader, it may be acceptable that the time to reach an article using the keyboard is three times higher than the time required reaching the article using the mouse. For an intranet of a company, which is used by its employees to get important information relevant for their daily work things are different. For such web units even a difference of a factor 2 between mouse and keyboard performance may not be acceptable. Our GOMS models for mouse and keyboard navigation can be used in the early design phases of a project to make predictions concerning the expected performance of mouse respectively keyboard navigation. Thus, they can help web designers to predict if their designs meet the intended level of keyboard support.
References [1] Chrisholm, W., Vanderheiden, G., Jacobs, I. (1999): Web Content Accessibility Guidelines 1.0. Available online under http://www.w3c.org/TR/ WCAG10/.
[2] Caldwel, B., Chrisholm, W., Slatin, J., Vanderheiden, G. (2006): Web Content Accessibility Guidelines 2.0. W3C Working Draft 27 April 2006. Available online under http://www.w3c.org/TR/WCAG20/complete.html. [3] US Department of Justice. Section 508 of the Federal Rehabilitation Act. Available online under http://www.section508.gov/. [4] Schrepp, M., Jani, R. (2005): Efficient keyboard support in web-pages. In Pruski, A. & Knops, H. (Eds.): Assistive Technology: From Virtuality to Reality. IOS Press, pp. 504-508. [5] Schrepp, M. (2006). On the efficiency of keyboard navigation in Web sites. Universal Access in the Information Society, Vol. 5, No. 2, 180-189. [6] Bu¨hler, C., Stephanidis, C. (2004): European Co-operation Activities Promoting Design for All in Information Society Technologies. In: Miesenberger, K., Klaus, J., Zagler, W., Burger, D. (Eds.), ICCHP Computer Helping People with Special Needs. Volume 3118, pp. 80-87. [7] Stephanidis, C., Salvendy, G. (1999): Towards an information society for all: HCI challenges and R&D recommendations. International Journal of Human-Computer Interaction, 11(1), pp. 1-28. [8] Jani, R., Schrepp, M. (2004): Influence of Accessibility Related Activities on the Usability of the Business Software. In: Miesenberger, K., Klaus, J., Zagler, W. & Burger, D. (Eds.), ICCHP - Computer Helping People with Special Needs. Volume 3118, pp. 52-59. [9] Keates, S., Clarkson, P.J, Coy, J., Robinson, P. (1999): Universal Access in the work-place: A case study. Proceedings of the 5th ERCIM Workshop, Dagstuhl, Germany, pp. 73-80. [10] Cooper, A. (1995): About Face: The essentials of User Interface Design. IDG Books, Foster City, CA. [11] Nielsen, J. (2000): Novice vs. Expert Users. In: Useit.com - Usable Information Technology. Available online under http://www.useit.com/ alertbox/20000206.html. [12] Card, S.K., Moran, T.P., Newell, A.. (1983): The Psychology of HumanComputer Interaction. Hillsdale, NJ, Erlbaum. [13] John, B.E., Kieras, D.E. (1996): The GOMS family of user interface analysis techniques: Comparison and Contrast. ACM Transactions on ComputerHuman Interaction, 3, 4, pp. 320-351. [14] Kieras, D. (1997): A Guide to GOMS Model Usability Evaluation using NGOMSL. In: Helander, M., Landauer, T.K. & Pradhu, P. (Eds.), Handbook of Human-Computer Interactions. Elsevier. [15] Olson, J.R., Olson, G.M. (1990): The growth of cognitive modelling in human-computer interactions since GOMS. Human-Computer Interaction, 5, 221-265. [16] John, B.E. (1995). Why GOMS? Interactions, 80-89. Okt. 1995. [17] Eichst¨adt, H. (2005): Interaktionen blinder Nutzer bei der Bedienung linearisierter Oberfl¨achen [Interactions of blind persons during the usage of linearized user interfaces]. In: C. Stary (Ed.), Mensch & Computer 2005: Kunst und Wissenschaft - Grenzu¨berschreitungen der interaktiven ART, Oldenbourg Verlag: Munich, pp. 61-70. [18] Keates, S., Clarkson, P.J. & Robinson, P. (1998): Developing a methodology for the design of accessible interfaces. Proceedings of the 4’th ERCIM Workshop, Stockholm, Sweden. [19] Raskin, J. (2000). The Humane Interface: New Dimensions for Designing Interactive Systems. ACM Press. [20] MacKenzie, I.S. (1992). Fitt’s law as a research and design tool in humancomputer interaction. HumanComputer Interaction, 7, 91-139.
