A Quantitative Approach to Web Usability and Interface Design James L. Mohler
Ronald Glotzbach
Nishant Kothary
Purdue University 1404 W. State Street W. Lafayette, IN 47907
[email protected]
Purdue University 401. N. Grant St. W. Lafayette, IN 47907
[email protected]
Purdue University 401. N. Grant St. W. Lafayette, IN 47907
[email protected]
Abstract:
(c )J AP
IT
Current usability testing involves the collection of data that often lacks objectivity and seldom adheres to rigorous standards of quantitative research. Because such studies are characterized by subjective opinions of raters using Likert scales, they are often snared by many problems, not the least of which is rater reliability. This contribution presents a technological method for testing user-interface componentry and real-time data collection. In addition to presenting the inquiry framework, this presentation will also provide an applied example that used the framework to implement a usability experiment.
Introduction
As interactive content on the Web continues to grow, many are striving to use various techniques to ensure the quality of the interactive content, and thus, the user experience. Neilson (1994) has provided various Likert type scales to rate the overall quality of the web experience pertaining to usability. Many others have also focused upon the topic (Cato, 2001; Fleming, 1998; Hackos & Redish, 1998). Yet, such scales are often subjective in nature, being dependent on the reliability of the rater who is using the instrument. Additionally, they are more global in nature, focusing on the “big picture” issues rather than the specifics of any individual element or control. As a plausible alternative, it is suggested that quantitative studies can be used in many instances to improve specific parts of online content, particularly when the content does not fit into the traditional page metaphor. This paper describes one such study in which the goal was to determine the best type of control to use for panning or scrolling a map. The contribution extends our research, performing the experiment to a much larger group of university students. The prior pilot student, which yielded very preliminary findings, is published elsewhere (Mohler, Glotzbach & Kothary, 2003). The paper will also detail the technological framework that permitted the real-time online test.
Method
(c )J AP
IT
This study presented each user with three sequential tests in which he or she was required to scroll across a map and click on four spatially-separated, clickable items. The tests were given in a random order to avoid test bias. The clickable items on each test were located in different, but equidistant locations on the map and the starting locations for the map began in the same location on each test. The time required to scroll and click on the four items was recorded and then a mean for each test was computed across the four items. These means where used to determine the differences in total time – and thus the potential effectiveness of each type of control. The participants of this study were student volunteers (n = 163) from those enrolled in an introductory Web class at Purdue University.
Figure 1. The various control types that were tested included (a) trackball, (b) arrow palette, and (c) border.
One method of scrolling was based on a trackball design as shown in Figure 1a. The second was based on a floating palette of directional arrows (Figure 1b) and the third was based on controls that were integrated around the perimeter of the map window (Figure 1c). Upon answering some demographic questions, the participants were instructed that they had to search for clickable items within a map using three different sets of interactive controls. No instruction was given pertaining how to use each set of controls. Table 1 presents the overall 5 number summaries for each test with units in seconds. Table 1 Five Number Summaries
Max Q3 Md Q1 Min
Test 1 (s) 49.533 31.363 22.952 17.452 10.681
Test 2 (s) 30.933 24.326 21.639 19.202 14.566
Test 3 (s) 44.221 31.039 24.327 20.959 16.302
Results Results of the matched pair’s comparisons indicated that the arrow palette control yielded the shortest response times. When compared with the trackball, it yielded a significant difference of 3.2 seconds (p =
