Tips and Tricks for a Better Usability Test Rolf Molich, DialogDesign, Skovkrogen 3, 3660 Stenlose Denmark,
[email protected]
Janice (Ginny) Redish Redish & Associates, Inc. 6820 Winterberry Lane Bethesda, MD 20817 USA
[email protected]
ABSTRACT
In this SIG experienced usability testers will exchange tips and tricks for practical usability testing.
usability test quality matches that of your prospective vendor before you sign the contract. •
Formative Evaluation of Information-Rich Web Sites (Ginny Redish). When the two major books in usability testing were written, the web was in its infancy. Now many of us are spending most of our time helping to make web sites usable. Information-rich web sites have different issues from e-shopping or application-on-the-web sites. What tips and techniques make for successful testing of information-rich web sites in the rapid development pace that is the norm for most web development teams?
•
Rigor in Usability Testing (Chauncey Wilson). We will discuss training and iterative review of facilitators. Some professionals already do this informally in their labs: After each study, they review their facilitation techniques and each other. Specifically, we will discuss the use of neutral prompts, a good script, appropriate use of statistics in analysis, feedback from clients, etc. The intent here is to reduce extraneous sources of variability. Tips will cover issues that include: the design of screeners for recruiting, pre- and post-test questionnaire design, choice of tasks, and good interview techniques.
Keywords
Usability testing, evaluation, usability test problems. INTRODUCTION TO SIG
This SIG will be a forum for exchanging practical tips and tricks for usability testing, in particular “think-aloud” usability testing. The SIG will concentrate mainly on new and advanced tips, and on mistakes that are often made even by experienced professionals. The point of departure for the discussion will be a handful of tips and tricks that the SIG proposers have come across during many years of practical usability testing. We have discovered most of these tips the hard way: By making mistakes, which users, mentors or observers have politely told us about. Other tips have come from watching usability professionals or hearing them talk about their testing practices. Still others come from web discussion forums, in particular CHI-WEB and “the-usability-list-that-must-notbe-named”. This SIG is intended for people who have already conducted several usability tests. People who are inexperienced with usability testing are also welcome but should note that the aim of the SIG is to exchange tips and tricks that are based on considerable practical experience. CONTENT
Each proposer will introduce a theme and lead a 30-minute exchange of tips and tricks within that theme. The themes that we will address are: •
Assessing the Quality of a Usability Test Firm (Rolf Molich). The scenario: You have decided to outsource usability testing locally or internationally. You want quality tests at a reasonable price. The Comparative Usability Evaluation study (see www.dialogdesign.dk/cue.html) has shown that there is sometimes no connection between resources required (hourly rate and number of hours required to run a test) and test result quality. We’ll discuss tips for determining whether your perception of
Chauncey E. Wilson, Bentley College, Design and Usability Testing Center Waltham, MA 02452 USA
[email protected]
The basic rule for deciding whether a tip belongs in this SIG is to take a look at the established textbooks in the area, [1] and [2]. A tip belongs in this SIG only if it is not mentioned in one of these fine basic textbooks. DRAFT SCHEDULE
We will divide the 90 minutes that are available for this SIG into three modules of 30 minutes each. Each module will focus on one of the above themes. In each module one of the SIG proposers will initially use 10 minutes to briefly introduce the theme and present practical tips. After this initial presentation, we will open a general discussion with the attendees to discuss our tips and solicit additional tips for discussion in the topic area. RELEVANCE TO THE HCI COMMUNITY
This SIG is a follow-up on a CHI2001 SIG with the same title, format and organizers (Rolf and Chauncey), but with entirely new themes.
We received an overwhelming response to our CHI2001 SIG: Five minutes before the official starting time, the room was crowded with more than 200 people, and the conference organizers had to turn away dozens more people who wanted to attend. Almost everybody stayed until the SIG was over. After the SIG we received more than 50 email requests for copies of our notes from the SIG (for copyright reasons we were not allowed to post these notes publicly on the net). DETAILS OF PROPOSERS
Rolf Molich owns and manages DialogDesign, a small Danish usability consultancy (www.dialogdesign.dk). Rolf teaches the Advanced Usability Testing Methodology tutorial for the NN Group, and managed the Comparative Usability Evaluation effort where seven professional teams and two student teams usability tested the same website.
Janice (Ginny) Redish has been working with private companies and government agencies as a consultant in usability and documentation since 1992. In 1979, Ginny founded the Document Design Center at the American Institutes for Research in Washington, DC, which she directed for 13 years, and where she set up one of the first independent usability test laboratories. Ginny is a graduate of Bryn Mawr College and has a Ph.D. in Linguistics from Harvard. Chauncey Wilson is Director of the Bentley College Design and Usability Testing Center and an assistant professor in the Information Design and Corporate Communications program. Chauncey was a product line development manager at BMC Software, Inc. for two years and has been an HCI architect and usability engineer for IDX Corporation, FTP Software, Dun & Bradstreet Software, and Digital Equipment Corporation. REFERENCES
1. Dumas, J.S. and Redish, J.C. A Practical Guide to Usability Testing, (Rev. Ed.) Intellect Books 1999 2. Rubin, J., Handbook of Usability Testing, John Wiley, New York NY, 1994
