Top Ten Myths about Usability Thomas S. Tullis, Ph.D. VP, Usability & User Insight Fidelity Investments
[email protected]
Simmons College November 12, 2009
1
2 Original photo: http://www.adamsavage.com/images/pix/mythbusters.jpg
Myth #1: Usability is Just Common Sense
Curved Barrel Machine Gun Life Magazine, 1953
For shooting around corners, where you can’t 3 see.
If usability is just common sense, then why do we have elevators like this…
4
Which of the following methods for re-ordering fields (e.g., columns of a table) is the most effective? Which is the least effective?
5
6
7
8
A
B
C
D
E
F
G
9
Mean Task Times (secs) (Error bars represent the 90% confidence interval)
The Loser
14 12 10 8 6 4 2 0 A. Drag/Drop On
B. Drag/Drop Between
C. Icons
D. Radio Buttons
E. Dropdown Menus
F. One Entry G. Four Field Entry Fields
The Winners
10
Actual Times vs. Predictions 6
C. Icons
r = -.07
Predictions (Lower=Better)
F. One Entry Field 5
G. Four Entry Fields D. Radio Buttons 4
E. Drop-down Menus 3
A. Drag/Drop On B. Drag/Drop Between 2 4
5
6
7
8
9
10
11
12
13
Actual Times (secs)
Tullis, T.S. (1993). Is User Interface Design Just Common Sense? Proceedings of the Fifth International Conference on Human-Computer Interaction, (HCI International '93), Orlando, Florida, USA, August 8-13, 1993, Volume 2. Elsevier 1993, ISBN 0-444-89540-X, pp. 9-14.
11
Myth #2: Usability is Just About Making Things Look Nice
Sleeper (1973)
12
Beautiful doesn’t necessarily mean usable. Usable doesn’t necessarily mean beautiful. But beautiful doesn’t preclude usable, nor does usable preclude beautiful.
Teapot theme inspired by Don Norman’s The Design of Everyday Things 13
Which of these sites is more visually appealing?
14
15
ACSI Score vs. Visual Appeal Rating 4.00
Neiman Marcus Banana Republic
Visual Appeal Rating
3.50
Apple
r = 0.21
3.00
2.50
Amazon 2.00
1.50
NewEgg
1.00 65
70
75
80
85
90
ACSI Score
Tullis, T.S., & Tullis, C.M. (2007). Statistical Analyses of E-commerce Websites: Can a Site Be Usable and Beautiful? Proceedings of HCI International 2007 Conference, Beijing.
16
17
Myth #3: Usability Can’t Be Measured
18 Photo: http://www.flickr.com/photos/minieng/2309432198/
Tobii 1750 eyetracking monitor
Video
19
Which of these designs for a messaging area would attract the most attention?
20
Design A
21
Design B
22
Design C
23
Design D
24
Design A
25
Design B
26
Design C
27
Design D
28
% of Participants Who Fixated Area for a Total of At Least 500 msec 60% 50% 40% 30% 20% 10% 0% Design A
Design B
Design C
Design D
29
Some traditional usability metrics: Percent Task Success 100%
96.5%
Mean 92.7%
97.8% Task Time (secs) 86.4%
90% 80% 70% 60%
140 120
40%
20% 100
10% 0%
4.5
4.2
4.1
4.0 Frequency Distribution of SUS Scores
4.0
4.0 3.5
2.9 18% 3.0 16% 60 2.5 14% 12% 40 NASA 2.0 10% 20 Wikipedia 8% 1.5 Task 1 Task 2 Task 3 Task 4 Task 5 06% 1.0 Task 2 Task 3 Task 4 Task 5 4% Task 1 Task 1 Task 2 Task 3 Task 4 Task 5 2% 0% 0-10 11- 21- 31- 41- 51- 61- 71- 81- 9120 30 40 50 60 70 80 90 100 80
% of Participants
20%
Mean Task Ease Rating (1-5, Higher=Better)
160
50%
30%
79.1%
180
SUS Score
30
Some less-traditional usability metrics:
Trimmel, M., Meixner-Pendleton, M., & Haring, S. (2003). Stress response caused by system response time when searching for information on the Internet : Psyochophysiology in ergonomics. Human Factors, Vol. 45, No. 4, pp. 615–621.
31
Some less-traditional usability metrics: Who can tell me what this is? Hint: This is what measures it!
Kapoor, A., Mota, S., & Picard, R. (2001). Towards a learning companion that recognizes affect. In AAAI Fall Symposium, November.
32
Some less-traditional usability metrics:
Reynolds, C.J. (2005). Adversarial Uses of Affective Computing and Ethical Implications. Ph.D. Thesis, MIT.
33
Myth #4: Usability Means Usability for the “Average” Person Photo: http://www.it4education.at/fubbcontent/lektionen /E/adjectives/adjquiz/tall_short.jpg
34
Several years ago, I was building a pool shed…
One of the decisions I had to make was how tall to make the 35 doorway.
What’s the average height of an American? About 66.5”
So if I make the doorway about 67” high… 36
But I don’t want President Obama to bump his head in the doorway when he comes over to swim at my house…
Photo: http://www.daylife.com/photo/08SRdCmdKh0M3
37
Height (inches)
Height Percentiles, in inches, for Americans (2002) 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 59 58 57 56 55
Women Men
5th
10th
15th
25th
50th
75th
85th
90th
95th
Percentile
Source: http://www.cdc.gov/nchs/data/ad/ad361.pdf
38
What’s the Real Answer? • Doors are a standard height: 80” • That’s not the 95th percentile of the population height– it’s more like the 99.9th percentile. • Architecture has evolved to a level of maturity where they have: – Standards – Standard building blocks
• And it’s much cheaper to make the doorway 80” high from the beginning than to go back later and retrofit one in! 39
So unless my good buddy Yao Ming comes over for a swim, I should be OK! Bottom Line: We should be making the “doors” to our websites and other products “taller”. 40 Original photo: http://www8.garmin.com/yao/content/media.html
In the year 2000 U.S. census, what percentage of Americans reported that they have a disability?
A. B. C. D.
5% 10% 15% 20%
41
42
How it might look to someone with cataracts
43
A Wikipedia article to someone with cataracts. 44
But how it can look to them by enlarging the text using browser text sizing.
45
How does someone who’s blind use the web?
Demo
Using screen-reading software.
46
So who is your “average user”?
Or is it this 68-year-old man who is a college graduate, has a disability, has broadband access, and speaks English at home?
Is it this 45-year-old woman who is a high school graduate, has no disabilities, dial-up access, and speaks English at home? Data from the Year 2000 Census about 100 Representative Americans…
47
My Stunning Conclusion from 6 Years of Graduate Training in Human Factors and 30+ Years of Work Experience in the Usability Field… People are different! And to achieve true usability for the broadest range of people, we need to pay attention to those differences in the design of our products and services. 48
Myth #5: Usability doesn’t have any real impact on our world.
Photo credit: http://static.panoramio.com/photos/original/8071466.jpg
49
Three Mile Island, March 29, 1979
Photos: http://www.animatedsoftware.com/hotwords/control_room/control_room.htm
50
This valve was stuck open, leading to loss of a large amount of reactor coolant, which caused a partial core meltdown.
The indicator light showing the status of the valve was actually showing what the valve had been commanded to do, not its actual state.
Bottom Line: Poor usability almost caused a nuclear meltdown. Drawing source: http://www.nrc.gov/reading-rm/doc-collections/fact-sheets/tmi-2.gif
51
Another Example…
In a quick study done by Canadian researchers right after the U.S. election, they found that 7% of shoppers “voted” for the wrong candidate when using this kind of butterfly ballot. Sinclair, R.C., Mark, M.M., Mooer, S.E., Lavis, C.A., & Soldat, A.S. (2000) Nature, 408, pp 665-666.
52
And…
53
Photo: http://www.usagold.com/wallpaper/coins_stack_43_lg.jpeg
Myth #6: Usability costs too much.
54
“Formal” Usability Testing…
55
There’s a wide range of options… Heuristic Evaluation or Expert Review
Photo: http://images.inmagine.com/img/inspirestock/ispi084/ispi084131.jpg
Photo: http://www.ergosign.de/en/user-centereddesign/usability-evaluation/expert-color.jpg
“Drive-by” usability testing “Café” studies (or Starbucks, mall, etc) Photo: http://staging.mddailyrecord.com/generationjd/files/2009/08/fe0791126d7ef6ba_man-laptop-starbucks.jpg
56
Lab Studies
From 5-6 participants to many more
Simmons College Usability Lab: http://gslis.simmons.edu/usability/equipment.html
57
Online Usability Studies
58
Online Usability Studies • Conducted over the web. • Users participate from their homes or offices at a time they choose. • They’re asked to go through some tasks using a website or prototype. • Data collected: – Task success – Task times – Subjective ratings – Clickstream data* * Depending upon the tool being used.
59
Results of Obama vs. McCain Website Evaluation Total effort to do this online study: about 6 person-hours.
Efficiency: % Success per Minute 90% 80% 70% 60%
Overall Ratings
50% 40%
7.0
20% 10% 0%
Obama
McCain
Rating (1-7, Higher=Better)
30% 6.0 5.0 Obama
4.0
McCain
3.0 2.0 1.0 Visual Appeal
Details: http://www.measuringux.com/Obama-McCain/index.htm
Ease of Finding
60
Some Tools for Conducting Online Usability Studies
Or you can “Roll Your Own” either very cheaply or for free: http://www.measuringux.com/RollingYourOwnOnlineStudy.ppt 61
Online Usability Studies
To be published January 29, 2010, by Morgan Kaufmann.
62
Myth #7: This must be usable because we built it in Flash (or AJAX, etc) Life magazine, 1950
63
Using the latest cool technology doesn’t make something usable.
Inventor Hugo Gernsback with his T.V. Glasses. Photo: Alfred Eisenstaedt/Time & Life Pictures/Getty Images Jan 01, 1963
64
The default settings in many new tools are often terrible from a usability perspective.
65
66
Myth #8: Usability is a fad. This too shall pass. Life Magazine Photo: Bill Ray Jan 01, 1972
67
The B-17 “Flying Fortress”
Usability’s heritage is in the human factors field, which began to emerge as a discipline during World War II. 68 Photo: http://www.military.cz/usa/air/war/bomber/b17/b17_infl.jpg
B-17 coming in for a landing, with its landing gear down and flaps fully extended.
Photo: http://lh5.ggpht.com/_1QUegZfskfE/R9QtQ6aREfI/AAAAAAAABco/jVkwCu04BOU/IMG_0007.jpg
69
But they would sometimes come in like this…
And crash land like this…
70
Photo: http://www.historyarticles.com/B17_COCKPIT.jpg
The B-17 Cockpit
One of these controls extends the flaps. Another virtually identical one raises the landing gear. 71
As a result of the work of the human-factors engineers, shapecoded controls were introduced into cockpits:
72
In the 1970’s, computers started to introduce new challenges for human factors people to solve…
73
So the early 1970’s was really the birth of interactive systems, which led to the first book, in 1973, on the design of these systems from the perspective of the user…
(As long as the user was male!) 74
Early Pointing Devices
75
And then in the 1980’s along came personal computers and solved all the usability problems…
76
And then in the 1990’s along came the web and once again solved all the usability problems…
A web page from 1992 by Tim Berners-Lee
Bottom Line: As technologies evolve, they create new usability problems that didn’t exist before. Usability issues aren’t going away, just evolving. 77
Graphic: http://konstructr.com/wp-content/uploads/2009/08/mpj043588000001.jpg
Myth #9: Usability doesn’t impact the bottom line. 78
If usability doesn’t impact the bottom line, then someone should tell Microsoft…
Just a few of the billboard ads at South Station, Boston, advertising Windows 7.
79
Usability ROI Case Study Task Times for Old vs. New Site Average Time (sec)
300 250
They calculated that the usability improvements yielded an annual savings of $1.2 million, plus an additional $550k of revenue annually.
200 Old Site New Site
150 100 50
Nu rs e
lic en Fo St ur at sur ye e M e ar a co p * lle Vo St ge s a te * r r te b eg i rd is tra * U. St S. tio ar l tin eg n * is g la a t b Ro us ors ad in co ess Hu nd * itio nt in ns g * lic en se *
0
Withrow, J., Brinck, T., and Speredelozzi, A. Comparative Usability Evaluation for an eGovernment Portal, Diamond Bullet Design Report, #U1-00-2, Ann Arbor, MI. Dec. 2000.
Task
Redesign of a state government portal. 80
Myth #10: Usability only applies to computers and web sites.
81
The good…
Oxo measuring cup in action! 82
The bad…
Photo: http://www.treehugger.com/bad-packaging-design-individually-wrapped-bananas-teaser-photo.jpg
The good… 83
The good…
The bad…
PaperPro springloaded stapler. Plastic packages that require a special tool to open. 84
The bad… Any clock radio that requires a user manual to operate.
Photo credits: ThisIsBroken.com
85
The good and the bad in the same product…
The serial number on the back of the iPod that some apps require you to enter. 86
To Recap: Top 10 Myths about Usability • • • • • • • • • •
Myth #1: Usability is Just Common Sense Myth #2: Usability is Just About Making Things Look Nice Myth #3: Usability Can’t Be Measured Myth #4: Usability Means Usability for the “Average” Person Myth #5: Usability Doesn’t Have Any Real Impact on our World Myth #6: Usability Costs Too Much Myth #7: This Must Be Usable Because We Built It in Flash (or AJAX, etc) Myth #8: Usability is a Fad. This Too Shall Pass. Myth #9: Usability Doesn’t Impact the Bottom Line Myth #10: Usability Only Applies to Computers and Web Sites. 87
A summary of this presentation will be available from: http://www.MeasuringUX.com/UsabilityMyths/
Thank You!
88
