Document not found! Please try again

Top Ten Myths About Usability - Measuring the User Experience [PDF]

2 downloads 135 Views 13MB Size Report
Jan 29, 2010 - 10. Mean Task Times (secs). (Error bars represent the 90% confidence ... International Conference on Human-Computer Interaction, (HCI ...
Top Ten Myths about Usability Thomas S. Tullis, Ph.D. VP, Usability & User Insight Fidelity Investments [email protected]

Simmons College November 12, 2009

1

2 Original photo: http://www.adamsavage.com/images/pix/mythbusters.jpg

Myth #1: Usability is Just Common Sense

Curved Barrel Machine Gun Life Magazine, 1953

For shooting around corners, where you can’t 3 see.

If usability is just common sense, then why do we have elevators like this…

4

Which of the following methods for re-ordering fields (e.g., columns of a table) is the most effective? Which is the least effective?

5

6

7

8

A

B

C

D

E

F

G

9

Mean Task Times (secs) (Error bars represent the 90% confidence interval)

The Loser

14 12 10 8 6 4 2 0 A. Drag/Drop On

B. Drag/Drop Between

C. Icons

D. Radio Buttons

E. Dropdown Menus

F. One Entry G. Four Field Entry Fields

The Winners

10

Actual Times vs. Predictions 6

C. Icons

r = -.07

Predictions (Lower=Better)

F. One Entry Field 5

G. Four Entry Fields D. Radio Buttons 4

E. Drop-down Menus 3

A. Drag/Drop On B. Drag/Drop Between 2 4

5

6

7

8

9

10

11

12

13

Actual Times (secs)

Tullis, T.S. (1993). Is User Interface Design Just Common Sense? Proceedings of the Fifth International Conference on Human-Computer Interaction, (HCI International '93), Orlando, Florida, USA, August 8-13, 1993, Volume 2. Elsevier 1993, ISBN 0-444-89540-X, pp. 9-14.

11

Myth #2: Usability is Just About Making Things Look Nice

Sleeper (1973)

12

Beautiful doesn’t necessarily mean usable. Usable doesn’t necessarily mean beautiful. But beautiful doesn’t preclude usable, nor does usable preclude beautiful.

Teapot theme inspired by Don Norman’s The Design of Everyday Things 13

Which of these sites is more visually appealing?

14

15

ACSI Score vs. Visual Appeal Rating 4.00

Neiman Marcus Banana Republic

Visual Appeal Rating

3.50

Apple

r = 0.21

3.00

2.50

Amazon 2.00

1.50

NewEgg

1.00 65

70

75

80

85

90

ACSI Score

Tullis, T.S., & Tullis, C.M. (2007). Statistical Analyses of E-commerce Websites: Can a Site Be Usable and Beautiful? Proceedings of HCI International 2007 Conference, Beijing.

16

17

Myth #3: Usability Can’t Be Measured

18 Photo: http://www.flickr.com/photos/minieng/2309432198/

Tobii 1750 eyetracking monitor

Video

19

Which of these designs for a messaging area would attract the most attention?

20

Design A

21

Design B

22

Design C

23

Design D

24

Design A

25

Design B

26

Design C

27

Design D

28

% of Participants Who Fixated Area for a Total of At Least 500 msec 60% 50% 40% 30% 20% 10% 0% Design A

Design B

Design C

Design D

29

Some traditional usability metrics: Percent Task Success 100%

96.5%

Mean 92.7%

97.8% Task Time (secs) 86.4%

90% 80% 70% 60%

140 120

40%

20% 100

10% 0%

4.5

4.2

4.1

4.0 Frequency Distribution of SUS Scores

4.0

4.0 3.5

2.9 18% 3.0 16% 60 2.5 14% 12% 40 NASA 2.0 10% 20 Wikipedia 8% 1.5 Task 1 Task 2 Task 3 Task 4 Task 5 06% 1.0 Task 2 Task 3 Task 4 Task 5 4% Task 1 Task 1 Task 2 Task 3 Task 4 Task 5 2% 0% 0-10 11- 21- 31- 41- 51- 61- 71- 81- 9120 30 40 50 60 70 80 90 100 80

% of Participants

20%

Mean Task Ease Rating (1-5, Higher=Better)

160

50%

30%

79.1%

180

SUS Score

30

Some less-traditional usability metrics:

Trimmel, M., Meixner-Pendleton, M., & Haring, S. (2003). Stress response caused by system response time when searching for information on the Internet : Psyochophysiology in ergonomics. Human Factors, Vol. 45, No. 4, pp. 615–621.

31

Some less-traditional usability metrics: Who can tell me what this is? Hint: This is what measures it!

Kapoor, A., Mota, S., & Picard, R. (2001). Towards a learning companion that recognizes affect. In AAAI Fall Symposium, November.

32

Some less-traditional usability metrics:

Reynolds, C.J. (2005). Adversarial Uses of Affective Computing and Ethical Implications. Ph.D. Thesis, MIT.

33

Myth #4: Usability Means Usability for the “Average” Person Photo: http://www.it4education.at/fubbcontent/lektionen /E/adjectives/adjquiz/tall_short.jpg

34

Several years ago, I was building a pool shed…

One of the decisions I had to make was how tall to make the 35 doorway.

What’s the average height of an American? About 66.5”

So if I make the doorway about 67” high… 36

But I don’t want President Obama to bump his head in the doorway when he comes over to swim at my house…

Photo: http://www.daylife.com/photo/08SRdCmdKh0M3

37

Height (inches)

Height Percentiles, in inches, for Americans (2002) 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 59 58 57 56 55

Women Men

5th

10th

15th

25th

50th

75th

85th

90th

95th

Percentile

Source: http://www.cdc.gov/nchs/data/ad/ad361.pdf

38

What’s the Real Answer? • Doors are a standard height: 80” • That’s not the 95th percentile of the population height– it’s more like the 99.9th percentile. • Architecture has evolved to a level of maturity where they have: – Standards – Standard building blocks

• And it’s much cheaper to make the doorway 80” high from the beginning than to go back later and retrofit one in! 39

So unless my good buddy Yao Ming comes over for a swim, I should be OK! Bottom Line: We should be making the “doors” to our websites and other products “taller”. 40 Original photo: http://www8.garmin.com/yao/content/media.html

In the year 2000 U.S. census, what percentage of Americans reported that they have a disability?

A. B. C. D.

5% 10% 15% 20%

41

42

How it might look to someone with cataracts

43

A Wikipedia article to someone with cataracts. 44

But how it can look to them by enlarging the text using browser text sizing.

45

How does someone who’s blind use the web?

Demo

Using screen-reading software.

46

So who is your “average user”?

Or is it this 68-year-old man who is a college graduate, has a disability, has broadband access, and speaks English at home?

Is it this 45-year-old woman who is a high school graduate, has no disabilities, dial-up access, and speaks English at home? Data from the Year 2000 Census about 100 Representative Americans…

47

My Stunning Conclusion from 6 Years of Graduate Training in Human Factors and 30+ Years of Work Experience in the Usability Field… People are different! And to achieve true usability for the broadest range of people, we need to pay attention to those differences in the design of our products and services. 48

Myth #5: Usability doesn’t have any real impact on our world.

Photo credit: http://static.panoramio.com/photos/original/8071466.jpg

49

Three Mile Island, March 29, 1979

Photos: http://www.animatedsoftware.com/hotwords/control_room/control_room.htm

50

This valve was stuck open, leading to loss of a large amount of reactor coolant, which caused a partial core meltdown.

The indicator light showing the status of the valve was actually showing what the valve had been commanded to do, not its actual state.

Bottom Line: Poor usability almost caused a nuclear meltdown. Drawing source: http://www.nrc.gov/reading-rm/doc-collections/fact-sheets/tmi-2.gif

51

Another Example…

In a quick study done by Canadian researchers right after the U.S. election, they found that 7% of shoppers “voted” for the wrong candidate when using this kind of butterfly ballot. Sinclair, R.C., Mark, M.M., Mooer, S.E., Lavis, C.A., & Soldat, A.S. (2000) Nature, 408, pp 665-666.

52

And…

53

Photo: http://www.usagold.com/wallpaper/coins_stack_43_lg.jpeg

Myth #6: Usability costs too much.

54

“Formal” Usability Testing…

55

There’s a wide range of options… Heuristic Evaluation or Expert Review

Photo: http://images.inmagine.com/img/inspirestock/ispi084/ispi084131.jpg

Photo: http://www.ergosign.de/en/user-centereddesign/usability-evaluation/expert-color.jpg

“Drive-by” usability testing “Café” studies (or Starbucks, mall, etc) Photo: http://staging.mddailyrecord.com/generationjd/files/2009/08/fe0791126d7ef6ba_man-laptop-starbucks.jpg

56

Lab Studies

From 5-6 participants to many more

Simmons College Usability Lab: http://gslis.simmons.edu/usability/equipment.html

57

Online Usability Studies

58

Online Usability Studies • Conducted over the web. • Users participate from their homes or offices at a time they choose. • They’re asked to go through some tasks using a website or prototype. • Data collected: – Task success – Task times – Subjective ratings – Clickstream data* * Depending upon the tool being used.

59

Results of Obama vs. McCain Website Evaluation Total effort to do this online study: about 6 person-hours.

Efficiency: % Success per Minute 90% 80% 70% 60%

Overall Ratings

50% 40%

7.0

20% 10% 0%

Obama

McCain

Rating (1-7, Higher=Better)

30% 6.0 5.0 Obama

4.0

McCain

3.0 2.0 1.0 Visual Appeal

Details: http://www.measuringux.com/Obama-McCain/index.htm

Ease of Finding

60

Some Tools for Conducting Online Usability Studies

Or you can “Roll Your Own” either very cheaply or for free: http://www.measuringux.com/RollingYourOwnOnlineStudy.ppt 61

Online Usability Studies

To be published January 29, 2010, by Morgan Kaufmann.

62

Myth #7: This must be usable because we built it in Flash (or AJAX, etc) Life magazine, 1950

63

Using the latest cool technology doesn’t make something usable.

Inventor Hugo Gernsback with his T.V. Glasses. Photo: Alfred Eisenstaedt/Time & Life Pictures/Getty Images Jan 01, 1963

64

The default settings in many new tools are often terrible from a usability perspective.

65

66

Myth #8: Usability is a fad. This too shall pass. Life Magazine Photo: Bill Ray Jan 01, 1972

67

The B-17 “Flying Fortress”

Usability’s heritage is in the human factors field, which began to emerge as a discipline during World War II. 68 Photo: http://www.military.cz/usa/air/war/bomber/b17/b17_infl.jpg

B-17 coming in for a landing, with its landing gear down and flaps fully extended.

Photo: http://lh5.ggpht.com/_1QUegZfskfE/R9QtQ6aREfI/AAAAAAAABco/jVkwCu04BOU/IMG_0007.jpg

69

But they would sometimes come in like this…

And crash land like this…

70

Photo: http://www.historyarticles.com/B17_COCKPIT.jpg

The B-17 Cockpit

One of these controls extends the flaps. Another virtually identical one raises the landing gear. 71

As a result of the work of the human-factors engineers, shapecoded controls were introduced into cockpits:

72

In the 1970’s, computers started to introduce new challenges for human factors people to solve…

73

So the early 1970’s was really the birth of interactive systems, which led to the first book, in 1973, on the design of these systems from the perspective of the user…

(As long as the user was male!) 74

Early Pointing Devices

75

And then in the 1980’s along came personal computers and solved all the usability problems…

76

And then in the 1990’s along came the web and once again solved all the usability problems…

A web page from 1992 by Tim Berners-Lee

Bottom Line: As technologies evolve, they create new usability problems that didn’t exist before. Usability issues aren’t going away, just evolving. 77

Graphic: http://konstructr.com/wp-content/uploads/2009/08/mpj043588000001.jpg

Myth #9: Usability doesn’t impact the bottom line. 78

If usability doesn’t impact the bottom line, then someone should tell Microsoft…

Just a few of the billboard ads at South Station, Boston, advertising Windows 7.

79

Usability ROI Case Study Task Times for Old vs. New Site Average Time (sec)

300 250

They calculated that the usability improvements yielded an annual savings of $1.2 million, plus an additional $550k of revenue annually.

200 Old Site New Site

150 100 50

Nu rs e

lic en Fo St ur at sur ye e M e ar a co p * lle Vo St ge s a te * r r te b eg i rd is tra * U. St S. tio ar l tin eg n * is g la a t b Ro us ors ad in co ess Hu nd * itio nt in ns g * lic en se *

0

Withrow, J., Brinck, T., and Speredelozzi, A. Comparative Usability Evaluation for an eGovernment Portal, Diamond Bullet Design Report, #U1-00-2, Ann Arbor, MI. Dec. 2000.

Task

Redesign of a state government portal. 80

Myth #10: Usability only applies to computers and web sites.

81

The good…

Oxo measuring cup in action! 82

The bad…

Photo: http://www.treehugger.com/bad-packaging-design-individually-wrapped-bananas-teaser-photo.jpg

The good… 83

The good…

The bad…

PaperPro springloaded stapler. Plastic packages that require a special tool to open. 84

The bad… Any clock radio that requires a user manual to operate.

Photo credits: ThisIsBroken.com

85

The good and the bad in the same product…

The serial number on the back of the iPod that some apps require you to enter. 86

To Recap: Top 10 Myths about Usability • • • • • • • • • •

Myth #1: Usability is Just Common Sense Myth #2: Usability is Just About Making Things Look Nice Myth #3: Usability Can’t Be Measured Myth #4: Usability Means Usability for the “Average” Person Myth #5: Usability Doesn’t Have Any Real Impact on our World Myth #6: Usability Costs Too Much Myth #7: This Must Be Usable Because We Built It in Flash (or AJAX, etc) Myth #8: Usability is a Fad. This Too Shall Pass. Myth #9: Usability Doesn’t Impact the Bottom Line Myth #10: Usability Only Applies to Computers and Web Sites. 87

A summary of this presentation will be available from: http://www.MeasuringUX.com/UsabilityMyths/

Thank You!

88