computer science (courses for Bachelor's degrees are taught at another engineering school on campus) . Here we describe our goals for the course and give a ...
TEACHING USER INTERFACE DESIGN BASED O N USABILITY ENGINEERIN G
JAKOB NIELSE N ROLF MOLIC H
For the last three years we have been teaching a course i n user interface design at the Technical University of Den mark as part of the curriculum for the Master's Degree i n computer science (courses for Bachelor's degrees are taught at another engineering school on campus) . Here we describe our goals for the course and give a brief list of its contents . We discuss our use of mandatory laboratory assignments i n more detail and also present results from studies of students' motivation for taking the usability course and their evaluation of different aspects of the comm .
deliberately deemphasize many of the more theoretical or psychologically oriented aspects of current HCI researc h since we cannot transform our students to true usability experts anyway within our time limits . Our goal was to teach the students the following thre e things which we hope that they will be able to remember in years to come when they get jobs in industry to desig n real systems : • The "five golden rules" for the process of user interface design : 1) Know the user. 2) Involve users during the design . 3) Coordinate the total user interface . 4) Empirical measurements . 5) Iterative design to remove usability problems . This list has been synthesized in our Danish textboo k (discussed later) from principles originally described i n various papers by John Gould and Clayton Lewis . • Nine principles for heuristic evaluation of the product of user interface design: - simple and natural dialogue - speak the user's language - minimize the user's memory load - be consistent - provide feedback - provide clearly marked exits - provide shortcuts - good error messages - prevent errors
Our course serves as an introduction to user interface desig n issues and corresponds to about 20% of the work load for a full time student in one semester of 14 weeks (about 12 0 hours) . This university has considerable freedom for th e students to select what courses they want and when they want to take them, so some of the students were at the undergraduate level while others were just about to start writing their Master's thesis . A few students (about 3 of the 40 students who take th e class every year) decide to specialize in user interface issue s in their thesis work, and a few more participate in other ad vanced user interface projects at the graduate level but write their thesis on other subjects . But for the vast majority of the students, this one class forms their only education i n user interface issues . To gain the most benefit from having students take this one, fairly short course, we decided to fo cus our teaching on a small set of usability engineering methods which can be fairly easily applied in practice . We
SIGCHI Bulletin July 1989
45
Volume 21, Number 1
1986
1987 198 8
Do you have a home computer? (% answering yes) 56% 57% if yes : how big is its working memory (average RAM) 392K 526K How many computer science courses have you taken in the software field? How many computer science courses have you taken in the hardware field? How many hours per week do you spend using a computer? Have you ever worked on programming a product which was to be used by others? if yes: how many weeks full time work
Averag e
88% 725K 7 .4 course s 2 .2 course s 14 hours 63 % ye s 28 weeks
What were your reasons for taking this course ? I am trying to learn about as many aspects of computing as possible, and user interfaces are yet another subject Most present computer systems are not sufficiently user friendly and I would like to learn how to make them better Computers exist to be used, so a course in using computing should be an element in my education together with th e many courses on constructing computing systems User friendliness is an important sales argument for computers, so it is important to know something about i t to make programs which will sell I have heard from others who have taken the course before that It is good I would be willing to take any computer science course Modern interactive systems are of more interest than traditional computer systems Personal computing is of more Interest than traditional mainframes I am interested In the "soft" aspects of engineering I need knowledge about user interfaces for a project which I am currently working on I would like to help computer users Since there will be a large demand for people with knowledge about user interfaces, it will be easier to get a goo d job if I have had a course in usability Teachers in a course on usability should be more pedagogical then other teachers This course looks as if it will be easy to get through Other reasons
1 .8 1 .4 1 .3 0.9 0.8 0.6 0.6 0.6 0.5 0.5 0.3 0.3 0.1 0.1 1 .0
Note : Students were asked to distribute 10 points among the reasons listed (including a category labelled "other reasons" ) according to the importance of the reasons for their decision to apply for the course . In this table, the reasons are sorted according to their average score . The scores in the table are the averages from the answers from the years 1986, 1987, and 1988 since the answers have re mained about the same over the years . An exception is the score for having heard from other students that the course i s good . Here we report only the score from the 1988 since not very many students could have heard of the course from othe r students the first two years it was offered . The reason "needing knowledge for a current project" was not among those listed in the questionnaire in 1986 and 1987 but was often listed under "other reasons" . The number reported in this table is the rating from 1988 .
Table 1. Result of questionnaire given to students at the beginning of the course . The last column gives the averages of all the answers throughout the years. • The thinking aloud method as the most practica l and universally applicable method for the empirical part of the five golden rules.
To ensure that we provide quality education, we have followed a principle which is very common at this universit y and observe a strict limit of 40 on the number of students accepted . Of course we might have had more students sit in on our lectures, but we believe that laboratory work is a n indispensable part of learning usability engineering .
Other subjects we cover include classification of dialogu e techniques ; use of models in usability work, layered models of interactive systems, predictive models (e .g . GOMS) ; programming psychology; help, intelligent help, documentation ; hypertext; statistical measurement techniques ; usability standards . We also spend about three lectures each year on larger user interface case studies taken from ou r consulting experience or the literature (e .g . the Olympi c Message System).
Table 1 shows the results of a questionnaire distributed t o students at the beginning of the course where they wer e asked about their motivation for signing up for this course . It seems that the most important motivation is a kind o f " completeness " argument to the extent that students fee l that user interfaces form one (of many) aspects of computing, and that they have to learn something about them t o get a comprehensive computer science education . This i s OK, since it corresponds with the way we have designed the course : Not to make them usability experts but to provide them with a set of practical tools they can use whe n
Student motivatio n The course is very popular among the students, and eac h year three times as many students sign up for the course a s we have room for in our laboratory for interactive systems .
SIGCHI Bulletin July 1989
46
Volume 21, Number 1
of the quality of the report . Since they only run three subjects, we do not grade on the basis of the number of usability problems found but rather on issues such as whethe r the tasks the students gave their subjects seem relevant for the experiment and on whether the students were able t o improve their experimental method from one experiment t o the next .
they design systems . From the table we also see that the students taking thi s course are more interested in software than in hardware (th e university teaches both), that they spend a fair amount o f time using computers themselves, and that over the year s more and more of them have bigger and bigger personal computers.
The first two years we have included an assignment in th e writing of user manuals since they form an important par t of the user interface . Since very many students expressed the wish to get a chance to actually design their own use r interface, we included a design assignment in 1988 and ha d to remove the manual writing to avoid overloading the students with work .
About two thirds of the students have practical experienc e (about half a year's worth) of producing products whic h were actually going to be used. We have found that these students often make valuable contributions to discussion s in class with examples drawn from their own experience . We cannot make practical experience a formal prerequisit e for the course, but it seems that so many computer scienc e students have jobs on the side anyway, that we can rest assured of always having many experienced students in th e class .
The design assignment was especially hard to grade since i t had more degrees of freedom than the other assignments . To reduce the variability somewhat, we chose to have th e students design an extension to an existing system which they got specified together with a few pages of suppose d user interface standards for the hypothetical company i n which the system was used . In spite of our frequent prais e of consistency at the lectures, many students still deviated from this interface standard in their design . For example , the standard specified a given method for users to acknowl edge or cancel an action when the system asked them t o confirm it . Even so, some students invented alternative interaction techniques for this dialogue step where consistency would normally be considered especially important . The usability standard and the existing design enabled us to pro duce a checklist for elements which should be in the students's designs and elements which must not appear i n their designs . Of course we also awarded points for more individual aspects of the designs, but in general the checklist made it possible for us to do the grading uniformly . Even so, it was still possible to exercise considerable creativity within the bounds of the assignment, and the resulting designs were quite different. Mostly they could be classified into one of two general classes but there was also a completely different, third design class which was the solution originally suggested by one of us but which was no t used by any of the students .
Mandatory assignment s As mentioned above, this course has a heavy emphasis o n laboratory work. This is mostly in the form of fairly large mandatory assignments in various areas of usability engineering, and we know from questionnaires that student s spend about half of their course work load on these assignments . Students are graded on the basis of their assignment wor k instead of through a traditional written exam . This motivates students to work harder in the lab and it reflects ou r course emphasis on practical aspects of usability engineering . Having to grade the reports carefully places a fairl y heavy workload on the teachers, however . To reduce this workload and to ensure uniform standards of grading we have relied on a system of checklists for possible observations for which we award points . A few students have been dissatisfied with this grading method and felt that it was too inflexible, but we feel that it is the best way to approximate objectivity in the grading process . The course evaluation questionnaire summarized in Table 2 indicates that students are quite satisfied with having mandatory assignments instead of traditional exams and tha t they also believe that they learnt something from them . It seems that students get their main benefit from conductin g their own thinking aloud study . When they actually observe real users having trouble using small and seemingl y easy programs, they realize that usability is much mor e than just a theoretically issue to be discussed in class .
The following list shows the assignments we have use d over the years : 198 6 • Analysis of the user interface of an automatic teller machine • Thinking aloud study of MacDraw (Apple Macintosh ) • Writing a user manual for a given rolodex program 198 7 • Measuring the usability of bank account statement s • Analysis of the user interface of two parts of the Danis h Videotex system (top level search facilities and som e airline information pages)
Each year we have had an assignment in analyzing the usability of some system according to our usability principles . Grading is mostly done by counting the number of usability problems found by each student. We also have student s do a thinking aloud study which is then graded on the basis
SIGCHI Bulletin July 1989
47
Volume 21, Number 1
1986
1987
1988
Ave .
4 .8
4 .6
4 .8
4.7
3 .9 4 .2 1 .6 4 .9 1 .6 1 .3 2 .0 1 .9 4 .2 3 .9
4 .8 4 .7 1 .3 4 .6 1 .6 1 .7 1 .7 2 .1 4 .1 3 .5
4 .8 4 .6 1 .7 4 .1 1 .5 1 .8 1 .6 2 .2 3 .4 4 .3
4.6 4.6 1 .5 4.5 1 .6 1 .7 1 .7 2. 1 3.9 3.9
2 .1
2 .5
2 .0
2.2
3 .1 3 .1 2 .7
3 .5 3 .4 2 .7
3 .6 2 .8 3 .8
3.5 3.1 3.1
4 .9 4 .1 3 .7
3 .4 2 .8 2 .0
4 .2 3 .8 2 .8
4.0 3.4 2 .6
Course evaluation (agree/disagree with the following statements) : In total, this is a good course Mandatory assignments are a better form for examination than traditiona l written exams for this course I have actually learnt something from doing the assignments The course content is too advanced and difficult I understand practically everything which is said during the lectures I have not learnt very much from this course The course Is too theoretically inclined compared to the practical utility Too little is demanded of the students in this course The teachers are not pedagogical enough The textbook is good It is hampering not to have lecture notes to all subjects not covered in the textbook The course is too narrowly focused upon immediately applicable practical methods and there is too little emphasis on more long-term principles It is bad to have a single computer (the Macintosh) dominate s o much at lectures and assignments The lecture notes are good I would not have been bothered by having an English textbook
Would later (after this course) want to - take advanced course in programming graphical user interfaces - take advanced course in experimental evaluation of user interfaces - do thesis work in the user interface area
Table 2 . Result of questionnaire distributed towards the end of the course . The rating scale used was : 1-agree completely; 2-Qlj'gagree somewhat; 3-neutral; 4-agree somewhat; 5-agree completely . In this table, evaluations have been sorted ac -
cording to their distance from the neutral point, 3 . older. A solution might be to change to another book bu t we have not yet found one having the emphasis on usability engineering we require for our course . It would be possible to use an English-language textbook but having a Danish book is an advantage for most students (excep t possibly for the one or two immigrants in each class) .
• Thinking aloud study of MacPain t • Writing a user manual for a small database syste m 198 8 • Analysis of the user interface of two audio response in formation systems (accessed via pushbutton telephones) • Designing an extension to a banking syste m • Thinking aloud study of MacWrite
Our interactive systems laboratory has 7 Macintoshes model Plus or SE . The first year we had only 20 students and 3 Macs which almost was a good fit except for somewha t too long waiting periods at peak load during periods o f mandatory assignments . So when the course capacity wa s doubled to 40 students in 1987 we decided to slightly mor e than double the laboratory capacity .
Practical Issue s We have been two teachers involved in giving this course . One with a background primarily in research and one with a background primarily in practice . This has contributed positively to the class since we have been able to cover a variety of perspectives and have even in some cases presente d contrasting views on some user interface issues. We have drawn on examples from practice in many cases, includin g some of the mandatory assignments .
According to Table 2, students have over the years agreed more and more that it was bad to base the course on th e Macintosh. One reason for this trend is probably that th e machines we use seem less and less spectacular compare d with other computers known by the students . In 198 6 most students had only seen text-based interfaces before taking our course and the reason we chose Macs for our lab was exactly that we wanted to expose them to graphical user interfaces . The actual goal of our lab work can still b e served quite nicely by the small-screen Macs, since we focus on general usability principles much more . than on specific "smart" interaction techniques which may well be come obsolete before our students graduate .
We have used the same textbook every year : R . Molich (Ed.) : Brugervenlige edb-systemer ("User friendly computer systems " , in Danish) . This book was not really written a s a university-level textbook but rather as a self-study boo k for practitioners . Therefore it has been necessary to supplement the book with lecture notes on the more advanced o r theoretical subjects . Table 2 indicates that the students like the book but are less satisfied with the lecture notes . Also , the book was published in 1986 and Table 2 shows that students tend to rate it slightly worse every year as it gets
SIGCHI Bulletin July 1989
48
Volume 21, Number 1
