Heuristic User Interface Evaluation: Three Case Studies of Dialog Design Renato Iannella Distributed Systems Technology Centre University of Queensland QLD, 4072, AUSTRALIA Tel: +61 7 3365 4310 Fax: +61 7 3365 4311 Email:
[email protected]
KEYWORDS User Interface Design and Evaluation; Heuristic Evaluation; User Interface Guidelines; Dialog Design; Usability Engineering; Human-Computer Interaction. ABSTRACT This paper discusses the use of heuristic guidelines in the design and evaluation of user interfaces. Closely following such recommendations can improve the user’s performance and reduce training costs. Three leading software applications are used as case studies in which the use of a simple set of heuristic guidelines has detected potential usability problems. Such problems can then be removed before more comprehensive usability evaluations are performed. INTRODUCTION The evaluation of user interfaces has traditionally held a low profile in software engineering. The application of such techniques has usually been either at the last stages of development or applied in a non-integrated fashion. This paper closely examines the dialog design of three leading software applications. The interfaces are summarised as they represent an excellent case study for the evaluation of user interfaces. Such real life examples provide concrete reference material for the application of Human-Computer Interaction (HCI) techniques in the software engineering lifecycle. Many have argued (Procter & Williams, 1992) that HCI should be owned by the software engineering community and that user interface concerns should be an integral part of the development cycle. Two of the major design aspects of user interfaces is consistency and feedback. Screen designs must look and behave consistently throughout the application and across similar applications. Appropriate feedback should also be provided for time consuming tasks and to indicate the status of options to the user. HEURISTIC GUIDELINES There is little question as to the benefits of user interface design guidelines and standards. Following user interface standards increases productivity for both the user and developer. Nielsen & Phillips (1993) outline the cost benefits of heuristic user interface evaluations and quote significant savings on increased expert user performance alone. Also, the costs of heuristic evaluations is lower than a formal usability evaluation. Other studies indicate that heuristic evaluations are a better method than cognitive walk-throughs for predicting specific problems (Desurvire et al, 1992). There are currently many thousands of usability guidelines available for developers. The guidelines cover a wide range of user interface areas and are available from a large number of sources. Guidelines are based on either heuristic knowledge or experimental evidence. The application of guidelines is usually a question of discovering the appropriate guidelines and extracting the relevant rules for the system being designed. Jeffries et al (1991) compared four usability techniques and report that heuristic evaluation identified the most serious problems with the least amount of effort. Nielsen (1992) also recommends that heuristic evaluations should be performed by a number of evaluators (three to five) to maximise results. However, Nielsen also states that usability specialists are more effective at finding problems, particularly specialists with expertise in the specific interface domain. There are three common classifications of guidelines: 1 International and national standards
2 System-dependent guidelines 3 System-independent guidelines The International Standards Organisation alone has developed approximately 70 standards with the same number of draft standards still to be released (Pangalos, 1992). Each standard covers a specific area of the user interface. System-dependent guidelines establish a degree of standardisation across computer environments. For example, the Common User Access (Berry, 1988) and Apple guidelines (Apple, 1987). System-independent guidelines provide non-hardware/software specific interface guidelines. They are usually gathered from diverse sources and presented in a manner that can be applied to specific interface environments. Guidelines are based on results from experiments, rule-of-thumb, or principles from human factors and interface design. Examples include Smith & Mosier (1986), Mayhew (1992), and Galitz (1993). The sheer number of guidelines may inhibit their use in the software development process. In some cases software has been developed to aid the designer in gathering and managing guidelines. An example from Smith & Mosier’s 944 guidelines is hyperSAM (Iannella, 1994). HyperSAM provides tools to manage the gathering, sorting, searching, annotating, and extracting of user interface guidelines. Thovtrup & Nielsen (1991) also recommend that tools should be provided that support the implementation of interfaces that follow usability guidelines. They also mention the lack of concrete examples of correct interface designs. The majority of guidelines can be sensibly cut down to include only the major rules that present the largest proportion of problems in user interfaces. Such a list from Molich & Nielsen (1990) is shown in Table 1. Table 1: Heuristic Guidelines
Simple and Natural Dialogue Speak the User’s Language Minimise the User’s Memory Load Be Consistent Provide Feedback Provide Clearly Marked Exits Provide Short-cuts Provide Good Error Messages Error Prevention The use of such simple and broad heuristics enables the interface to be challenged throughout the design stages. Nielsen & Molich (1990) outline the major advantages of heuristic evaluations to include the following: • • • •
inexpensive intuitive and easy to motivate people to do it does not require advance planning can be used early in the development process
One disadvantage is that it sometimes identifies usability problems without providing any direct solutions. CASE STUDY ONE Microsoft Excel1 is a sophisticated spreadsheet program available on a wide range of personal computers. Excel has undergone a number of revisions adding increased functionality and utilising new user interface techniques. However, the Value Axis Scale dialog has been identified as being misleading as it contravenes a number of user interface guidelines. The Value Axis Scale dialog allows the scale on a chart’s axes to be recalculated, minor and major units changed, and various other attributes modified. The Value Axis Scale dialog has been modified a number of times over the lifetime of the application. Unfortunately the designers have not identified a number of serious usability problems and have not followed applicable user interface guidelines. Consistency within user interface components and feedback are two of the areas that are lacking in the Value Axis Scale dialog
1. Microsoft Excel is a trademark of Microsoft Corporation
The Value Axis Scale dialog has evolved over four major versions. The dialog is not a frequently used part of the software which may have influenced the time spent on its design. Nevertheless, it should be designed with the full knowledge of usability heuristics. To view this from another perspective; the less frequently used dialog screens require just as much design effort as they should be as intuitive as possible. That is, seldom used functions, by their very nature, should follow the heuristic ‘Minimise the User’s Memory Load’. Figure 1 shows the Value Axis Scale dialog from Excel version 1.5.
Figure 1: Excel Version 1.5 Dialog
The minimum and maximum text fields are for the starting and ending ranges of the value axes. The major and minor units specify the value of the axes increments. The Category (horizontal) axes crossing value may also be specified. The other important part of this dialog is the series of check boxes under the Automatic title. These correspond to the five text fields. With these check boxes on, the values in the text fields are ignored and are calculated automatically. Figure 2 shows the Value Axis Scale dialog from version 4.0. In this version, the Automatic check boxes have moved to the left of the dialog. The Automatic title has been shortened and placed under the dialog box title. A new option for the Category axes has been added at the bottom of the dialog. A number of extra buttons has also been added to the right-side of the dialog and the Tick Label Position radio buttons have been removed.
Figure 2: Excel Version 4.0 Dialog
From these changes, it is clear that a problem has been identified with this dialog. In particular, the movement of the ‘Auto’ check boxes to the left of the text fields represents a major interface layout change. Dialog Evaluation From a study of the heuristic guidelines, four problems have been identified with the Value Axis Scale dialog. Each has been classified under the heuristics from Table 1. 1 The two Category Axis functions have been separated. (Heuristic: Minimise the user’s memory load.)
2 Conflicting options are possible for the Category Axis. This major problem will lead to an error situation if both options are set by the user. This is because both options cannot be satisfied at once. For example, a user could enter ‘1000’ into the ‘Crosses at:’ text field and turn the ‘Category Axis Crosses at Maximum Value’ check-box on. (Heuristic: Error prevention.) 3 Unclear use of ‘Automatic’ scale settings. The word ‘Auto’ under the main dialog title applies to all the check boxes listed under it. There is no clear indication that this is the case. In fact, the check boxes seem to apply to the values entered for each text field option. (Heuristic: Provide feedback.) 4 Inconsistent use of check boxes. Check boxes allow the user to select none, one, or more values from a list of options. For example, in this dialog, a user’s perception of the ‘Minimum’ check box may be that if they enter a value into the text field and turn the check box on, then that number would apply to the axes attribute. This is in fact not the case as the opposite is true. If the check box is turned on, the value in the text field is ignored and the ‘automatic’ value is used. (Heuristic: Be consistent.) Dialog Redesign With the above problems identified, it is now possible to redesign the dialog to address each point by following usability heuristics. Figure 3 shows an example of the first redesigned dialog.
Figure 3: Excel Dialog Redesign A
The advantages of this dialog interface include (with respect to the four interface problems listed previously): 1 The Category Axis options are now clearly shown grouped together. (Heuristics: Minimise the user’s memory load.) 2 The Category Axis options are now shown as radio buttons which correctly constrains the user to choose only one option. By doing this, a third option, ‘Crosses at Default’ was added. (Heuristic: Error prevention.) 3 The ‘Automatic’ and ‘Manual’ settings are now separated more effectively by increasing the white space. The two columns of text fields and check boxes are more visual. (Heuristic: Provide feedback.) 4 By providing appropriate feedback, the check boxes now behave consistently. For example, when a user turns one of the check boxes on, the label and value for the Manual calculation is dimmed and the user can perceive its actions. (Heuristic: Be consistent.) There is still a problem with the last two points in that the check boxes are not totally consistent with user interface guidelines. They are still ‘floating’ in white space with no obvious relationship with any text indicating the function of each check box. A second redesign of the dialog appears in Figure 4. In this case radio buttons have been utilised to enforce points 3 and 4 above. Only one radio button may be on at any time for each of the four axes scale options. Also the text fields are dimmed when inactive. The second design of the Value Axis Scale dialog now follows the applicable usability heuristics. The dialog should now be more consistent and clearer to use. In this particular case, since Excel has been on the market for a number of years, there are some other issues to address if the dialog was to be changed in a future version (Telles, 1990). CASE STUDY TWO The second case study examines the FrameMaker2 publishing system (Macintosh version 3.0.1i). The example discussed here represents more than just following heuristics as in the previous case study. FrameMaker has three formatting functions
2. FrameMaker is a trademark of Frame Technology Corporation.
Figure 4: Excel Dialog Redesign B
which can apply to paragraphs, characters, and tables. The user is able to create, modify, and delete style sheets (tags) for these three document formatting functions. Unfortunately, the way in which this has been implemented is not consistent across all three functions. The top of Figure 5 shows the Paragraph Format dialog screen involved in the formatting of paragraphs tags. There is also a floating dialog which displays all the paragraph tags so that the user can easily apply them to the document (‘¶ Catalog’ on the bottom right of Figure 5). On the bottom left of Figure 5 is an example of the Properties popup menu from the main Paragraph Format dialog. These are the full set of options that can be modified for each paragraph tag. The top of Figure 6 shows the Character Format dialog screen involved in the formatting of character tags. As with paragraph tags, there is also a floating dialog which displays all the character tags (‘ƒ Catalog’ on the bottom of Figure 6). As can be easily seen, use of both the paragraph and character formatting functions are similar.
Figure 5: FrameMaker Paragraph Dialogs
The top of Figure 7 shows the Table Format dialog screen involved in the formatting of table tags. The first problem encountered here is that there is no floating dialog which displays all the table tags as with the paragraph and character formats. To apply a table tag, the user must select the table, display the Table Format dialog, and select from the list of tags on the bottom left of the dialog. Immediately, the user notices inconsistencies with the use of table tags. Deletion of table tags is also presented in an inconsistent manner. As with character and paragraph tags, removal of tags is via the Catalog dialog. Each Catalog dialog has a ‘Delete…’ option on the bottom. Since table tags have no Catalog dialog, the designers have placed this option in the Properties popup menu (see the bottom of Figure 7). The problems identified with the Table formatting dialog represents a significant dialog design flaw. The inconsistent manner in which the application and deletion of table tags is handled is misleading for users. The interface model used for paragraph
Figure 6: FrameMaker Character Dialogs
Figure 7: FrameMaker Table Dialogs
and character tags should have been followed for table tags to minimise the user’s memory load. The placement of the ‘Delete from Catalog…’ option in the Properties popup menu for table tags also presents an inconsistent and misleading interaction model. The solution to this problem is to follow the model used by the paragraph and character tag dialogs and include a Catalog floating dialog for table tags. CASE STUDY THREE The third case study examines the Microsoft Mail3 electronic mail software (Macintosh version 3.1). The dialog under examination is the message addressing screen which allows message recipients to be selected into three categories: 1 Primary (To), 2 Carbon Copy (Cc), or 3 Blind Carbon Copy (Bcc). Figure 8 shows an example message addressing screen from Microsoft Mail. In general this is an effective dialog, however, one area in which it does lack is in the specification of the recipient category. The dialog can be improved by providing a quicker interaction method for modifying the recipient categories. (Heuristic: Provide short-cuts)
3. Microsoft Mail is a trademark of Microsoft Corporation.
Figure 8: Microsoft Mail Addressing Dialog
The message addressing dialog assumes that the user will select one of the ‘Add as:’ buttons (To, Cc, Bcc) before selecting the recipients. All recipients added after this selection will be categorised as such. The recipient names will then be preceded by ‘Cc:’ or ‘Bcc:’. If a user error is made (ie a recipient is added as the wrong category) the following must be performed: 1 2 3 4 5
select the recipient name, remove the recipient name, select a new recipient category, reselect the original recipient, and add to the recipient list.
However, with a simple redesign of the interaction required in this dialog, these five steps can easily be reduced to two steps. The change would be as follows: 1 select the recipient name, and 2 select a new recipient category. This would then instantly change the category for that recipient. The key here is that the dialog design has not changed, but the user interaction model is now more flexible. This allows the user quicker access to modify the message recipient list. DISCUSSION The three case studies discussed in this paper are typical examples of the lack of attention in following usability heuristics. They have each identified specific areas in which the application of user interface guidelines—during the design stages— may have prevented usability problems. The first case study highlights the need to maintain consistency for use of interface objects and to provide feedback on their status. Confusion will prevail if check boxes are implemented inconsistently across and within applications. The design of every dialog must be scrutinised and challenged. A number of designs should be generated and with simple heuristics, the most effective implemented. The second case study has highlighted the need to provide a consist interaction model for similar parts of an application. The designs for the character and paragraph formatting dialogs should have been directly mapped to the table formatting functions. This would have provided a much more consistent and effective interface. The third case study highlighted the need to provide quicker mechanisms for users in carrying out typical tasks. The dialog design was in fact suitable, but the interaction implementation could have been improved to assist the user. This type of improvement can greatly increase the user’s performance, particularly the experienced users. In all three cases, the need for explicit usability testing of the new designs is warranted. Heuristic evaluation may identify the potential interface problems but is no substitute for a full usability test with real users. In the examples reported here, only one experienced specialists was used. Adding more evaluators, as suggested by Nielsen (1992), would increase the number of problems found as well as producing different re-designed interfaces. However, just by using one evaluator, the more serious design problems were clearly identified. CONCLUSION This paper has highlighted the importance of following user interface guidelines in designing interfaces. In particular, it has used real applications and identified inadequacies in the interfaces and applied heuristic guidelines to develop an improved interface. This is an important exercise utilising common HCI tasks and can be seen as an effective case study for user interface design. The exercise could be extended to include usability evaluations by testing the new dialog designs with real users and recording their usages. In this case, it is clear to see that the simple application of a small set of heuristic guidelines can
have a significant effect on the dialog interface. A full usability evaluation may not have identified any problems with the dialogs evaluated. Hence the heuristics may have been the only evaluation technique used on the dialogs. The software engineering process needs to be sympathetic to the increasing applicability of HCI techniques. The heuristic evaluation of screen designs should be a mandatory task for software developers. If this is performed early, then large scale changes may not be required from final usability tests. As software engineers accepts HCI, then future international standards, such as European Community Council Directive 90/270/EEC stipulating usability requirements for commercial software, will not have such a significant effect on the industry. ACKNOWLEDGEMENTS The research reported in this paper was undertaken whilst the author was an employee of Bond University. The DSTC Pty Ltd has supported the author in the continuation of this work. REFERENCES Apple Computer. Human Interface Guidelines: The Apple Desktop Interface. Addison Wesley, 1987. Berry, R E. Common User Access—A consistent and usable human-computer interface for the SAA environments. IBM Systems Journal 27.3 (1988): 281–300. Desurvire, Heather W & Kondziela, Jim M & Atwood, Michael E. What is Gained and Lost when using Evaluation Methods other than Empirical Testing. Proceedings of People and Computers 7 HCI92, British Computer Society (1992): 89– 102. Galitz, Wilbert O. User-Interface Screen Design. Boston: QED Publishing Group, 1993. Iannella, Renato. HyperSAM—A Practical User Interface Guidelines Management System. Proceedings of 2nd Annual CHISIG (Queensland) Symposium—QCHI94, Bond University, Australia, 15th July 1994. (To appear) Jeffries, R & Miller, J R & Wharton, C & Vyeda K M. User Interface Evaluation in the Real World: A Comparison of Four Techniques. Proceedings of ACM Human Factors in Computing Systems CHI’91 (1991): 119–124. Mayhew, Deborah J. Principles and Guidelines in Software User Interface Design. New Jersey: Prentice Hall, 1992. Molich, Rolf & Nielsen, Jakob. Improving a Human–Computer Dialogue. Communications of the ACM 33.3 (Mar 1990): 338–348. Nielsen, Jakob. Finding Usability Problems Through Heuristic Evaluation. Proceedings of ACM Human Factors in Computing Systems CHI’92 (1992): 373–380. Nielsen, Jakob & Molich, Rolf. Heuristic Evaluation of User Interfaces. Proceedings of ACM Human Factors in Computing Systems CHI’90 (1990): 249–256. Nielsen, Jakob & Phillips, Victoria L. Estimating the Relative Usability of Two Interfaces: Heuristic, Formal, and Empirical Methods Compared. Proceedings of ACM/IFIP Human Factors in Computing Systems INTERCHI’93 (1993): 214– 221. Pangalos, G J. Consistency and standardization of user interfaces. Information and Software Technology 34.6 (June 1992): 397–401. Procter, R H & Williams, R A. HCI: Whose problem is IT anyway? Proceedings of IFIP TC2/WG2.7 Working Conference. Finland (10–14 Aug 1992): 385–396. Smith, Sidney L & Mosier, Jane N. Guidelines for Designing User Interface Software. The MITRE Corporation, Bedford, MA 01730-0208, USA. Report: ESD-TR-86-278 MTR-10090. 1986. Telles, Marcy. Updating an Older Interface. Proceedings of ACM Human Factors in Computing Systems CHI’90 (1990): 243–247. Thovtrup, Henrik & Nielsen, Jakob. Assessing the Usability of a User Interface Standard. Proceedings of ACM Human Factors in Computing Systems CHI’91. (1991): 335–341.
