Getting the Message Across: Visual Attention ... - Semantic Scholar

Getting the Message Across: Visual Attention, Aesthetic Design and What Users Remember Alistair Sutcliffe & Abdallah Namoune Manchester Business School, University of Manchester, UK +44 161-306 3315

[email protected] them, and animated adverts seem to distract from users’ search performance [3]. Furthermore there is evidence that eyetracking measures of attention are not correlated with user-reported areas of interest [7].

ABSTRACT An investigation into user attention and design quality in websites is described. The investigation combined reported attention to areas of interest, with free memory recall and a questionnaire to assess whether areas of interest that users attended to were also remembered and rated well in terms of aesthetic design and usability. Users’ memory of areas of interest and design features agreed reasonably well with their reported attention. The sites which were rated more attractive overall had an open layout, extensive use of animations which drew users’ attention, and good aesthetic design ratings. Aesthetics was the more important determinant for overall attractiveness; whereas content, brand and usability were more important for overall preference. Based on the analysis, design guidelines for directing user attention are proposed.

Since eyetracking is expensive both in terms of devices and the resources necessary to process the large quantity of data produced, alternative methods for capturing users’ attention might prove to be more effective and economical. Theories of user attention differ in their emphasis from processes dominated by bottom-up saliency-determined attention, whereby users’ attention is governed by the visual complexity and relative saliency of objects in an image, to more top-down goal-directed mechanisms that direct visual attention to areas of interest related to the users’ tasks and knowledge [2, 22]. Reported areas of interest coupled with post-test memory recall of website content should therefore provide a good means of testing the saliency of different areas, as well as users’ interest in the content. In contrast, eyetracking can find areas that may be salient but ultimately uninteresting for users.

Categories and Subject Descriptors H5.2. Information interfaces and presentation (e.g., HCI): User Interfaces; Methodology, Screen Design.

General Terms

Since aesthetics may well have a bearing on visual attention [12, 14], integrating evaluation of aesthetics, usability and communication of information via free recall memory should provide a richer picture of a website’s effectiveness. Questionnaire-based evaluation techniques have been proposed for aesthetics, for example pragmatic and hedonic qualities [6] or classic and expressive aesthetics [13]; however, the interaction between these measures and user attention, and good or bad impressions of design quality, has not been investigated. In this paper we explore the possible link between the design quality, user attention and memory to test the hypothesis ‘what is eye-catching is interesting and memorable’.

Design, Evaluation, Human Factors.

Keywords Visual attention, aesthetics, user attitude, web page design quality.

1. INTRODUCTION Website designers need to get key messages across to their audience and entice users to explore a site; guidelines emphasize the need for attractive and interesting home pages [6, 12, 14]. Design therefore needs to direct users’ attention to important information as well as providing an interesting user experience. However, the link between attention and the reaction of users to a design is far from clear. Aesthetic design as well as traditional usability principles such as a consistent, well structured layout should help to focus users attention; however, the role played by aesthetics design qualities in attracting users’ interest and attention is poorly understood.

The study investigates users’ reported attention in six operational websites to investigate the relationships between design features, user attention and comprehension, as manifest in memory recall. The paper is organized as follows. A method and materials section is followed by a summary of eyetracking results on six websites which have been reported previously [20]. The next section gives results on users’ reported areas of interest, memory and ratings of design quality. The paper concludes with a discussion of how the measures – eyetracking, reported areas of interest, memory and design quality – ratings can be interpreted in terms of the relationship between subjective and objective measures of attention, users’ memory of, and attitude to a design.

Eyetracking has become a popular means of evaluating the effectiveness of a design in attracting and holding user attention [10]. Some studies have demonstrated that users do not fixate on banner adverts in websites, even though they are aware of

2. RELATED WORK

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. DIS 2008, February 25-27, 2008, Cape Town, South Africa. Copyright 2008 ACM 978-1-60558-002-9/08/0002…$5.00.

Eyetracking studies have investigated how people read news web pages [15], suggesting that the first entry point for most web pages is located in the upper left corner and is usually a headline. In studies on banner adverts in web pages when users were asked to recall static or animated adverts, no significant effect was found on the ability to recall and recognize banner

11

too dense and complex. It has small animations in the centre above the top stories. The expected goals were to interest users in the news stories and site sub-sections. Nylon Magazine: an information provider with a blockstructured format which also uses image extensively. Animation is used on the central image. Communicating the information content, and interesting users in features were expected. PCWorld: an e-commerce site retailing computers and related technology. This site has a traditional block-structured columnar layout. Animated banner adverts are used in the central column. Interesting users in the range of products on offer should be the goal of any e-commerce site. TigerDirect: an e-commerce site retailing computers as a direct competitor to PCWorld, although with a less well known brand. One banner advert is present at the top of the page. The goals are similar to those of PCWorld.

ads [1]. In a similar study on the effects of banner ads on visual search and recall memory, Hornof and colleagues used eyetracking and reported that although banner ads distracted the users and significantly increased their search time, unexpectedly, memory recall for animated banners was worse than for static banners. Eyetracking data revealed that banner ads were rarely directly looked at, with only 11.7% of the banners being fixated on [3]. Eyetracking studies on different types of websites investigated the order of web pages being viewed, the gender, and the task at hand [16]. Participants were exposed to 11 popular websites in four categories: shopping, business, search, and news. Each user viewed eight web pages from the four types of website. Half of the participants were explicitly asked to remember the content of the websites, whereas the other half were asked to free-browse the web pages. The findings showed that the nature of the task did not significantly influence the fixation patterns of users although gender, the viewing order of a web page, and the interaction between page order and site type influenced user eye movements. Zhang instructed participants to identify and count text strings on a web page which contained an animated distractor [23]. She found that animation decreased user performance while searching for information, and the effect of animation on performance was determined by the complexity of the task. The more similar an animation was to the task, the greater was the negative influence on user performance; and brightly coloured animations interfered more than dull ones. In spite of many studies the attempt to connect a wide variety of eyetracking metrics and scan-path patterns to usability problems, and thereby create a predictive model, have been unsuccessful [5]. However, Eger et al. [4] suggest that use of eyetracking can improve the accuracy of retrospective problem report protocols in usability problem diagnosis, so the role of eyetracking as a standalone or combined forensic technique for usability analysis deserves further research. None of the above studies attempted systematically to unfold the relationship between web page design, eye movements, user attention, memory, and user attitude.

3. STUDY METHOD AND MATERIALS

Figure 1. Top row: IntelliPage, Nike; middle: BBCNews, Nylon Magazine; bottom, PCWorld, TigerDirect.

Six websites (see figure 1) were chosen to represent a diverse set of applications and different design styles. Two sites (Nike, IntelliPage) had a graphical look and feel and used extensive animation. Four sites followed a more traditional columnar block structured layout. Three sites had strong well known brands (BBC, Nike, PC World) and three did not. For each site we recorded our expectations of the design goals as a benchmark for evaluation outcomes in terms of effectiveness of communication and aesthetic appeal.

Thirty subjects (15 males, 15 females, mean age 24, range 1846 years), who were students and researchers at the University of Manchester, took part in the investigation. Most subjects (26/30) had 3 years or more Internet experience, and all used the Internet daily or at least once a week. The subjects were familiar with the BBCNews website (68% had visited it frequently, i.e. > 5 times) and some were aware of the PCWorld site (33% had visited it 2-5 times), while only 3 had visited Nike 2-5 times. None of the other sites were known to the subjects.

Nike: an e-commerce site with a strong brand image and design emphasis. The Nike site has a more open design format and makes extensive use of animation and graphics, in a sequence of men and animals running. Good aesthetic design was expected given the quality brand image. IntelliPage: an information provider site that also sells design services of the company. It has an open design format so in that sense it is similar to Nike. Animation was used on the main central image with two effects running concurrently. Good aesthetics should be a priority to interest users in the organization and sell its design services. BBCNews: the BBCNews site follows a traditional blockstructured columnar layout and can be criticized for being

12

The experimental procedure was as follows: (a) The participants completed a pre-test questionnaire recording their Internet experience, whether they had viewed any of the sites before, and their interest in the subject matter of the sites. (b) The participants were asked to view the home pages of each site for 60 seconds on a 17-inch monitor with screen resolution set to 1024*768 pixels. Users’ eye movements were recorded with a Tobii 1750 eyetracker. The number of fixations, duration of fixations, and dwell time by screen area (heat map) were analysed. The subjects were instructed to browse the page for any items that might interest them, given the purpose of the site. (c) Subjects recorded their perceived attention to different areas on each site by verbal reports and marking the areas on a screen-dump image. They ranked the areas of the screen by order of interest. (d) They completed a free recall memory test by listing any objects or areas of each site that they could remember and rating each item as either positive (liked), neutral, or negative (disliked). (e) The participants then rated each site on three questionnaires: classic and expressive aesthetic scales taken from [11]; perceived usability [9]; and a satisfaction questionnaire to capture the users’ overall rating of the application’s attractiveness and its effectiveness, with a task-related scenario, e.g. willingness to purchase on an ecommerce website. The order in which the home pages were presented was counter-balanced. Participants were explicitly instructed not to click on links within the pages; however, they could freely scroll down to view all the areas of the page.

Figure 2. Heat-maps for BBCNews (upper left), Nylon Magazine (upper right), PCWorld (lower left), and TigerDirect (lower right).

Reported areas of interest were captured by a combination of the user pointing to part of the screen, verbally describing the features of interest and drawing the area boundaries on a printed screen dump while viewing the application. The screen dumps and application were then removed and the user requested to recall and record in writing any areas, features or objects they remembered. The items recalled were rated as good, bad, neutral; and subsequently categorized as: user interface feature, functions, information content, other.

Figure 3. Heat-maps for open structure sites-IntelliPage (left) and Nike (right). In the Nike and IntelliPage sites, as illustrated in figure 3, fixations were focused on animations and salient images without any evidence of a layout order; furthermore there were fewer distinct fixated objects/areas. In these sites users’ attention seemed to be located on one main area with 5-6 subareas.

Subjects were paid £10 for the experiment, which concluded with a de-briefing interview to elicit their opinions about the designs, features that interested them, reasons for preferring any particular design, and criticism of design quality.

The areas fixated showed a strong influence for animation in four sites: Nike, IntelliPage, Nylon and the BBCNews, with less effect on PCWorld and TigerDirect even though these sites did have animated banner adverts. This appears to confirm previous findings of users selectively ignoring banner adverts, while interesting animations received considerable attention.

4. RESULTS 4.1 Eyetracking analysis

4.2 Perceived Attention

This section summarizes the eyetracking analysis; for further details, the reader is referred to [20]. Patterns of fixation densities illustrated as ‘heat maps’ of overall fixation durations were influenced by website structure (see figures 2 and 3). The sites with a column-block structure (BBC, Nylon, PCWorld and TigerDirect) all had high fixation densities in all columns with most areas receiving fixations. There were more intense fixations in the upper half of the page, and fewer in the lower half. Nylon had fewer more intense fixations on the central image. In contrast, sites with a more open structure (Nike and IntelliPage) had high fixation densities in the central parts of their graphical images, following the sequence of animations. The density of fixations was less evenly distributed, giving the impression that parts of the screen were not attended to, whereas in the block structured sites most areas appeared to receive some attention.

Users’ reported attention to areas of interest (AoIs) on each site was calculated as an average interest rating for each area multiplied by the % of users who rated that area. Figure 4 illustrates the top five fixation durations and and top five interest areas (circled) reported by users. Mismatches between fixations and interest areas are underlined in the key.

13

Figure 4a. IntelliPage Fixations: 1 middle box, 2 right box, 3 animated ball, 4 background image, 5 right menu. Interest (circled): 1 animated ball, 2 man on beach, 3 background image, 4 middle box, 5 right box.

Figure 4b. Nike Fixations: 1 animated text, 2 shoe & text, 3 men running, 4 text at centre, 5 menu captions. Interest (circled): 1 men running, 2 animated giraffes, 3 shoe & text, 4 menu at bottom, 5 text at centre.

Figure 4c. BBCNews Fixations: 1 around the world section, 2 main story, 3 top stories, 4 mid-section with images and stories, 5 other stories. Interest (circled): 1 main story & main image, 2 left second story & image, 3 BBCNews video & audio + Newsround /image, 4 mid-section with images and stories, 5 top stories.

Figure 4d. Nylon Fixations: 1 left menu, 2 writing at centre, 3 main animation, 4 Nylon cover & subscribe, 5 Nylon mall. Interest (circled): 1 main animation, 2 bottom animation, 3 left menu, 4 Nylon cover & subscribe, 5 Nylon mall .

Figure 4f. TigerDirect Fixations: 1 products at bottom, 2 left menu, 3 testimonials, 4 e-machines, 5 woman & laptop. Interest (circled): 1 woman & laptop, 2 products at bottom, 3 hard drive & deal of the day, 4 images of products at centre, 5 logo.

Figure 4e. PCWorld Fixations: 1 products at bottom, 2 main images, 3 description of product images, 4 left menu, 5 banner ads on left side. Interest (circled): 1 main product images, 2 images & description of products, 3 left menu, 4 jargon buster banner ad (pop-up), 5 AOL animation at bottom.

14

All fixations that fell within each structure area were counted and then the areas ranked for total fixations. The areas generally corresponded with the heat maps, although some higher densities on the heat maps were merged within one structural area.

cells and then comparing cells with fixation densities of >1% total fixations in the heat maps, and the users’ retrospectively reported areas of interest. Fixated cells which fell within an AoI boundary were scored. Where the boundaries of fixation cells and reported areas showed partial overlapping, agreement was scored as 0.5. The summary of attention measures is given in table 1.

In IntelliPage (figure 4a) the AoIs were all animated. In Nike (figure 4b), AoIs also followed animations which were organized in a sequence with the central animation unfolding while three animations occupied background areas (men running, a giraffe, and a bird flying).

Table 1. Agreement between reported areas of interest and high density fixation areas. R = Reported AoIs as % of total; F = Fixated areas as % of total; FR = Fixated and Reported as % of Reported

Areas of interest followed the animations but also included the menu sections at the base of the screen. There was a reasonably close correspondence between the objective (eyetracked fixations) and subjective (reported AoIs) measures of attention, although the rank order was different. Subjective and objective attention were in close agreement for Nike, apart from the second area of interest (giraffe animation) which was reported as interesting but in the high rank fixations. However, the heat map does show this part of the animation received some attention, so some areas of interest may be registered from less intense fixations.

Site

R

F

FR

Nike

100

76

76

IntelliPage

100

94

94

BBCNews

83

66

80

Nylon

100

77

77

PCWorld

84

26

31

TigerDirect

87

35

40

User attention covered most areas of the sites as shown by the high % reported areas of all area blocks in the site. The scores of lower than 100% for BBC, TigerDirect and PCWorld simply reflect lower-level block sub-divisions which were not identified separately. Fixations scores (table 1, column 2) also showed good coverage by the same measure of areas fixated as a % of all areas, except for PCWorld and TigerDirect. These lower scores reflect the threshold effect on the fixation density calculation, as many areas lower down their home page were below the 1% cut off, even though they registered some fixations in the heat maps. All sites had a high proportion of areas which were both fixated and reported AoIs. Once again the exception was PCWorld and TigerDirect; however their lower scores are partly an artefact of their low fixation %, and users remembering interesting areas lower down their home pages. These sites, in particular, suggest that users scan pages with low fixation densities and then remember AoIs from variety of locations within the page, while discarding many areas of high initial attention.

On the BBCNews page (figure 4c), the main story and image, top stories and mid-section were remembered and fixated. It appears that initial areas of interest revealed by eyetracking (e.g. around the world) were subsequently discounted as not interesting when attention was reported. Nylon (figure 4d) had one item in the interest list (bottom animation) which was not fixated, while the centre text was fixated but not rated as interesting. However, the animation did receive some fixations in the heat map, so it was scanned and remembered. In PCWorld (see figure 4e) the product images at the top of the page and the left-hand menu were in the top five remembered items; however, some product areas (right hand side) were not recalled in any detail. The banner adverts were only partially effective (left hand not reported). In TigerDirect (figure 4f) the central product adverts, and the logo (even though it was not an AoI) were in the top 5 remembered areas.

Nike and IntelliPage scored the highest means for reported AoI interest ratings of 3.78 and 2.78 respectively on 5-point scale where 5 was the highest. BBCNews and Nylon were close behind with 2.74 and 2.28, while PCWorld and TigerDirect trailed with average ratings of 1.75 and 1.93.

PCWorld and TigerDirect showed a weaker correspondence between the subjective and objective measures, which may reflect a user strategy of scanning these sites, since the heat maps also showed a less dense, more distributed pattern. The reported interest reflects the items which they briefly fixated on but attracted attention in the longer term. In their subjective record users reported interest in only a sub-set of the intensely fixated areas, so it appears that they were selectively ignoring some areas.

The total reported AoIs were classified by type of media, as illustrated in table 2. Animations were the most frequently remembered media type for the open structure sites. Text and Image/Text combinations were more frequent for BBCNews and TigerDirect, while Nylon and PCWorld had a spread of media. The banner adverts were reported areas in PCWorld but not for TigerDirect.

In these block-structured sites, users fixated on the products but ignored some of the adverts. In contrast, the open structure sites (Nike, IntelliPage) showed a stronger correspondence between fixations and reported areas of interest. Nylon had a close correspondence and appeared to follow the open structure pattern even though it was a hybrid design.

From de-briefing reports, in Nike the order of interest was linked to the unfolding sequence of animations (text, man, giraffe) which led the user towards the product menus lower down the page. The people and the giraffe animations were AoIs reported by many subjects but the main animation and its message were not (power to your feet). Users found the animations competed for their attention in IntelliPage, and only the man with the laptop and the pin ball animation were recalled accurately. The sequence of animations in Nike all followed in a smooth order, whereas the IntelliPage animations

Users’ reported attention to all areas on each site was compared to their eyetracked fixation durations. Each site was divided into areas based on its structure and functionality, i.e. segmented display areas, menus, images, animated areas, link lists, etc. The subjects’ reported interests were mapped to these potential areas of interest. Reported AOIs were compared with heat map fixation densities by dividing the screens into 0.5 cm

15

ran concurrently and competed with each other. Animations in the centre of the page attracted user attention in the Nylon page and were rated as interesting. Images of products in the centre of the page attracted attention in PCWorld and TigerDirect, followed by product images lower down the page. However, banner adverts in both sites were only partially effective. Table 2. Totals of AoIs categorized by media type.

Table 4. Free recall memory for each site, like-dislike weighted scores. IntelliP

Nike

BBC

Nylon

PCW

Tiger

Content

7

20

46

8

22

17

Image

20

11

0

1

1

-4

Logo

1

10

12

3

5

0

Anim

17

64

13

24

7

0

Nav.

3

9

15

7

15

4

Search

N/A

N/A

3

N/A

2

3

Colour

1

9

2

4

7

-2

Total

35

123

91

47

59

18

Media type Total AoIs reported

Text

Text & Image

Image

Anim. video

Navig. /links

Nike

7

1

0

0

5

1

IntelliPage

8

3

0

1

4

0

BBCNews 18

6

4

1

1

6

Nylon

10

3

0

2

3

2

PCWorld

19

4

1

3

5

5

TigerDirect 23

7

9

0

2

5

related memory (39%), whereas unexpectedly, IntelliPage, which had an animation-intensive design, produced more content and image memory and was not favoured with a high likeability rating. The best four sites for positive memory were Nike, BBC, Nylon and PC World. This suggests that some other factors may be influencing users’ memory and attitude toward the sites and their content even though all received approximately equal attention. IntelliPage may be suffering from animation design problems while PCW has more positive memory than Tiger even though the measures of attention coverage and interest rating were similar.

4.3 Memory The sites differed significantly in the total number of items remembered (F 2.74, df 5, p