Clustered insights - Semantic Scholar

International Journal of Market Research Vol. 55 Issue 1

Clustered insights Improving eye tracking data analysis using scan statistics Christian Purucker

University of Würzburg

Jan R. Landwehr

Goethe University Frankfurt

David E. Sprott

Washington State University

Andreas Herrmann

University of St. Gallen

Analysis of eye-tracking data in marketing research has traditionally relied upon regions of interest (ROIs) methodology or the use of heatmaps. Clear disadvantages exist for both methods. Addressing this gap, the current research applies spatiotemporal scan statistics to the analysis and visualisation of eye tracking data. Results of a sample experiment using anthropomorphic car faces demonstrate several advantages provided by the new method. In contrast to traditional approaches, scan statistics provide a means to scan eye tracking data automatically in space and time with differing gaze clusters, with results able to be comprehensively visualised and statistically assessed.

Introduction Marketing research using eye tracking methods is broad. Recent growth in the use of these methods has been driven by tremendous technological advancements in eye tracking equipment with resultant adoption by a wide range of researchers and practitioners (Duchowski 2002). Various methods have been used to analyse eye tracking data, but most procedures rely upon traditional eye tracking metrics (Duchowski 2007), such as the number of fixations calculated for regions of interest (ROIs) that are superimposed on the stimulus.

Received (in revised form): 5 October 2011

© 2013 The Market Research Society DOI: 10.2501/IJMR-2013-009

105

Clustered insights

Although often used in marketing research, there are various concerns associated with the use of ROIs that must be considered. In particular, ROIs must be defined a priori (which often proves difficult). As such, specification of ROIs can often be arbitrary and may not be directly grounded in the literature. Further, subregions included within an ROI might add noise to the gaze measures acquired (see Heller et al. 2006). Of greater concern is that the ROI might not account for what was actually perceived by the viewer; rather ROIs merely capture whether an observer looks directly at the location where the experimenter expects him or her to look. While one approach – the heatmap technique (e.g. Pomplun et al. 1996; Wooding 2002) – has been developed to address this latter concern, the lack of a statistical criterion for heatmaps has reduced the use of the tool to illustrative purposes in research. The aim of the current research is to illustrate some of the critical concerns with ROI-based methods, and to develop a new approach that allows for a data-driven identification of eye tracking point clusters in space and time. This new method allows for hypotheses testing and a unique approach to data visualisation. Our paper begins with a short review of recent research in marketing that has employed eye tracking methodology. Next, a review of traditional techniques for analysing eye tracking data using ROIs is provided, along with a discussion of methodological concerns with such approaches. After introducing a new analytical method for eye tracking data based on spatiotemporal scan statistics, an experiment is reported that uses automobile car face stimuli to compare ROI-based analysis and scan statistics. The implications of our research for eye tracking methodology and product design are provided.

Theoretical background Eye tracking in marketing Eye tracking methodology has been employed in marketing research and related fields using stimuli ranging from packaging and the design of product labels to advertising in various forms of media. An overview of eye tracking research in relevant, top-tier marketing journals is provided in Table 1. Numerous works have been published, many with a strong practical focus. For example, Bojko et al. (2005) used eye tracking methods to examine consumers’ visual gaze behaviour when selecting products with different drug labels. Similarly, Bix et al. (2009) employed eye tracking data to examine viewing time spent on warning statements and other

106

International Journal of Market Research Vol. 55 Issue 1

regions of drug packaging. Eye tracking methods have also been employed in online marketing to address the optimisation of information placement in search engines (Cutrell & Guan 2007), prediction of salient areas in web pages (Yesilada et al. 2008; Buscher et al. 2009), and the assessment of banner ads by investigating the banner blindness phenomenon (Hervet et al. 2010). Other recently published work has demonstrated the benefits from combining eye tracking data with established measures in marketing, as in the case of choice tasks (Shao et al. 2008; Meißner & Decker 2010). Moreover, work presented in industry settings, such as the ESOMAR conferences, illustrates the implementation of eye tracking in other contexts, like shelf testing (Hamaekers & Laan 2011) and placement of advertisements in real-life situations (Thomas-Smith & Barnett 2010), suggesting a widespread use of eye tracking technologies in proprietary research as well. Overall, there is a strong tradition and increasing interest in using eye tracking methods to enhance our understanding of various marketing topics.

Traditional eye tracking data analysis with regions of interest The most common eye tracking analysis techniques are based on regions of interest (Duchowski 2002). The widespread use of ROIs is clearly illustrated in Table 1, with nearly all studies using these eye tracking methods. This situation is not surprising since the use of ROIs is straightforward and ROI analysis is typically available in standard software packages. ROI-based methods are used for analysing patterns in eye tracking data regarding specific areas of a stimulus, such as the total gaze duration on a specific element of the advertisement (see Pieters & Wedel 2007). In ROI analysis, geometric forms are defined by the experimenter based on theoretical considerations and drawn around specific parts of a visual stimulus (see Duchowski 2007). Classical eye tracking metrics (e.g. the total gaze duration) are then calculated from these regions and most often tested against one another by using complex statistical models. The shapes of ROIs are often rectangles that are drawn either tightly (see Chandon et al. 2009) or loosely (see Brasel & Gips 2008) around the visual areas of interest. While prior research has often relied upon classical eye tracking metrics based on ROIs, there are at least three limitations that researchers need to consider when using these tools. First, data quality is closely related to how ROIs are defined and therefore highly dependent upon the researcher’s definition of them. For example, a certain ROI might consist of subregions that behave in

107

108 Experimental websites with ads

Stimulus material Real magazine advertisements

Mean gaze duration

Feature ads

Real feature advertisements

Unspecified

Package and Notice, price included reexamination of ROI

Products

Spatial variation at certain time frame

Mean gaze duration

Supermarket shelf diagrams

Unspecified, overlap

n.a.

Brand, picture, text, overall ad

Eye tracking methodology ROIs ROI definition Metrics Brand, picture, Unspecified Number of text fixations, total gaze duration Relative and Areas with Unspecified, mean fixation advertisements likely frequencies rectangular

n.a. Real, experimental TV commercials

Real magazine Effects of forms of visual complexity in advertisements ads on attention

Impact of branding and customer attention on TV ad avoidance Chandon et al. Effect of in- and 2009 (JM) out-of-store factors on attention, evaluation Zhang et al. Effect of feature ad 2009 (JMR) characteristics on attention, sales

Teixeir et al. 2010 (MS)

Pieters et al. 2010 (JM)

Author(s), year, (journal) Topic Aribarg et al. Relationship 2010 (JMR) between attention and recognition of ads Kuisma et al. Impact of animation 2010 (JIM) and format in online ads on attention, memory

Table 1  Overview of marketing papers using eye tracking technology

Mediation analyses (classical and Bayes)

Logistic regression with random effect

Dynamic probit model

Multivariate multilevel regression

Multiple regression, ANOVAs

Statistics Random stopped sum model (Bayesian)

Eye tracking improves sales prediction, optimization of attention (continued)

Main results Ad layout affects recognition, strong focus on model deduction Interaction of animation and format, ad recognition without direct gazes Feature complexity lowers attention, design complexity elevates attention Central brand position facilitates avoidance, brand pulses decrease No direct relationship between attention and evaluation

Clustered insights

Effects of key ad elements on attention capture

Effect of involvement and picture position in ads on attention

Garcí et al. 2000 (ACR)

Topic Relationship between font layout and peripheral ad fixations Attention to TV ads and position while fast forwarding Information processing during learning of really new products Determination of brand salience in supermarkets Effects of processing goals on attention when viewing ads

Pieters & Wedel 2004 (JM)

Van der Lans et al. 2008 (MS) Pieters & Wedel 2007 (JCR)

Feiereisen et al. 2008 (JPIM)

Brasel & Gips 2008 (JM)

Author(s), year, (journal) Baraggioli & Brasel 2008 (ACR)

Experimental advertisements

Pictorial elements

Brand, picture, text, overall ad

Real magazine advertisements

Unspecified

Unspecified, overlap

Mean fixation duration, Number of fixations

Total gaze duration, selection

Total gaze duration, selection

Brand, picture, headline, text, etc.

Real magazine advertisements

n.a.

Unspecified

n.a.

Supermarket shelf diagrams

Number of fixations, total fixation duration Locations of fixations

Unspecified

Ad elements

Experimental print advertisements

Real TV commercials

Eye tracking methodology ROIs ROI definition Metrics Mean fixation Advertisements, Unspecified, duration, text likely number of rectangular fixations Overall, central Grid overlay Number of gaze region, brand points, deviation

Stimulus material Websites with texts and advertisements

ANOVAs

Extended Hidden Markov model Multivariate hierarch. regression (Bayesian) Multivariate multilevel regression

Repeated measures ANOVAs Pearson productmoment correlations

Statistics ANOVAs

(continued)

Attention focus on central screen region, better memorization Increase in attention indicates comprehension or difficulties Bottom-up processes account for 2/3 of the brand salience Viewer goals affect attention distribution on relevant ad elements Relationship between text size and attention to ad, attention transfer Interaction of position and involvement on attention

Main results Character spacing affects peripheral attention

International Journal of Market Research Vol. 55 Issue 1

109

110

Topic Relationship between fixations on ads and memory formation Impact of time pressure and motivation on attention, choices Effects of repeated ad presentation on attention

Fox et al. 1998 Adolescents’ (JA) attention towards warnings on ads for legal drugs Lohse 1997 Effect of ad (JA) characteristics on consumer information processing Pieters et al. Impact of time 1997 (ACR) pressure and motivation on attention, choices

Pieters et al. 1999 (JMR)

Pieters & Warlop 1999 (IJRM)

Author(s), year, (journal) Wedel & Pieters 2000 (MS)

Dissimilar rectangles, overlap

Brand, picture, Experimental choice sets with ingredients goods

Unspecified

Unspecified, overlap

Unspecified, likely rectangular

Warning messages, overall ad

Headline, picture, text, pack shot

Experimental Display ads, yellow page ads listings

Real print advertisements

Experimental/ real ads for products

Dissimilar rectangles, overlap

MANOVA, repeated measures ANOVAs MANOVA

ANOVAs, MANOVA, correlations Mean fixation duration, Number of saccades

Nested gamma and heterogeneous models

Statistics Negative binomial hierarchical model ANOVA, conditional logit

Total fixation duration, Percentage attending Mean fixation number, Total fixation duration

Mean fixation duration, Number of saccades Mean gaze duration

Eye tracking methodology ROIs ROI definition Metrics Brand, picture, Unspecified, Fixation text overlap frequencies

Brand, picture, Experimental choice sets with ingredients goods

Stimulus material Real magazine advertisements

Table 1  Overview of marketing papers using eye tracking technology (continued)

Strong relationship between attention and choice, effect of time pressure (continued)

Chosen ads were looked at longer, effect of size and colour on attention

Chosen brands were looked at longer, pressure, motivation affect emphasis Attentional decrement for repeated ad presentation, stable scan paths Inconclusive results, call for further research

Main results Fixations on brand and picture facilitate brand memory

Clustered insights

Print advertisements with small changes Physical, varied shelf displays

Experimental print advertisements

Stimulus material Experimental advertisements

Eye tracking methodology ROIs ROI definition Metrics Unspecified Mean gaze Headline, duration picture, text, pack shot Unspecified Total gaze Headline, duration, picture, text, skipped pack shot elements Unspecified Total fixation Warning duration, messages, Percentage overall ad attending n.a. n.a. Locations of fixations, durations and numbers Relationship between recall and eye tracking measures Choice process with three stages identified

ANOVAs

Main results Groups of customers with distinct patterns of attention identified Repetition decreases attention, scan path stays unaffected

ANOVAs

ANOVAs, MANOVA, Chi‑Square test

Statistics Latent class regression

ACR = Advances in Consumer Research, IJRM = International Journal of Research in Marketing, JA = Journal of Advertising, JAR = Journal of Advertising Research, JCR = Journal of Consumer Research, JIM = Journal of Interactive Marketing, JM = Journal of Marketing, JMR = Journal of Marketing Research, JPIM = Journal of Product Innovation Management, MS = Marketing Science

Topic Effects of physical ad properties on attention Impact of repetition of ads and motivation on attention Krugman et al. Adolescents’ 1994 (JAR) attention towards warnings on ads for cigarettes Choice process Russo & for consumer Leclerc 1994 goods subject to (JCR) investigation

Author(s), year, (journal) Rosbergen et al. 1997 (JCR) Pieters et al. 1996 (ACR)

International Journal of Market Research Vol. 55 Issue 1

111

Clustered insights

opposing manners to the overall ROI, but remain unnoticed. Moreover, such areas within the ROI unrelated to the experimental manipulation might add noise to the underlying signal (see Heller et al. 2006). Thus, misspecification and/or variability in defining ROIs clearly, threatens the reliability and generalisability of eye tracking results. Even worse, the exact definition of the ROIs remains rather unspecified in most academic works. The particular form of an ROI (e.g. the width and any potential overlap with other ROIs) then can only be guessed by the reader. Still, for a reliable interpretation of reported effects and replication by other researchers, this type of knowledge is necessary. A second concern with ROI analysis involves how recorded no-gaze points are handled. While traditional ROI methods do not consider such data points, one cannot assume that the viewer did not observe the object simply because no gaze point was recorded. For example, in one advertising study, participants remembered advertisements that they had not actively looked at (Kuisma et al. 2010), which suggests that peripheral perception mechanisms are also at work, in accordance with physical characteristics of the human eye (see Irwin 1992). The pure vicinity of a gaze point to an ROI might thus be sufficient. The use of ROIs as a decision criterion in eye tracking research, however, does not account for this and might lead to false conclusions. Such a focal area bias is even more likely to occur when ROIs are tightly fitted around a specific stimulus feature. The third and perhaps most important limitation of ROI analysis is that its effects can be discovered only if they have been previously hypothesised by the researcher. Unexpected effects are unlikely to be identified, unless they exist within the predetermined ROI. While, at first glance, this may not seem a limitation, there is little doubt that details about the viewer’s perception and attention are likely to be missed with ROI analysis. In particular, high amounts of data reduction associated with ROI analysis may distort conclusions drawn from the data and may overlook important physiological details. In other words, ROI analysis focuses on whether one’s gaze falls into predefined regions, rather than examining what the user actually sees within the region. Such a constraint suggests that ROI analysis may actually miss the central benefit of eye tracking methods in research (see Pomplun et al. 1996). This final limitation is true for practically all studies that rely solely on ROI-based eye tracking data analysis and has in part led to the development of heatmaps (which have had strong appeal to marketing research practitioners; Bojko 2009). In summary, there are clear limitations associated with the use of ROI as the basis of eye tracking methodology. Despite these limitations, and as

112

International Journal of Market Research Vol. 55 Issue 1

noted before, ROI analysis is heavily used by researchers in marketing and other fields. Given the increasing use of eye tracking methods by marketing practitioners and researchers, an alternate method that addresses the previously noted concerns would be useful.

Spatiotemporal scan statistics: an alternative eye tracking data analysis The primary goal of the current research is to develop a new method for analysing eye tracking data. Before doing so, we provide a summary of a related (but distinct) approach to visualising such data. Heatmaps or attentional landscapes, as first proposed by Pomplun et al. (1996), were developed to resolve some of the issues associated with ROIs by visualising what the viewer actually sees. Heatmaps approximate the human field of vision by drawing Gaussian distributions around registered gaze points, summing them up, and colouring or blurring the particular heights of the resulting map. Heatmaps visualise complex eye tracking data in an intuitive and visually compelling way and thus enjoy widespread use in practical eye tracking research (e.g. Hamaekers & Laan 2011). There is at least one notable limitation to heatmaps (Bojko 2009), namely the lack of any statistical criterion associated with the technique. This limitation makes it difficult to use this tool in quantitative research and often relegates the use of this technique to data visualisation. Therefore, researchers interested in testing theory are left in a difficult position, given that heatmaps do not provide any mechanism for testing hypotheses, and metrics based on ROI analysis might bias results due to the previously detailed concerns. In the current research, we address this situation by introducing an entirely new approach to analysing eye tracking data that makes use of spatiotemporal scan statistics. Scan statistics are widely used in epidemiological research to identify disease clusters along a temporal dimension and in geographical space at the same time (see Kulldorff et al. 1998). While scan statistics are not explicitly limited to the analysis of epidemiological data (Kulldorff 1997), use in other contexts is rare. Cluster-based scan statistics have been used successfully to analyse fMRI data and were introduced as an alternative to traditional voxel-by-voxel analysis (see Heller et al. 2006). The application of these statistics to eye tracking data has never been tested, even though the implementation in this context is promising. Applying the scan statistic to data assumes that a point process can account for the distribution of previously observed data, such as the Bernoulli

113

Clustered insights

model. In a typical epidemiological use of the technique, a researcher observes a population of people that live in a specific geographical space, and records disease information for those people for a certain period of time. A typical research question is whether a specific disease occurred more often in particular regions of the area (e.g. in big cities, near rivers), or to a particular point in time (e.g. summertime or winter), or both (see Kulldorff et al. 1998). The spatiotemporal scan statistic provides an answer to this question by calculating a likelihood ratio for several regions of the test space, indicating whether the probability of getting a disease is higher (or lower) in the particular region than expected under the null hypothesis, or not. The scan statistic automatically tests many different regions within the space (both geographically and temporally), which is usually done by a circular scanning window that systematically roams around the test space, varying in size to identify large, as well as smaller, regions with differing probabilities. The researcher can define specific boundaries for the scanning window, such as a maximum window size. From a more technical perspective, random binary counts are approximated by the Bernoulli model, with particular probabilities for the occurrence of cases (individuals with a disease) and non-cases (healthy individuals, respectively). Finally, a likelihood ratio statistic is calculated and significance is evaluated with Monte Carlo simulation (Kulldorff 1997). The spatiotemporal scan statistic can be applied to eye tracking data since the data structure is similar to that of the data used in epidemiological research. In a typical marketing setting (e.g. where stimuli are systematically varied in terms of feature set), gaze points observed under a particular experimental (or baseline) condition are treated as non-cases, while points observed under another experimental variation are treated as cases. Based on the observed eye tracking data, the probability of a gaze point being a case can thus be calculated for a certain region in space; a similar procedure can be conducted for the area outside of that region. Under the null hypothesis, one would assume that cases occur randomly and thus roughly equally often inside a certain region of the whole observation area than outside that region. While, under the alternative hypothesis, the probabilities would differ. From the observed data (observed cases inside and outside a particular region), a likelihood for each region can be calculated, indicating regions where the number of observed cases differs highly from the expected amount and the probabilities are therefore likely to differ. This procedure is applied to the whole study region by moving around a scanning window that systematically varies in space and time. The window also varies in size as it moves across the eye tracking data plane, so that for all points all

114

International Journal of Market Research Vol. 55 Issue 1

possible window sizes are tested (a feature that is convenient for eye tracking data given that the ROI definition may be problematical). As noted earlier, the likelihood ratio statistic is calculated from this process and significance is evaluated via a Monte Carlo simulation process (Kulldorff 1997). Table 2 provides a summary of the key differences between the traditional ROI analysis and the scan statistic. In the current research, the scan statistic is applied to perception of car face designs that systematically vary in terms of design features. Of course, this procedure could be applied to other marketing research contexts – for example, when testing product labels or shelf configurations. Prior to reporting the results of the study, a brief overview of this emerging research topic is provided to equip the reader with the necessary background knowledge. Table 2  Summary of key differences between tracking methods

Common type of study

Reference condition Common type of analysis Number of analyses needed

Size of analysis region Metrics

Data distribution assumptions Data visualisation

Type of eye tracking analysis Classical ROI Spatiotemporal scan Experiment, experimentally Field study, experiment, natural manipulated stimuli or experimentally manipulated stimuli Analysis within a single condition Reference condition needed possible Theory driven, predefined ROI Data-driven and theory driven Separate analyses for spatial and Single analysis for both, temporal differences differences in spatial plane and temporal plane Fixed throughout analysis Variable throughout analysis Number of gaze/fixation Total fixation duration points Number of fixations Mean fixation duration First fixation duration Time to first fixation Normal distribution Binomial distribution Histograms for specific ROI Significance map over whole stimulus region

Experiment Anthropomorphism in product and car design Anthropomorphism – the tendency of people to apply human traits to non-living objects – is emerging as an important design principle that can make products more (or less) appealing and recognisable to customers

115

Clustered insights

(e.g. Aggarwal & McGill 2007). For example, research has examined the influence of car faces on subjective ratings of liking with Landwehr et al. (forthcoming) showing with real market data that certain combinations of affective features of a car face (namely aggressive headlights combined with a friendly grille) are preferred over other combinations, and that such features can impact consumer behaviour. Research on anthropomorphic car designs, or emotional car faces, is particularly interesting for the purpose of the current research, since there is a vast body of basic research suggesting a short-term cognitive processing advantage for threatening faces (see Öhman et al. 2001) that should be measurable with eye tracking protocols. In particular, some studies suggest that threatening faces are processed more accurately and faster than friendly faces, which is often explained with theories on biological preparedness (e.g. Öhman et al. 2001). In accordance with these theories, there is evidence from research on phobics that threatening stimuli are fixated on more rapidly, yet avoided in the longer term (Rinck & Becker 2006). Finally, other recent research using eye tracking methods has demonstrated that car faces are looked at in a way comparable to human faces, pointing to the predominant role of headlights (Windhager et al. 2010). Building on these foundations, it is expected that people will avoid threatening design features when forced to look at aggressive anthropomorphic car fronts.

Participants The sample included undergraduate and graduate students (N = 38), with a mean age of 26.8 years and a standard deviation of 11.1 years, including 12 females, who participated in the experiment for partial course credit. All participants had at least corrected-to-normal vision. The within-subject design used in this experiment allowed the use of a relatively small sample.

Stimuli Car faces varied systematically in terms of: headlights (threatening vs friendly; the eyes of a human face); a lower air vent (threatening vs friendly; the mouth) and side air vents (threatening vs friendly; the cheeks) for a set of eight car fronts. Research employing schematic faces provided guidance for creating the car stimuli (Lundqvist et al. 2004). The emotion evoked by the features was assessed in a pretest and could be confirmed for the headlight design and the lower air vent design. Figure 1 provides two examples of completely threatening and completely friendly car faces.

116

International Journal of Market Research Vol. 55 Issue 1

Figure 1 Example stimuli, illustrating the different design variants for headlights, side air vents and lower air vent Note: the car front on the left shows the threatening features, the car front on the right the friendly features.

Procedure In the main experiment using a within-subject design, each image from the picture set was presented to participants in a random order. Each car image was shown for 10 seconds and participants rated each picture on several scales directly after viewing. Before starting the experiment, participants were acclimated to the eye tracker and a standard calibration procedure was carried out. For the collection of eye tracking data, a Tobii X120 standalone eye tracking unit was used. The unit captured the data with an accuracy of 0.5° at a sampling rate of 120 Hertz. Stimuli were presented on 19-inch Samsung SyncMaster 913TM with a screen resolution of 1,280 × 1,024 pixels. Each picture covered 63% of the screen’s width and 52% of the height of the screen, so that the car faces resembled real car fronts seen from a distance of approximately 5.5 metres. Participants were seated about 55 centimetres away from the screen. Head movements were allowed in a box with a side length of roughly 30 centimetres. In a within-subject design, eye tracking data for all participants were recorded for each of the stimuli (which occurred in a randomised order). Trials were indicated with trigger signals in the data output; if 20% or more of the data points were missing, participants were excluded, as were participants who had more than 200 missing data points in a row. Overall, nine data sets were excluded (final sample, N = 29 participants).

Calculation and parameters for the spatiotemporal scan statistics Raw data for all participants, including tracking coordinates (x, y) every 8.3 milliseconds over the total time interval of 10 seconds, were combined

117

Clustered insights

for each experimental condition. To reduce the calculation time for the scan statistic, a spatiotemporal grid (with a resolution of 320 squares, in time steps of 200 milliseconds) was extended over the data structure, and gaze frequencies were calculated for each of the grid points. In order to calculate the scan statistic based on a Bernoulli model, cases and controls needed to be defined. Specifically, all conditions containing a certain variant (e.g. a non-threatening headlight) were merged and treated as cases, whereas all of the other conditions (e.g. with a threatening headlight) were merged and treated as controls. The scan statistic was calculated using SaTScan (v9.0 Martin Kulldorff) with a space–time retrospective analysis being performed using the Bernoulli model as a discrete scan statistic. The statistical test was set to scan for high and low rates, and clusters which expanded over the whole time span were allowed. In all other aspects, the default settings of the software were used. Figure 2 provides a visual representation of the data aggregation process and scan procedure. In the current research, clusters with the largest likelihoods are reported first. To reduce the complexity of the output and to take into account the physical constraints of human vision, two reporting criteria were employed. First, no clusters are reported that have centres in other clusters; this situation may arise since the scanning window automatically tests for different positions and window sizes, which could result in smaller significant clusters within bigger regions of significance. Second, cluster sizes were restricted to approximately 5° of the human visual field. Besides the most likely (or primary) clusters, several secondary clusters were also identified. Only those with the absolute largest difference in gaze density between two tested conditions are considered herein since these regions are of elevated practical interest, as this indicates regions on the stimulus surface, where participants look often in one condition and only rarely in the other. For visual reporting purposes, the most likely clusters are reported with spatial coordinates of circles defined through x, y and radius r. Moreover, the time frame of the cluster (ranging from 0 to 10 seconds) is reported, as well as the number of total gaze points in a particular cluster across all conditions. In terms of metrics, relative risk (RR) indicates whether in the case conditions there are more (indicated when RR > 1) or fewer (indicated when RR < 1) points likely to be observed in a particular region. The log likelihood ratio (LLR) is also reported. In the plots, clusters are drawn with the identified circle parameters. Clusters with RR > 1 are marked with a ‘+’ in their centre and clusters with RR < 1 are marked

118

International Journal of Market Research Vol. 55 Issue 1

Data aggregation Threatening headlights

Scan procedure

1

Cases

2 …

3 N Participant

Non-threatening headlights

1 2 3 …

Non-cases

Stimulus groups

N Figure 2 Schematic diagram of analysis procedure Note: eye-tracking data for all stimuli were collected for each of the participants in a within-subject design. Data were grouped with respect to a particular design feature, and divided into ‘case’ and ‘non-case’ observations. By moving a scanning window (systematically varying in size) across the data plane, the spatiotemporal scan statistic identifies significant clusters on the base of the case to non-case ratio. In the graph, grouping for the comparison ‘threatening headlights against non-threatening headlights’ is shown (other grouping procedures were performed in an analogous fashion).

with a ‘–’. The transparency of a cluster was defined by the absolute value of the difference between the number of expected cases and the observed number of cases in a cluster.

Results ROI analysis Two sets of ROIs were defined: a set with a narrow definition, where ROI-rectangles were drawn as closely around the car features as possible, and a set with a wider definition, where ROI-rectangles were twice the

119

Clustered insights

size as the previous but did not overlap (see Figure 3 and Table 3). ROIs for the left and right headlights were conjoined for analysis as these can be considered as a single unit of design. For the calculation of fixations, the inbuilt Tobii fixation filter was used with a fixation radius of 35 pixels (see Lamm et al. 2010). As typically done, a series of ANOVAs was calculated for each ROI. For each model, the stimulus features (i.e. headlight, side air vents or lower air vents) served as the independent variables. The total fixation duration (TFD) for each feature served as the single dependent variable for each of the ANOVA models. For the wider ROI definition, in the ROI of the enlarged headlight region, there were significant main effects for the headlight design [F(1,28) = 7.219, p = 0.012] and the side air vents design [F(1,28) = 11.229, p = 0.002]. In particular, threatening headlights led to shorter fixation durations, while the threatening side air vents led to longer fixation durations. Other effects did not reach significance (ps > 0.370). The design of the side air vents had a significant effect on the enlarged lower air vent region, with the threatening design lowering the TFD

Figure 3 ROI definitions for the stimulus Note: on the left side, the wide definition is shown, and on the right side, the narrow definition is superimposed.

Table 3 Descriptive statistics for total fixation durations across all experimental conditions and relative ROI sizes Wide ROI Headlights

Narrow ROI

Size (%)

m (s)

sd

Size (%)

m (s)

sd

2.3

1.90

0.82

1.3

1.38

0.87

Side air vents

2.2

0.25

0.32

1.0

0.15

0.23

Lower air vent

3.7

0.74

0.89

2.1

0.50

0.69

120

International Journal of Market Research Vol. 55 Issue 1

[F(1,28) = 10.476, p = 0.003]. All of the other variants had no significant effect (ps > 0.159), and particularly, no direct effect of the lower air vents design could be observed. For the narrow ROI definition, the directions of the reported effects were similar to the wider ROI definition. Thus, the directions are not reported. A significant effect of the side air vents design was found on the TFD in the headlight region [F(1,28) = 8.951, p = 0.006], whereas the other design variations did not display a significant effect (ps > 0.136). Regarding the lower air vent region, a significant effect of the side air vent design was found [F(1,28) = 7.567, p = 0.010], and the other variants did not reach significance (ps > 0.130).

Spatiotemporal scan statistics Comparing all threatening headlight conditions against the non-threatening headlight conditions, a most likely cluster was identified located on the right headlight (x = 864, y = 352, r = 90.51, time frame = 0–10 s; population = 47,990; RR = 0.88; LLR = 295.515; p = 0.001). A secondary cluster was identified in the hood region (x = 672, y = 288, r = 90.51, time frame = 0–5 s; population = 19,443; RR = 0.88; LLR = 123.994; p = 0.001). The spatial positions of these two clusters, as well as the other identified clusters, are visualised in Figure 4. This figure also shows positions of the identified clusters with respect to the narrow ROI definitions from the ROI analysis. Comparing conditions with a threatening lower air vent (or mouth), the most likely cluster was detected close to the centre of the lower air vent (x = 608, y = 544, r = 90.51, time frame = 5–10 s; population = 13549; RR = 0.81; LLR = 238.149; p = 0.001). The secondary cluster with the highest population was found on the grill (x = 672, y = 416, r = 64.00, time frame = 5–10 s; population = 9,177; RR = 1.22; LLR = 208.383; p = 0.001).

Comparison of ROI analysis and spatiotemporal scan statistics As expected, the narrow ROI definition led to fewer and less significant results, in comparison to a wider definition. The only direct effect of a feature on the corresponding ROIs was observed in the wide ROI condition, where threatening headlights resulted in a lower total fixation time. This finding corresponds with theoretical assumptions based on processing speed and avoidance of threatening stimuli (e.g. Rinck & Becker

121

Clustered insights

600

400

y

200

0

600

400

y

200

0

2006), but differences between ROI definitions illustrate the vulnerability of ROI-based analyses. While general results from the ROI analysis were confirmed with scan statistics, important differences emerged. For example, by comparing the threatening headlight conditions with all other conditions, a cluster on the right headlight was identified with the scan statistics that was looked at less often across the entire timeframe. The situation was less

0

200

400

600 x

800

1000

1200

Figure 4 Significant clusters from the scan analysis Top: all conditions with threatening headlights tested against all of the other conditions. Bottom: all conditions with the threatening lower air vent tested against all of the other conditions.

122

International Journal of Market Research Vol. 55 Issue 1

clear for the left headlight, where both a small, less-fixated cluster as well as a larger, more-fixated cluster were observed. Both clusters are clearly in the vicinity of the headlight and in part overlapping with the ROI definitions for the left headlight. Moreover, as indicated by the transparency in Figure 4, the absolute value of gaze differences is largely smaller in these two secondary clusters. This demonstrates that treating the headlights as a single ROI, while theoretically reasonable, might not be adequate for the ongoing gaze processes, a finding that easily would have been missed by relying on ROI analysis. Moreover, this serves as an example of a possible subregion bias that could have affected findings in the ROI analysis and would not have been discovered by the classical method. The ROI finding that the threatening design of the lower air vent exerted no visual effect needed to be revised based upon the results of the scan statistic. In fact, the spatiotemporal scan identified a most likely cluster with fewer gaze points (than expected as compared to control conditions) directed to the mouth region of the car face in the second half of the time interval. The cluster was positioned in the centre of the feature. The ROI analysis would have led to a fundamentally different interpretation since observed effects overlapped with portions of the stimulus that did not show the effect, and thus these regions added noise to the statistical comparison. Again, one can conclude that this threatening feature design led to fewer gazes, which in turn supports the initial findings concerning the processing of threatening features. Overall, the spatiotemporal scan analysis provided a more precise description of the gaze pattern and thereby extends findings from ROI analysis. In addition, it should be noted that the advantages of eye tracking data analysis with the spatiotemporal scan statistic directly address the weaknesses ascribed to ROI-based analyses, as will be pointed out in more detail in the discussion.

Discussion In the current research, we introduced and tested a novel method for analysing eye tracking data via spatiotemporal scan statistics. The new method, which has not been applied to eye tracking data previously, not only identified statistically distinct clusters that were suitable for hypothesis testing, but also enabled an unbiased exploratory look at complex eye tracking data. While several limitations exist for classical eye tracking methods based on the definition of ROIs, these problems do not

123

Clustered insights

occur with spatiotemporal scan analysis. In our study, results derived from the ROI analysis were at least partially confirmed by the new method, but clear differences emerged with greater details emerging from the scan statistic.

Benefits of spatiotemporal scan statistics The advantages of our approach relate to the critique raised earlier about ROI-based analysis. Regarding the first concern, the existence of subregions within an ROI does not directly affect the statistical extractions of differing gaze clusters with the scan analysis, while it may distort ROI-based statistics. This situation is clearly illustrated in our research. Moreover, with the exception of extraction parameters in the scan procedure (which can easily be reported and replicated), the scan statistic does not rely on any subjective judgements of the researcher (as does the ROI method) and therefore could lead to more standardised procedures. In terms of the second ROI limitation (i.e. the perception of non-fixated objects), scan statistics are clearly able to indicate if gaze differences occurred in the vicinity of a feature of interest. While object perception with no recorded gaze points at all is likely to be a problem for any eye tracking method, the identification of nearby gaze differences using scan statistics can clearly help resolve identification problems related to the visual field of acuity. Indeed, certain research questions could be assessed in this way by looking for gaze clusters at a certain distance from the ‘to-be-perceived’ object, which would obviously exceed the possibilities of ROI-based analysis. Regarding the third concern, scan statistics can be implemented to visualise complex eye tracking data in a comprehensive manner. Such an approach allows researchers to gain insight into participants’ gaze behaviour beyond what would be typically hypothesised and operationalised with ROIs. In our example, this idea proved useful when examining the design features of the automobile’s side air vent. Eye tracking data analysed via scan statistics allows researchers to identify important gaze differences that may not have been proposed to exist a priori. Unlike heatmaps, which have initially been proposed to plot ‘what the viewer actually did see’, the new method provides a statistical criterion for what was observed. Finally, the new method provides several additional advantages over ROI-based methods. In particular, spatiotemporal scan analyses provide

124

International Journal of Market Research Vol. 55 Issue 1

(in a single procedure) information about several positions within a stimulus wherein significant clusters can be found; ROI-based analyses can accomplish such a task only by calculating multiple ANOVAs (with adjustments for alpha inflation). On a similar note, the new procedure not only incorporates spatial dimensions, but also considers the timeline of the gaze data – a dimension that would otherwise require separate statistical testing.

Implications for product design Consistent with theoretical expectations, threatening features are looked at less often when compared to non-threatening features in the long term. This finding not only supports previous literature on biological preparedness (see Öhman et al. 2001; Rinck & Becker 2006), but also shows that these evolutionary mechanisms can be triggered by anthropomorphic product design as reflected by eye tracking methods. Overall, anthropomorphic product design is capable of eliciting specific viewer reactions, as was demonstrated in our study regarding the effect of threatening car faces on viewers’ gaze patterns. Product designers, as well as advertisers, should be aware of this and comparable mechanisms, as the success of certain products might be impacted by their ability to capture viewers’ attention. Future research might address how mechanisms of biological preparedness relate to anthropomorphic product design and how these mechanisms can be effectively employed on a market scale.

Limitations and future research The use of spatiotemporal scan statistics on eye tracking data is new, so several aspects of this method could use additional research to improve the tool’s application. As with any approach driven by exploration (e.g. searching for gaze clusters on a stimulus surface that correlate with the measured customer variables), findings may lead to spurious correlations and caution should be exercised (a thorough discussion of this issue regarding fMRI data is provided by Vul et al. (2009)). As a consequence, implementation of scan statistics (at least for academic theory testing) should be hypothesis driven and based on solid research expectations of where to look for significant gaze clusters. This point is illustrated in the current research, where we plotted theoretically defined ROIs into the plots to guide interpretation derived from the scan analysis.

125

Clustered insights

Future research could implement the proposed scan statistic in other marketing research applications. This eye tracking method may be especially useful for research on product labels, package design or feature composition, where an experimental approach (similar to the one used in this research) could be implemented. In particular, scan statistics would prove useful where gaze patterns, as well as gaze counts, change over space and time in response to subtle changes in stimuli. For example, packaging and shelf placement could be examined by studying the impact of low-level pictorial features (e.g. luminance and contrast) on attention capture and direction of gaze. Finally, the new method could be extended to other experimental designs, where viewers themselves decide on the fixation duration of the stimulus, or where the stimulus itself is dynamic rather than static.

Summary In summary, the current work helps move us closer to answering the age-old question: ‘What does the viewer really see?’ This question initially led Pomplun et al. (1996) to the development of heatmaps, and now has motivated the current work on scan statistics, which can provide statistically reliable results. The use of spatiotemporal scan statistics is not only attractive to academic research in marketing, as it provides a method capable of countering many weaknesses of ROI-based analysis, it is also capable of improving practitioners’ work in the field of marketing – a field where the use of heatmaps is widespread.

References Aggarwal, P. & McGill, A.L. (2007) Is that car smiling at me? Schema congruity as a basis for evaluating anthropomorphized products. Journal of Consumer Research, 34, 4, pp. 468– 479. Aribarg, A., Pieters, R. & Wedel, M. (2010) Raising the BAR: bias adjustment of recognition tests in advertising. Journal of Marketing Research, 47, 3, pp. 387–400. Baraggioli, F. & Brasel, S.A. (2008) Visual velocity: content font effects and incidental online ad exposure. Advances in Consumer Research, 35, pp. 600–606. Bix, L., Bello, N.M., Auras, R., Ranger, J. & Lapinski, M.K. (2009) Examining the conspicuousness and prominence of two required warnings on OTC pain relievers. Proceedings of the National Academy of Sciences of the United States of America, 106, pp. 6550–6555. Bojko, A. (2009) Informative or misleading? Heatmaps deconstructed. Proceedings of the 13th International Conference on Human–Computer Interaction, pp. 30–39.

126

International Journal of Market Research Vol. 55 Issue 1

Bojko, A., Gaddy, C., Lew, G. & Quinn, A. (2005) Evaluation of drug label designs using eye tracking. Proceedings of the Human Factors and Ergonomics Society (HFES) 49th Annual Meeting, pp. 1033–1037. Brasel, S.A. & Gips, J. (2008) Breaking through fast-forwarding: brand information and visual attention. Journal of Marketing, 72, 6, pp. 31–48. Buscher, G., Cutrell, E. & Morris, M.R. (2009) What do you see when you’re surfing? Using eye tracking to predict salient regions of web pages. Proceedings of the 27th International Conference on Human Factors in Computing Systems, pp. 21–30. Chandon, P., Hutchinson, J.W., Bradlow, E.T. & Young, S.H. (2009) Does in-store marketing work? Effects of the number and position of shelf facings on brand attention and evaluation at the point of purchase. Journal of Marketing, 73, 6, pp. 1–17. Cutrell, E. & Guan, Z.W. (2007) What are you looking for? An eye-tracking study of information usage in web search. Proceedings of the Conference on Human Factors in Computing Systems, 1 & 2, pp. 407–416. Duchowski, A.T. (2002) A breadth-first survey of eye tracking applications. Behavior Research Methods, Instruments & Computers, 34, 4, pp. 455–470. Duchowski, A.T. (2007) Eye Tracking Methodology. Theory and Practice (2nd edn). London: Springer. Feiereisen, S., Wong, V. & Broderick, A.J. (2008) Analogies and mental simulations in learning for really new products: the role of visual attention. Journal of Product Innovation Management, 25, 6, pp. 593–607. García, C., Ponsoda, V. & Estebaranz, H. (2000) Scanning ads: effects of involvement and of position of the illustration in printed advertisements. Advances in Consumer Research, 27, pp. 104–109. Hamaekers, W. & Laan, R. (2011) Getting into the real world of the shopper. Using eye tracking in a multi mode research approach. Paper presented at the ESOMAR World Research Conference, Insights. Heller, R., Stanley, D., Yekutieli, D., Rubin, N. & Benjamini, Y. (2006) Cluster-based analysis of fMRI data. NeuroImage, 33, 2, pp. 599–608. Hervet, G., Guérard, K., Tremblay, S. & Chtourou, M.S. (2010) Is banner blindness genuine? Eye tracking internet text advertising. Applied Cognitive Psychology, doi: 10.1002/ acp.1742. Irwin, D.E. (1992) Visual memory within and across fixations, in Rayner, K. (ed.) Eye Movements and Visual Cognition. Scene Perception and Reading. New York: Springer, pp. 146–165. Kuisma, J., Simola, J., Uusitalo, L. & Öörni, A. (2010) The effects of animation and format on the perception and memory of online advertising. Journal of Interactive Marketing, 24, 4, pp. 269–282. Kulldorff, M. (1997) A spatial scan statistic. Communications in Statistics. Theory and Methods, 26, 6, pp. 1481–1496. Kulldorff, M., Athas, W.F., Feuer, E.J., Miller, B.A. & Key, C.R. (1998) Evaluating cluster alarms: a space–time scan statistic and brain cancer in Los Alamos, New Mexico. American Journal of Public Health, 88, 9, pp. 1377–1380. Lamm, C., Meltzoff, A.N. & Decety, J. (2010) How do we empathize with someone who is not like us? A functional magnetic resonance imaging study. Journal of Cognitive Neuroscience, 22, 2, pp. 362–376. Landwehr, J.R., McGill, A.L. & Herrmann, A. (forthcoming) It’s got the look: the effect of friendly and aggressive ‘facial’ expressions on product liking and sales. Journal of Marketing.

127

Clustered insights

Lundqvist, D., Esteves, F. & Öhman, A. (2004) The face of wrath: the role of features and configurations in conveying social threat. Cognition and Emotion, 18, 2, pp.  161–182. Meißner, M. & Decker, R. (2010) Eye-tracking information processing in choice-based conjoint analysis. International Journal of Market Research, 52, 5, pp. 591–610. Öhman, A., Lundqvist, D. & Esteves, F. (2001) The face in the crowd revisited: a threat advantage with schematic stimuli. Journal of Personality and Social Psychology, 80, 3, pp. 381–396. Pieters, R. & Wedel, M. (2004) Attention capture and transfer in advertising: brand, pictorial, and text-size effects. Journal of Marketing, 68, 2, pp. 36–50. Pieters, R. & Wedel, M. (2007) Goal control of attention to advertising: the Yarbus implication. Journal of Consumer Research, 34, 2, pp. 224–233. Pieters, R., Rosbergen, E. & Hartog, M. (1996) Visual attention to advertising: the impact of motivation and repetition. Advances in Consumer Research, 23, pp. 242–248. Pieters, R., Wedel, M. & Batra, R. (2010) The stopping power of advertising: measures and effects of visual complexity. Journal of Marketing, 74, 5, pp. 48–60. Pomplun, M., Ritter, H. & Velichkovsky, B. (1996) Disambiguating complex visual information: towards communication of personal views of a scene. Perception, 25, 8, pp. 931–948. Rinck, M. & Becker, E.S. (2006) Spider fearful individuals attend to threat, then quickly avoid it: evidence from eye movements. Journal of Abnormal Psychology, 115, 2, pp.  231–238. Shao, W., Lye, A. & Rundle-Thiele, S. (2008) Decisions, decisions, decisions. Multiple pathways to choice. International Journal of Market Research, 50, 6, pp. 797–816. Teixeira, T.S., Wedel, M. & Pieters, R. (2010) Moment-to-moment optimal branding in TV commercials: preventing avoidance by pulsing. Marketing Science, 29, 5, pp. 783– 804. Thomas-Smith, C. & Barnett, G. (2010) Seeing is believing. Viewing engagement in placebased media. Paper presented at the ESOMAR World Research Conference, WM3. Van der Lans, R., Pieters, R. & Wedel, M. (2008) Competitive brand salience. Marketing Science, 27, 5, pp. 922–931. Vul, E., Harris, C., Winkielman, P. & Pashler, H. (2009) Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspectives on Psychological Science, 4, 3, pp. 274–290. Wedel, M. & Pieters, R. (2000) Eye fixations on advertisements and memory for brands: a model and findings. Marketing Science, 19, 4, pp. 297–312. Windhager, S., Hutzler, F., Carbon, C.C., Oberzaucher, E., Schaefer, K., Thorstensen, T. et al. (2010) Laying eyes on headlights: eye movements suggest facial features in cars. Collegium Antropologicum, 34, 3, pp. 1075–1080. Wooding, D.S. (2002) Eye movements of large populations: II. Deriving regions of interest, coverage, and similarity using fixation maps. Behavior Research Methods, Instruments & Computers, 34, 4, pp. 518–528. Yesilada, Y., Jay, C., Stevens, R. & Harper, S. (2008) Validating the use and role of visual elements of web pages in navigation with an eye-tracking study. Proceedings of the 17th International Conference on World Wide Web, pp. 11–20. Zhang, J., Wedel, M. & Pieters, R. (2009) Sales effects of attention to feature advertisements: a Bayesian mediation analysis. Journal of Marketing Research, 46, 5, pp. 669–681.

128

International Journal of Market Research Vol. 55 Issue 1

About the authors Christian Purucker is Research Associate at the University of Würzburg, Germany. He received his Diploma in Psychology at the University of Würzburg in 2009 with a focus on methodology and neuropsychology. In 2012, he earned a PhD in management with a focus on marketing from the University of St. Gallen, Switzerland. His research interests include consumer psychology, product perception and human–machine interaction. To date, he has conducted various investigations in these fields employing a range of methodological approaches, including eye-tracking and cross-cultural survey methodologies. Jan R. Landwehr is Professor of Marketing and holds the Chair for Product Management and Marketing Communications at Goethe University Frankfurt, Germany. He received a diploma degree in psychology from the University of Würzburg, Germany, and a PhD in marketing from the University of St. Gallen, Switzerland. His research focuses on product design/aesthetics, symbolic communication, sustainable consumer behaviour, and the genesis of emotional preferences. His research has been published or is forthcoming in journals such as Marketing Science, Journal of Marketing, Information Systems Research, and in a number of conference proceedings. David E. Sprott is Associate Dean for Graduate, International and Professional Programs and the Boeing/Scott and Linda Carson Chaired Professor of Marketing at Washington State University, College of Business. He received his Bachelor of Business Administration and MBA degrees from Kent State University. He earned a PhD in marketing, with an emphasis on psychology and consumer decision making, from the University of South Carolina in 1997. Professor Sprott’s research interests include various issues related to consumer decision making, social influence, information provision and marketing public policy. His research has been published in the field’s top journals such as the Journal of Consumer Research, Journal of Marketing, Journal of Marketing Research, Journal of Consumer Psychology, Journal of Retailing, and at various international conferences. Andreas Herrmann is Professor of Marketing and the Director of the Centre for Customer Insight at the University of St. Gallen, Switzerland. He graduated from the Coblenz School of Corporate Management (WHU) in business administration. After further study at WHU and the Kellogg Graduate School of Management, he received his PhD in 1991. He was appointed the chair of business administration and marketing at the University of Mainz in 1997 and moved to the University of St. Gallen

129

Clustered insights

in 2000. His primary research interests include market-orientated product design, pricing, behavioural economics and brand management. Andreas Herrmann has published 15 books and more than 250 scientific papers and articles in leading international journals such as the Journal of Marketing, the Journal of Marketing Research and Marketing Science. Address correspondence to: Dr Christian Purucker, Chair of Psychology III, University of Würzburg, Röntgenring 11, 97070 Würzburg, Germany. Email: [email protected]

130

Copyright of International Journal of Market Research is the property of Warc LTD and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.