Gaze Data Visualization Tools: Opportunities and ...

Download PDF
3 downloads 0 Views 324KB Size Report
Comment
saccades. Figure 1-Part C shows points belonging to .... In William S. Cleveland and M. E. McGill, editors, Dynamic. Graphics .... M.T. Maybury and W. Wahlster,.
Gaze Data Visualization Tools: Opportunities and Challenges Rameshsharma Ramloll National Rehabilitation Hospital, Washington D.C. 20010, {[email protected]}

Cheryl Trepagnier, Marc Sebrechts

Jaishree Beedasy

Department of Psychology, Catholic University of America, {[email protected]}

Faculty of Engineering University of Mauritius, {[email protected]}

sustained broadening diversity of communities interested in this technology is noteworthy. Eye tracking has typically been a means to study eye movements because of its relevance to cognition, to develop novel computer input devices [20] or to develop gaze contingent displays for entertainment [40]. Recently, eye tracking is being investigated in rehabilitative therapies where the aim is to train gaze behavior in autistic children [42][36].

Abstract

Addressing data visualization challenges typically involves applying lessons from visualization theory to inform design and implementation approaches. This process is shaped to a large extent by the availability of tools that are aimed at enabling visualization designers to focus on visualization design rather than on low-level software engineering. Recently, such tools have become powerful enough to be used effectively. We discuss the ideation process informing our design approach and describe the use of Macromedia Flash MX 2004 for the rapid prototyping of a gaze data visualization tool. We highlight selected gaze data visualization ideas to illustrate the most innovative aspects of our design. In particular, we explain our strategy to reveal the underlying mechanisms that produce the summarizing visual constructs and why this is important. We introduce a new technique for visualizing gaze data for dynamic stimuli. The novelty of this approach is that it avoids the traditional frame-by-frame analyses typically carried out for such stimuli. Keywords: gaze data, visualization tools, saccade, fixation, scan-path, direct manipulation, animation, sonification, in-context visualization, and nets

1.2 Typical gaze data types of interest The first stage of the design of our visualization tool involves the identification of the typical gaze data types of interest to researchers. Table 1 and Table 2 show some basic data types that gaze data analysts have voted to be of interest [24]. It is to be noted that the Tables are not exhaustive and only the most commonly used data types have been presented. Table 1 First and second order gaze data

1. Introduction 1.1 Why study gaze data?

First Order Data x, y, (z) data

Second Order Data Fixations

Pupil diameter

Saccades

Blink rate

Pursuit eye movements

Table 2 Third and fourth order gaze data

Eye tracking technologies have in the recent years experienced a surge of interest in a wide range of research communities as illustrated by the increasing frequency of published research papers on the subject [11]. Eye tracking studies are typically either top-down, informed by cognitive theory or design hypotheses, or bottom-up, analyzed without having prior recourse to theories relating eye movements to cognitive activity. The bottomup camp can be illustrated by the early studies of how people look at pictures [7], how eye movement behavior is influenced by visual stimuli [41] and the more recent studies of how people look at web content [14][16][37]. The top-down camp can be illustrated by recent work investigating the effects of learning on smooth pursuit during transient disappearance of a visual target. Here, the driving motivation is to discover whether observed effects are the result of changes in visual or motor processing [29]. Another top-down example is the examination of novel approaches to predict fixations based on stimuli properties. In this case, the driving hypothesis is that eye movements are believed to be quasi-random and driven by low-level image structure [32]. The use of eye tracking in human computer interface studies has evolved from a promising approach to one that actually delivers results due to improvements in supporting hardware and software [21]. Furthermore, the

Third Order Data Scan-paths

Fourth Order Data Scan-path shape

Total fixation time within areas of interest

Scan-path variability

Matrix of transition probabilities between areas of interest

Scan-path complexity

1.3 Gaze data visualization tools It has been suggested that visualization tools are necessary for facilitating the understanding of large volumes of data because the visual cortex dominates perception and key aspects of the perception process occur rapidly without conscious thought [48]. It is not be surprising that researchers studying gaze behavior should be eager to embrace data visualization tools. This is because eye-tracking experiments typically produce high volumes of data. There has been a steady stream of improvements (Table 3) to analyze and visualize eyetracker data for static 2D stimuli [26][46][47]. Researchers also have been more outspoken about the tools they wish to have to help them in their gaze data analysis tasks and have developed prototypical systems to illustrate their individual approaches [17][37]. Some

1

academic prototypes, such as GazeTrackerTM, have even evolved into commercial systems [26].

from eye tracker hardware implementation processes. This trend can actually be beneficial. This is because the problem of propriety data format results more from political and marketing choices than technical constraints. The division of labor leads to a new software-hardware equilibrium that will pressure visualization tool developers to give high priority to the development of interoperability features necessary to enable their software to access as many raw data formats of eye tracker devices as possible. In this situation, the marketing objectives will meet that of the gaze data visualization tool objectives and the interoperability issues will sort themselves out.

Table 3 A sample of gaze data visualization and analysis tools Commercial ClearViewTM (TobiiTM system) GazeTrackerTM (EricaTM system) DAQTM(ISCANTM system)

Academic Research VizFix [17] WebEyeMapper & WebLogger [37] GazeTrackerTM [26]

The contributions of most of these tools will be presented in the context of our design approach in section 2. Improvements regarding gaze data processing and visualization for dynamic stimuli are still in the early stages. Most techniques are based on frame-by-frame analyses of video data. This process can be time consuming and computationally intensive. So far, the authors have not come across a simple and intuitive way to visualize gaze data for dynamic stimuli.

2. Basic concepts informing our approach We now describe the main elements considered during our ideation process for the creation of the first prototype of our gaze data visualization tool. Our broad goal is to develop human centered visualizations [49] of eye data in order to produce effective discovery tools that will allow eye movement researchers to gain more insight into their data.

1.4 Barriers to progress

2.1 Visualization design theory and technology applied to gaze data visualization

Determining the meaning of eye movements is a difficult problem to solve, as there is very little evidence of gaze behavior that will help distinguish between a meaningless fixation such as an unintentional one and a deliberate or purposeful one [31]. The evaluation of visualizations remains a hard problem. However, some researchers have illustrated that the application of workload assessment methodologies [18] to data comprehension tasks can provide a sound method to distinguish between the effectiveness of various visualizations [35]. Eye-tracker systems are often effective tools only in the hands of specialists who have had significant practice in the use of the technology. An experimenter new to the technology will most frequently find (1) the technology not plug-and-play, (2) the gathering of reliable gaze data problematic because of the lack of data validity sensors such as automatic calibration drift detectors and (3) the making of meaningful inferences from the high volume of gaze data difficult. An outstanding issue with eye-tracker systems is that while tools for processing, visualizing and analyzing gaze data are continuously being developed and improved, these tools are usually tracker system specific partly because the raw data format is frequently proprietary. Thus, it is often the case that custom gaze data processing and visualization software need to be developed from ground-up for the eye-trackers used in many eye data monitoring laboratories. This state of affairs may also lead to duplication of effort for every specific eye-tracker. Typically current major players in the eye tracking community market both hardware and associated software. Thus, the commercially available gaze data visualization and analysis tools are, as might be expected, designed for a given tracker. As the industry matures, there is an increasing tendency for analysis and visualization tool development efforts to get dissociated

It is good practice to base the design of a visualization tool on principles of effective information presentation, graphic design and visual perception [4]. A good initial set of guidelines that can be used to inform the design of the data visualization is the body of perception studies and surveys of graphical practice. For example, Becker and Cleveland’s guidelines could be used as time saving shortcuts to the design of effective visualizations [2]. They ranked various visual perceptual tasks according to the degree of difficulty (1:easy to 7:hard): (1) Position along a common scale (easy), (2) Position along identical, nonaligned scales, (3) Length, (4) Angles and slopes, (5) Area, (6) Volume, (7) Color and density (hard). Glyphs, graphical entities that convey one or more data values via attributes [43], are likely to be an adequate technique to represent higher order gaze data as described in Table 2 and Table 3. Typical geometric attributes of glyphs include shape, size, orientation, position, and direction/magnitude of motion, while their appearance attributes include color, texture, and transparency. Experimenting with the design of glyphs resulting from selections of attributes singly or in combinations and evaluating their individual merits is a pre-requisite to the design of an effective gaze data visualization tool.

2.2 Opportunities for direct interaction with visualizations Gaze data visualizations tend to look overcrowded and chaotic if all the points of regard are considered at one time. This problem is exacerbated as the gaze data is collected over extended periods. We discuss how directmanipulation and brushing are techniques that hold significant potential for effective gaze data visualizations.

2

2.2.1 Brushing. Direct manipulation [39] is one of the most important features in information visualization systems. Brushing, for example, is a process where the user can highlight, select, or delete a subset of elements by pointing to the elements with a pointing device [45]. Brushing can be implemented in the context of gaze data visualization by making every representation including that of the very basic element, i.e. the point of regard, interactive so that clicking on it yields say some further detailed relevant information.

centric visualizations is that no assumptions need to be made about the boundary of objects constituting the stimuli. One illustration of gaze data visualization using this technique is the Fixation Map [46], which is a tool for conveying the most frequently fixated areas in an image. This illustrates a third order data visualization described in Table 2. 2.3.1Static stimuli. Analysis of gaze data for static stimuli e.g. pictures is laborious and it is even more so for dynamic stimuli e.g. interactions with objects in virtual 3D environments or video. For static stimuli, the state of the practice is to provide users with the ability to specify the regions of the stimuli they are interested in manually. For example, the experimenter will painstakingly outline the various regions of the face picture to specify the relevant regions of interest, e.g. the nose, left eye, right eye, mouth, cheeks, chin and so on, during an experiment aimed at studying how people look at faces.

2.2.2 Focus and Context. Focus-plus-Context techniques may be used to make sense of the high volume of clustered and overlapping gaze data representations. The limitation of the viewing space leads to the problem that either an overview of all data or a zoomed view of an interesting subspace can be shown at the same time with a uniform (linear) magnification factor. Nonlinear magnification [27] has advantages over traditional (linear) magnification techniques in that it allows magnification of the focus region without overlapping or the need of separated views. The fisheye lens [15] describes a simple form of a distortion technique in which the view is distorted in a manner similar to that produced by a very wide-angle camera lens; the magnification in the middle is higher than the magnification near the border. Bier et al. [5] propose Magic Lens filters that provide many different methods for changing the visual representation of information as the filters pass over the workspace. Filters can be used for increasing or decreasing detail or for altering selected regions of the stimuli in some meaningful way. This approach is already used extensively in the creation of ‘hot spots’ in many commercial gaze data visualizations (for e.g. as in ClearViewTM) where fixations are highlighted by changing the characteristic of the stimuli at the fixation points. These representations are deemed to add context to the visualizations. Within the realm of eye-tracking research, context has also a broader meaning. Land et al. [25] for example refer to “Object-related actions” as a neat way to combine eyetracking data with other participant behaviors such as reaching and manipulation movements. These behaviors in this case are viewed as the context in which the gaze pattern arises. Land et al. were interested in studying the relationship between gaze and activities of daily living. The realization of the needs for such context visualizations will continue to expand the range of visualization functionalities that future tools will have to provide.

2.3.2 Dynamic stimuli. Such manual approach is not scalable for video stimuli and sophisticated feature extraction image processing algorithms need to be used. The most recent version of GazeTrackerTM has started to provide this functionality. The obvious research direction to follow in this area is to develop better object extraction algorithms for video and automate the gaze data processing procedure so that it becomes easier to analyze the data in the context of the objects captured on film. 2.3.3 Gaze data in context with gaze bearing objects. The interest for getting an insight about the gaze behavior of a sample of subjects at a glance and for exploring gaze behavior during more complex interactions, such as that involved during interactions with a computer GUI interface, have highlighted the need for gaze visualizations that are presented in context with the gaze bearing objects. For example, recent systems enable the mapping of fixation points to visual stimuli in some typical dynamic human computer interfaces [8][37]. These systems provide user interface object specific gaze data for user and system initiated display changes such as window scrolling and pop-up messages. At the time of writing, such systems are not yet available commercially.

2.4 Accurate reporting of data quality While significant efforts have been made to encourage researchers to report the quality of their data in eyemovement research [30], very little has been said about how to represent the quality of gaze data so that this information becomes part of standard gaze data visualizations. It is also important for the visualizations of gaze data to include representations of data validity especially when eye-tracker systems are still frequently prone to poor calibrations and calibration drifts. Since the validity of gaze data changes over the stimulus space, it is important for the representation to help distinguish between areas where gaze data validity is acceptable and other areas where the data validity is less so. The TobiiTM

2.3 Object v/s region centric visualization The distinction between region centric and object centric is as follows. In the first case, a fixed region of the screen displaying the stimulus is of interest while in the second case, the selection is that of an object. In the first case, the selected region does not move. In the second case, the selected region moves with the associated object. Both approaches are useful depending on the nature of the study and type of stimuli. The main advantage of region

3

–ClearViewTM system has recently introduced this functionality.

environments into the visualization environment itself. Interactive stimuli such as an interactive prototype of a graphics user interface can be easily be implemented in a Flash MXTM environment and stored in a file format that can be understood by the gaze visualization environment. In our opinion, future gaze visualization environments will not separate stimuli presentation functionality from the gaze data visualization and analysis components. Another advantage of creating interactive or passive visual stimuli in an object oriented and graphics centric environment is the facilitation of gaze data analysis. No specialized image processing operations are required to isolate objects of interest in the scenes and to determine their relationships with gaze data.

2.4.1 Capturing and representing gaze data quality. New ways need to be investigated to integrate gaze visualizations with visual elements that will give an idea of the degree of accuracy of the calibration and the resolution of the eye-tracker. In our experience measuring calibration drifts do not tend to be uniform at various points on the screen, thus such calibration errors are difficult to model. In cases where recalibration can prove to be costly, methods to correct for calibration errors are needed. To achieve this goal, it is important to be able to identify whether errors are systematic or random. Systematic errors can theoretically be corrected if a correction function is found to remap the calculated gaze points. Fuzzy based approaches can be applied to derive this correction function. Visualization tools that enable users to shift gaze data points may help identify whether calibration errors are systematic or random. This strategy is proposed in the VizFix prototype [17][19] keeping in mind that it will probably work only if the systematic error is linear.

2.7 Algorithm animation In our opinion, it is not enough to produce visualizations that reduce the high volume of gaze data to a form that can be easily understood at a glance. It is also important for the user to have whenever possible a view of the process that leads to the resulting visualization. The main reason for opening up the visualization process for users to see the operations of the algorithms driving it is that this may help the viewer understand the summarizing constructs that are often used to describe gaze behavior. For example, it is important that the visualization tool provides an indication of the functioning of the various data summarizing algorithms such as fixation algorithms. This will help the resolution of debates regarding what a particular summarizing construct such as a fixation or a saccade [23] exactly means. Users of visualizations should also be provided with opportunities to see in real-time the effect on changes in algorithm parameters on the summarizing constructs. In particular, new ways need to be developed to illustrate the operation of fixation algorithms in real time and make use of concepts from the current body of information visualization research in both the visual and auditory media to inform our algorithm animation approach. Under such conditions, experimenters may be more motivated to present their results with the necessary detailed information about the parameter settings of the algorithms driving their visualizations.

2.4.2 Better timing measurements. Timing is a very important issue in eye tracking and has a direct impact on the validity of captured gaze data. Even though a wide range of experimental data in eye tracking research report timing measurements to the millisecond level, it is well known that the software clock is not that accurate. The experimenter can request actions, e.g. polling a computer clock, to be done at specific times but it is the operating system that decides when to honor the request. This situation is not acceptable if sub millisecond ocular events need to be studied. Integrating reliable timing modules with gaze data capture tools remains a challenge even though there have been significant improvements in this area. For e.g. the E-prime software [13], which allows timing measurements of millisecond accuracy on a current desktop, can be integrated with the eye tracker system to ensure that the timing accuracy is known.

2.5 Legibility of the visualizations

2.8 Novel technique for avoiding frame-by-frame analysis of gaze data

The labeling of a visualization consisting of high densities of gaze positions is not trivial especially if it needs to be done automatically. This is an important issue more so in the case where the visualization has to be published on a static medium such as paper. Successful approaches used in geographical data mapping research [12] can be investigated to study their applicability to our labeling problem.

Frame-by-frame analysis of gaze data for dynamic stimuli is typically tedious and time consuming. The typical current workflow is to apply real time object extraction image processing algorithms to extract objects of interest and to perform further processing to produce object centric gaze data. We will show further that doing frame-by-frame analysis of dynamic stimuli is not always necessary. While capturing object centric data can be effectively achieved using this method, the later does not lead to a compact and intuitive visualization. The solution we propose provides an alternative to the necessary frame-by-frame approach. Our approach deals with the capture and representation of gaze data obtained

2.6 Powerful stimuli creation and management functionalities One item that stands out on the wish list of experimenters is the availability of functionalities that will enable the loading of a wide range of stimulus files such as pictures, animations and interactive interface

4

during the interaction with 3D (non-stereoscopic) desktop objects. The method we propose also leads to a compact visualization that is as good on paper as on the screen. This method, as it stands currently, is not generic and targets only gaze data visualization problems involving interactions with non-stereoscopic 3D desktop objects.

4. A discussion of selected functionalities of prototype Because of space constraints we will focus on the most innovative aspects of our design namely, (1) the direct interaction with the visualization to meet data comprehension demands, (2) the uncovering of the algorithmic processes driving the visualizations and (3) an alternative way to do gaze data analysis of gaze data for dynamic stimuli which avoids the labor and computational cost of frame-by-frame analysis for a specific class of 3D stimuli namely 3D objects which can developed/flattened easily using geometric projection techniques.

2.9 Interoperability Designers of visualization tools should make every effort to enclose or encapsulate information about the underlying data in an open form that will allow it to pass easily between different computing systems. This approach will certainly provide more opportunities for implementing interoperability.

4.1 Interacting directly with the visualization

3. Choice of development platform and impact of visualization system architecture

Our attempt to integrate brushing functionality into our visualization tool can best be explained by referring to actual examples. The snapshots in Figure 1 were obtained from a first prototype involving the mapping of each relevant gaze data-type to a given graphical layer constructed from a set of MovieClip objects. This mapping facilitates the task of implementing systems that provide the viewer with direct control of the transparency, saliency, zooming level and stacking depth of graphics representing various data types. Other graphical properties can be directly manipulated thereby boosting the potential for experimenting with a range of user initiated visualization optimizations to better address data discovery queries. In the visualization example, based on our prototypical system, (Figure 1-Part A) each point of regard or gaze position is represented with a clickable symbol. After the application of a gaze fixation algorithm, fixation clusters are represented as discs, and saccades are represented by kites.

Once we have decided and prioritized the requirements our design, we identify a development platform that will help us achieve our goals most efficiently. The dominant role that software architecture plays in the design and construction of effective visualization has long been recognized [9]. The foundations of visualization architecture, the quality of the interaction and representation rendering are deeply influenced by the tools available for its implementation. Multimedia software tools that provide the flexibility required to produce powerful interaction possibilities together with high quality rendering quality are just appearing on the market. We target off the shelf tools which in addition to providing the required rendering quality are also likely to decrease the odds of producing visualization systems which are huge monolithic inflexible and locked to a specific data file format. In particular, an environment such as the Flash MXTM Pro 2004 object oriented development can be used for the fast prototyping of 2D gaze data visualization because it is an object-oriented environment with a strong graphics centric architecture. The simplest graphics element of this development environment is the MovieClip object. This element can be nested and combined to produce more complex graphical objects. The MovieClip object has a range of programmatically settable attributes to control its visual appearance and response to most mouse events such as clicking, dragging and hovering among others. Such inbuilt features speed up the prototyping of brushing, linking and zooming functionalities of visualization tools. This development environment also provides in-built facilities that allow data to be stored as Extensible Markup Language (XML). XML can be used to store any kind of structured information, and to enclose or encapsulate information in order to pass it between different computing systems which would otherwise be unable to communicate. This functionality will certainly encourage data visualization tool designers to make their data formats more open, thereby leading to more opportunities for interoperability to emerge.

Figure 1 Different views of the same gaze data In this prototype, the user is able to point and click to select the various data types he or she is interested in. In Figure 1 (Part A, D, E), the arrows show the direction of the scan-path, the color of each arrow is the same as the color of the source gaze data inside a given spatial cluster. Figure 1-Part B shows gaze positions belonging to saccades. Figure 1-Part C shows points belonging to fixation clusters. Figure 1-Part D shows the scan-path. Figure 1-Part E shows in-context representations of fixation clusters by changing the brightness of the stimuli (hot-spot visualization). Each representation can be animated independently to match the way the scan-path develops over time. The speed and direction of the

5

animation is also under the control of the user. The animation of the scan-path as it unfolds provides richer information about the nature of gaze behavior than if the scan-path were presented using a static image. Clicking on any visual element of the visualization also provides more information about that element. For example, clicking on a disc will provide information such as time, location and order in a given cluster.

able to show in real-time, the impact of the choice of fixation algorithms and their parameters on the visualizations produced.

4.2.2 Using sonification to deal with visual clutter. A common problem arising from animations of spatiotemporal processes in a constrained area is the exponential rise of overlapping glyphs which occludes earlier event descriptors or make the emergence of new glyphs difficult to discern. For example, at a fixation, there is typically an aggregation of points of regard that often overlap each other as the duration of the fixation grows. So, when animating a scan-path in our prototype, often, new emerging points of regard would cease to become apparent because they start to occlude other sets of points of the same color that preceded them. A number of researchers have explored the use of sounds in data visualizations [1][33][34][35]. We use auditory events to deal with this problem. A simple approach is to sonify the fixation production and scan-path animations in order to illustrate the various stages of the process. Data sonification guidelines [6], available to inform our sonification strategies, are valuable shortcuts to speed up our prototype development. If the production of a point of regard is associated with a specific sound, then, when the scan-path and fixation constructions are replayed, it is easier to become aware of instances when the next point of regarded is being considered even if the latter is not visually evident. Another example where sonification can help is in the rapid comparisons of fixation scan-paths by inspection. Each fixation could be mapped to a given auditory glyph with pitch mapped to y coordinate of the fixation and lateral stereo position mapped to its x coordinate. Each stimulus will thus be associated with a particular sequence of sounds spaced appropriately to represent a given fixation scan-path. There is a need to evaluate whether such an approach is a viable alternative to compare scanpaths rapidly and reliably as a complementary approach to other abstract analytical tools such as the string-edit method to determine resemblance between sequences [22].

4.2 Revealing underlying algorithmic processes With current multimedia tools available for integration in programming environments, it is increasingly easier to implement visualizations capable of showing in real time the generation of results for each of these fixation algorithms. Briefly, the animations could describe the functioning of a procedure to determine fixations by showing how each gaze position being currently considered is being assigned to some cluster based on its current separation from the center of the currently considered cluster, and how the cluster progresses to a fixation point once its duration exceeds some set minimal threshold. We believe that such animations will provide more opportunities to encourage the user to understand the visualizations rather than just use them. In our prototype, by changing the parameters of the chosen fixation algorithm, the user can see in real-time which points cease to be counted as saccades, which clusters grow and which ones start segregating into different groups. There can be no better argument that this to convince experimenters that their analysis is only meaningful if they report all the details of the algorithms used in their visualizations. 4.2.1 Revealing impact of changes in algorithm choice or parameter modifications. Figures 2 and 3 illustrate the result of two fixation algorithms, the first dispersion based and the second velocity based, on the same set of gaze positions.

4.3 An alternative to frame-by-frame analysis of gaze data for a special class of dynamic stimuli

Figure 2 Double fixations identified

Gaze studies with static 2D stimuli are common. For example, there is a significant interest in the marketing community to study gaze patterns on advertisements [28]. There is however one area which has not received enough research interest so far. This is the visualization of gaze data for interactive 3D (non-stereoscopic) desktop objects. We stress here that we are not discussing gaze depth or vergence [10] in a 3D scene (stereoscopic) on a 2D display. There are a number of online retail sites that allow shoppers to examine products that can be spun around and zoomed for a closer look. There may be some interest in analyzing the gaze patterns of potential customers on such objects in order to identify features

Figure 3 Single fixation identified In Figure 2 the gaze data are identified as two fixations and in Figure 3 only one fixation is found. The filled sector in the bottom left quadrant is a handle that can be clicked to produce a pop-up with details of the relevant fixation. The size of the 3-D ring represents the number of gaze points that constitute the fixation. The point that is conveyed here is that the visualization system should be

6

they like. We propose fixation nets as a compact way to represent such information. It is to be noted that our approach to replace frame-by-frame analysis is limited to dynamic stimuli produced from interacting with nonstereoscopic 3D objects that can be directly manipulated or oriented by a user e.g. a car on a web site that can be spun around for inspection.

interaction and identify the fixation points together with the polygons they are covering. It is important for this last step to precede the flattening operation because such projections change the spatial relationships between points on the surface. Doing a fixation analysis after the object is flattened is likely to be needlessly complex and error prone. After the 3D object is flattened, every polygon previously covered by gaze positions and a fixation point will be overlaid with the latter’s respective representations at appropriate positions.

4.3.1 Using fixation nets. A trivial way to analyze the gaze data during interactions with a desktop 3D object would be to record each frame of the interaction and do a laborious frame-by-frame analysis. The alternative method we propose can be described through the following steps: (1) the subject interacts with the 3D object and is gaze tracked, (2) the gaze data is processed to identify fixation positions using a suitable fixation algorithm, (3) each gaze position and fixation point is mapped to the relevant polygon of the 3D object, (4) and at the end of the interaction, the 3D object is flattened and overlaid with the appropriate gaze visualizations as shown in Figures 4 and 5. This visualization not only provides information about the user interaction at a glance but it can also be reproduced in a 2D static medium which is important for publication on traditional media e.g. paper.

5. Concluding remarks There are still substantial efforts to be made to help state of the art gaze data visualization tools progress from experimental systems with (1) huge monolithic architectures, (2) tight coupling to data file formats, (3) visualizations informed by hidden processes to simpler but more powerful systems with (1) light weight flexible modules (2) significant direct interaction capabilities, (3) high quality visual and sound rendering (4) open visualizations with visible underlying processes. There is also a need to investigate alternatives to frame-by-frame analysis of gaze data for dynamic stimuli.

6. References [1]

[2]

Figure 4 Visualizing gaze data obtained during the examination of an L-shaped block

[3]

[4]

Figure 5 Visualizing gaze data obtained during the examination of a 3D car model

[5]

4.3.2 Creating fixation nets. The problem of creating 2D dimensional nets from 3D dimensional solids is a wellstudied problem in Geometry and this technique is often used in the creation of 3D environments during the texture mapping phase where the texture or skin of an object is created by firstly flattening out the 3D object, adding textures to the flattened surface [3] and then allowing the system to stretch the texture appropriately to recreate a textured 3D object. Our 3D environment containing the object to be spun during an examination will capture the position of gaze with respect to every polygon looked at. A real-time fixation identification algorithm such as the I-DT algorithm [38] derived from Widdle’s data reduction algorithm [44] will examine the gaze data with respect to the viewing screen during the

[6]

[7] [8]

[9] [10]

7

Alty J. and Rigas, D. (1998), Communicating Graphical Information to Blind Users Using Music: The Role of Context, in C.-M. Karat, A. Lund, J. Coutaz & J. Karat (eds.), Proceedings of CHI’98: Human Factors in Computing Systems, ACM Press, pp.574–81. Becker R. A. and W.S. Cleveland (1988). Brushing Scatterplots. In William S. Cleveland and M. E. McGill, editors, Dynamic Graphics for Statistics, Pages: 201-224. Wadsworth. Bennis C., Vézien J. and I. Gérard (1991). Piecewise surface flattening for non-distorted texture mapping, International Conference on Computer Graphics and Interactive Techniques archive Proceedings of the 18th annual conference on Computer graphics and interactive techniques, Pages: 237 – 246, Year of Publication: 1991, ISSN:0097-8930, Publisher ACM Press New York, NY, USA. Bermudez J., Agutter J., Westenskow D., Foresti S., Zhang Y., Gondeck-Becker D., Syroid N., Strayer D. and F. Drews. (2000). Data Representation Architecture. Visualization Design Methods, Theory and Technology Applied to Anesthesiology in Proceedings of ACADIA ‘2000. Washington D.C. Oct 20-22. Pages: 91-102. Paper can be viewed at http://www.cromdi.utah.edu/text_files/acadia%202000.pdf Bier E. A., Stone M. C., Pier K., Buxton W., and DeRose T. (1993). Toolglass and Magic Lenses: The See-through Interface. In James T. Kajiya, editor, Computer Graphics (SIGGRAPH '93 Proceedings), volume 27, Pages: 73-80. Brown L.M., Brewster, S.A., Ramloll, R., Burton, M. and Riedel, B., Design Guidelines for Audio Presentation of Graphs and Tables, in the Proceedings of ICAD 2003 workshop on Auditory Displays In Assistive Technologies (University of Boston, MA). Buswell G. T. (1935). How people look at pictures. Chicago, IL: University of Chicago Press. Crowe E. and N. H. Narayanan (2000). Comparing interfaces based on what users watch and do. Proceedings of the First ACM Symposium on Eye Tracking Research & Applications, Association for Computing Machinery (ACM) Press, pp. 29-36. Dix A., Beale R. and A. Wood(2000). Architectures to make Simple Visualisations using Simple Systems Advanced Visual Interfaces, Palermo, Italy. Pages: 51-60. ACM Press. Duchowski A, Medlin E, Cournia N, Murphy H, Gramopadhye A, Nair S, Vorah J, Melloy B. 3-D eye movement analysis.Behav Res Methods Instrum Comput. 2002 Nov; 34(4): Pages: 573-91.

[11] [12] [13] [14]

[15]

[16]

[17]

[18] [19]

[20]

[21]

[22] [23]

[24] [25] [26]

[27] [28] [29] [30]

[31]

Duchowski A.T. A breadth-first survey of eye-tracking applications.Behav Res Methods Instrum Comput. 2002 Nov; 34(4). Pages: 455-70. Edmondson, S., Christensen, J., Marks, J. and S. Shieber. (1996). A general cartographic labeling algorithm. Cartographica, Vol. 33, No.4, Winter 1996, pages 13-23. E-prime product website accessed on Feb 2004 http://www.pstnet.com/products/e-prime/ Fukada R. and H. Bubb (2003). Eye tracking study on Web-use: Comparison between younger and elderly users in case of search task with electronic timetable service in PsychNology Journal, 2003, Volume 1, Number 3, Pages: 202- 229 Furnas G. W. (1986). Generalized Fisheye Views. In Proceedings of the Conference on Human Factors in Computing Systems (CHI'86), Pages: 16-23. ACM Press. Goldberg J. H., Stimson M. J., Lewenstein M., Scott N. and A. Wichansky (2002). Eye Tracking in Web Search Tasks: Design Implications. In Proceedings of the Eye Tracking and Related Applications Symposium 2002, ACM Press, Pages 51-59. Halverson, T. and Hornof A. (2002). VizFix Software Requirements Specifications, Computer and Information Science, University of Oregon, http://www.cs.uoregon.edu/research/cmhci/VizFix/VizFixSRS.pdf Hart S. and Wicken, C. (1990), Workload Assessment and Prediction, in H. R. Booher (ed.), MANPRINT, An Approach to Systems Integration, Van Nostrand Reinhold, pp.257–296. Hornof A. J. and Halverson, T. (2002). Cleaning up systematic error in eye tracking data by using required fixation locations. Behavior Research Methods, Instruments, and Computers, 34(4), Pages: 592-604. Available at www.cs.uoregon.edu/~hornof/downloads/RFLs.pdf Jacob R.J.K. (1998). The Use of Eye Movements in HumanComputer Interaction Techniques: What You Look At is What You Get. ACM Transactions on Information Systems, 9 (3), 152-169, 1991. Also reprinted with commentary in Readings in Intelligent User Interfaces, ed. M.T. Maybury and W. Wahlster, Morgan Kaufmann, San Francisco, 1998, Pages: 65-83. Jacob R.J.K. and K.S. Karn, (2003). Eye Tracking in HumanComputer Interaction and Usability Research: Ready to Deliver the Promises (Section Commentary), in The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research, ed. by J. Hyona, R. Radach, and H. Deubel, pp. 573-605, Amsterdam, Elsevier Science, 2003. Josephson S and Holmes ME. (2002). Attention to repeated images on the World-Wide Web: another look at scanpath theory. Behav Res Methods Instrum Comput. 2002 Nov;34(4):539-48. Karn K. S. (2000). “Saccade pickers” vs. “fixation pickers”: the effect of eye tracking instrumentation on research , Source Eye Tracking Research & Application archive, Proceedings of the symposium on Eye tracking research & applications 2000. Karn K.S., Ellis S. and C. Juliano (1999). The Hunt for Usability: Tracking Eye Movements. SIGCHI Bulletin available at www.acm.org/sigchi/bulletin/2000.5/eye.html Land, M., Mennie, N., & Rusted, J. (1999). Eye movements and the roles of vision in activities of daily living: making a cup of tea. Perception, 28, 1311-1328. Lankford C. (2000). Gazetracker: software designed to facilitate eye movement analysis, Eye Tracking Research & Application archive, Proceedings of the symposium on Eye tracking research & applications 2000 table of contents, Palm Beach Gardens, Florida, United States, Pages: 51 – 55, Year of Publication: 2000 ISBN:1-58113-280-8 Leung Y. K. and M. D. Apperley (1994). A review and taxonomy of distortion-oriented presentation techniques. ACM Transactions on Computer-Human Interaction (TOCHI), 1(2): Pages126-160. Lohse G. L. (1997). Consumer eye movement patterns on Yellow Pages advertising, Journal of Advertising. 26(1), Pages: 1-13. 8 Madelain L, Krauzlis RJ. (2003). Effects of learning on smooth pursuit during transient disappearance of a visual target. J Neurophysiol. 2003 Aug; 90(2):972-82. McConkie G. W. (1981). Evaluating and reporting data quality in eye-movement research. Behavior Research Methods & Instrumentation, 13, 97-106.

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39] [40] [41] [42] [43]

[44]

[45]

[46]

[47]

[48] [49]

8

Posner M. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3-25. Rajashekar, U., Cormack L. K. and A. C. Bovik (2003) Image features that draw fixations IEEE International Conference on Image Processing, Vol: 3, Barcelona, Spain, September 14-17, 2003 Page(s): 313-316 Ramloll R and S. Brewster (2002). An Environment for Studying the Impact of Spatialising Sonified Graphs on Data Comprehension, Sixth Information Visualisation 2002 conference (London), IV2002 Proceedings, IEEE Computer Society Press, Pages: 167-175 (available at the IEEE digital library) Ramloll R. (2000). Supporting Cooperative Work Through Ubiquitous Awarenes- Filtration Mechanisms (Ph.D. Thesis) http://ramesh.ramloll.com/Pub/RamlollPhDThesis2000.pdf Chapter 5 Pages: 90-104 Ramloll R., Brewster, S., Yu, Wai and B. Riedel (2001). Using non-speech sounds to improve access to 2D tabular numerical information for visually impaired users, presented at IHMHCI2001,10-14 September 2001, Lilles, In Proceedings of BCS IHM-HCI 2001 (Lille, France), Springer, Pages: 515-530. Ramloll R., Trepagnier C., Sebrechts M. and A. Finkelmeyer (2004). A Gaze Contingent Environment for Fostering Social Attention in Autistic Children: in Proceedings of the Third ACM Symposium on Eye Tracking Research & Applications, Association for Computing Machinery (ACM) Press, Pages 1926; the article can also be viewed at http://ramesh.ramloll.com/Pub/autistHelicop.pdf Reeder R. W., Pirolli, P. and Card, S. K. (2001). WebEyeMapper and WebLogger: Tools for Analyzing Eye Tracking Data Collected in Web-use Studies.CHI 2001, Seattle, Pages 19-20, ISBN:1-58113-340-5 Salvucci D. D. and Joseph H. Goldberg (2000)., Identifying fixations and saccades in eye-tracking protocols, Proceedings of the symposium on Eye tracking research & applications 2000, Palm Beach Gardens, Florida, United States Pages: 71 - 78 ISBN:1-58113-280-8 Shneiderman, B. (ed.) (1993), Sparks of Innovation in Human– Computer Interaction, Publisher, Intellect; (May 1993) ISBN: 156750079X Staker I. and R. A. Bolt (1990). A gaze -responsive Self Disclosing Display. ACM CHI’90, Pp: 3-9. Theeuwes, J., Kramer, A. F., Hahn, S., and Irwin, D. E. (1998). Our eyes do not always go where we want them to go: Capture of the eyes by new objects. Psychological Science, 9, 379-385. Trepagnier C., Sebrechts M. M. and R. Peterson (2002). Atypical Face Gaze in Autism. Cyberpsychol Behav 2002 Jun; 5(3); Pages: 213-217 (ISSN: 1094-9313) Ward MO (2002) A taxonomy of glyph placement strategies for multidimensional data visualization. In Information Visualization, December 2002, Volume 1, Number 3-4, Pages 194-210, Palgrave Macmillan Journal publication Widdel H. (1984). Operational problems in analyzing eye movements. In A. 13. Gale & F. Johnson (Eds.), Theoretical and Applied Aspects of Eye Movement Research (pp. 21-29). New York: Elsevier. Wills G. J. (1996). 524,288 Ways to Say ``This is Interesting''. In Proceedings of the IEEE Symposium on Information Visualization 1996 (InfoVis 1996), Pages: 54-61. IEEE Computer Society Press. Wooding D. S. (2002). Fixation maps: quantifying eyemovement traces, Eye Tracking Research & Application Symposium, ETRA 2002, New Orleans, Louisiana, ISBN: 158113-467-3, ACM Press, Pages 31-36. Wooding DS, Mugglestone MD, Purdy KJ, Gale AG. (2002). Eye movements of large populations: I. Implementation and performance of an autonomous public eye tracker. Behav Res Methods Instrum Comput. 2002 Nov;34(4):509-17. Zeki S. (1992). The Visual Image in Mind and Brain. Scientific American, 267(3): Pages: 42-50. Zhang J., Johnson K., Malin J. and J. Smith. (2002) Humancentered information visualization. In Proceedings of the International Workshop on Dynamic Visualizations and Learning July, 18.-19., 2002 Knowledge Media Research Center (KMRC)Tübingen, Germany