2014 Specialist Meeting—Spatial Search
Card et al.—1
The VERP Explorer— A Tool for Applying Recursion Plots to the Eye-Movements of Visual-Cognitive Tasks STUART K. CARD Consulting Professor Department of Computer Science Stanford University Email:
[email protected]
with
D
ÇAĞATAY DEMIRALP
JESSE CIRIMELE
Post Graduate Fellow Department of Computer Science Stanford University Email:
[email protected] [email protected]
Graduate Student Department of Computer Science Stanford University Email:
[email protected]
esigns in human-computer interaction (HCI) often involve trading between spatial and textual representations to achieve a nuance of representation that makes a task faster to
execute, easier to learn, or less prone to error. Such designs can be very effective, but they can also be subtle, and it can be difficult to understand the mechanisms in play. Even generally successful interfaces can still hide bad combinations of interface, task, and context that could be improved were they identified. One method of approaching this problem is to run chronometric experiments with contrasting conditions. Aside from being expensive for development work, this method is at such an aggregate level that it often does not provide much access or insight into the underlying mechanisms at work. Another method is cognitive simulation (Kieras, 2014). The intent is to specify the likely mechanisms at work and to validate them by their ability to predict chronometric or other data. The validated simulator can then be put to work on inferring other consequences of the design with some claim to knowing why. While this method has advantages, it is even more expensive and is most practical for large projects or projects close to an existing model that can provide a starting point. Figure 1. The VERP Explorer.
2014 Specialist Meeting—Spatial Search
Card et al.—2
A third method is to construct a tool that makes the mechanisms at work visible by when applied to samples of user behavior. In this paper, we propose such a method and tool, The VERP
Explorer (VERP stands for Visualization of Eye-Movements based on Recurrence Plots), an interactive visualization based recurrence plots. Eye-movement sequences are taken of users performing visual-cognitive tasks with the subject system. These are mapped into recurrence plot visualizations to highlight patterns of quasi-sequential behavior. In our system, these patterns are then back-mapped into—and overlaid on—the eye-movement scene to help characterize and provide insights into the behavior. Recurrence plots are a type of non-linear analysis that has been used in the study of dynamical systems and other areas (Eichmann et al., 1978; Marwan, 2008). Recently it has been applied to eye-movements (Anderson et al, 2013). Our tool extends and integrates eye-movement and recursion plot analysis into an interactive tool, simplifying exploratory analysis. Eye-movements can be thought of as a sequence of eye gaze positions fi parameterized by time. To obtain the matrix [
] that is the basis for a recurrence plot, we start with the first eye-position f1 and compare it to
all the other eye-positions in the sequence, including itself. If the distance d( fij ) between the two compared eye positions is within some small distance , then we put a 1 at that position in the matrix, otherwise a 0.
rij
1,
d( fij )
0, otherwise
We color white the cells whose value is 1 and black the others. Figure 1 shows an example of such a recurrence plot in the VERP Explorer. The data are a sample of eye-movements generating in our previous studies of the visual-cognitive task of emergency medical checklist use (Crimele et al., 2014; Wu et al., 2014). At the left in the figure is an image of the medical checklist, on which eye-movements from the eye-tracker have been superimposed. The doctor in this task was trying to use the checklist to answer the question, “What is the dose for Vasopressin?” On the right is the corresponding recurrence plot. As can be seen, the white areas are scattered in a disorganized fashion reflecting the disorganization of the search (Figure 1). Recurrence plots of other checklist designs are different, reflecting the visual-cognitive processes required to extract needed information. A small region of the recursion plot has been brushed (the nearly square brownish area). This “brushing” highlights some of the dots on the checklist eye-movement scene, causing them to turn red, indicating that the square-ish area on the recurrence refers to red text in that area. The VERP Explorer also allows us to go in the opposite direction. The area brushed on the eye-movement scene panel highlights the red areas on the recurrence plot, from which we can tell that the doctor actually looked at the area where the answer is, failed to see it, and came back for an intensive re-examination of that same area later. Examining the recurrence graphs for different designs of the emergency checklists suggests the way in which the designs affect the details of the search. Many of the patterns can be broken down into “motifs” that signal certain types of behavior. These patterns can be quantified by recurrence quantification analysis (RQA) and thereby aggregated and compared. By using this tool,
2014 Specialist Meeting—Spatial Search
Card et al.—3
we have identified a pattern of search that looks a lot like that described by information foraging theory on a miniaturized scale. Doctors search for an information patch using searching saccades (explore), then read some information in the patch using fixations (exploit). If they fail to find what they want, they repeat the pattern. This pattern identified, of the doctor using multiple foraging cycles, suggests there is great room for improvement for this checklist and that the target is too similar to other targets or is surrounded by distracting elements, or is not unique, etc. With The VERP Explorer, we could quickly generate and test some alternatives. We believe VERP is a practical tool and we are making it into an application downloadable from the web. References Anderson, Nicola C, Bischof, Walter F., Laidlaw, Kaitlin, Risko, Evan F., and Kingstone, Alan (2013). Recurrence quantification analysis of eye movements. Behavior Research 45: 842–856. Cirimele, Jesse, Wu, Leslie, Leach, Kirsten, Card, Stuart, Harrison, T. Kyle, Chu, Larry, Klemmer, Scott R. (2014). RapidRead: Step-At-A-Glance Crisis Checklists. 8th International Conference on Pervasive Computing Technologies for Healthcare (Oldenburg, Germany, May 20–23, 2014). Eckmann, J.P., Kmphorst, S. Oliffson, & Ruelle, D. (1987). Recurrence plots of dynamical systems. Europhysics Letters, 4(9): 973–977. Kieras, David E. (2014). Towards accurate and practical predictive models of active-vision-based visual search. CHI 2014. Marwan, N. (2008). A historical review of recurrence plots. European Physics Journal Special Topics 164: 3–12. Pirolli, P. and Card, S. K. (1999). Information foraging. Psychological Review 106(4): 643–675. Wu, Leslie, Cirimele, Jesse, Leach, Kirsten, Card, Stuart K., Chu, Larry, T Kyle Harrison, Klemmer, Scott R. (2014). Supporting crisis response with dynamic procedure aids. ACM Conference on Designing Interactive Systems (Vancouver, Canada, June 21–15).
