International Journal of Human-Computer Interaction
ISSN: 1044-7318 (Print) 1532-7590 (Online) Journal homepage: http://www.tandfonline.com/loi/hihc20
Comparing Ocular Parameters for Cognitive Load Measurement in Eye-gaze Controlled Interfaces for Automotive and Desktop Computing Environments Pradipta Biswas, Varun Dutt & Pat Langdon To cite this article: Pradipta Biswas, Varun Dutt & Pat Langdon (2015): Comparing Ocular Parameters for Cognitive Load Measurement in Eye-gaze Controlled Interfaces for Automotive and Desktop Computing Environments, International Journal of Human-Computer Interaction, DOI: 10.1080/10447318.2015.1084112 To link to this article: http://dx.doi.org/10.1080/10447318.2015.1084112
Accepted online: 14 Oct 2015.
Submit your article to this journal
Article views: 14
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=hihc20 Download by: [Indian Instiute of Technology Mandi]
Date: 18 October 2015, At: 06:23
Comparing Ocular Parameters for Cognitive Load Measurement in Eye-gaze Controlled Interfaces for Automotive and Desktop Computing Environments Pradipta Biswas1, Varun Dutt2 and Pat Langdon1 1
ip t
University of Cambridge, UK
2
cr
Address correspondence to Pradipta Biswas at:
[email protected];
[email protected]
us
Abstract
M an
Eye-gaze tracking is traditionally used to analyse ocular parameters for investigating visual psychology, marketing study, behaviour analysis and so on. In recent time, eye-gaze trackers are also being used to control electronic interfaces in assistive technology, automobile control and even in consumer electronic products like smartphones and tablets. However
d
there are not many attempts to combine these two streams of research on active and passive
pt e
uses of eye-gaze trackers. This paper compares a few ocular parameters to estimate users’ cognitive load in eye-gaze controlled interfaces.
We found that average velocity of a
ce
particular type of micro-saccadic eye movement called Saccadic Intrusion is most indicative of users’ cognitive load compared to pupil dilation and eye-blink based parameters. Results from our study can be used to develop new metrics of cognitive load measurement as well as
Ac
Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015
Indian Institute of Technology, Mandi, India
to design intelligent gaze controlled interfaces those respond to users’ cognitive load.
1. Introduction Evolution of eyes from simple photoreceptor cells to an organ supporting complex vision during the evolutionary event known as Cambrian Explosion, has surprised many scientists including Charles Darwin. The complex eyes in human do not only support a complicated
visual system which is yet to be completely reproducible by machine but also work as an excellent medium to express affective states. Charles Darwin in his book The Expression of The Emotions in Man and Animals written in 1872, indicated a correlation between widening and narrowing of eyes with emotional states. Inthe first decade of 19th century, Redlich [16] and Westphal [20]related pupil dilation with physical task demand, or even thinking of
ip t
physical task, while Hess [7] reported change in pupil dilation with respect to viewing of
cr
increase in cognitive load results in a sudden hike in pupil dilation which can be measured by
us
a set of metrics calculated through Wavelet transform of the pupil signal considering driving simulator [12, 13], aviation [14] or map reading [9] tasks.
M an
However analysing pupil signal is not the only way of measuring cognitive load and researchers also investigated eye blinks and eye movement for detecting mental workload. In his 1991 report for Navy Personnel Research and Development Centre, Kramer [10]
d
presented a detailed review of endogenous blink and pupil dilation based techniques for
pt e
mental workload detection. Although the relation between mental workload and rate of eye blinks is debatable but most study found a reliable correlation between average duration of
ce
blinks and mental workload in single and dual task situations. Recent advancement of digital electronics and in particular image processing systems contributed to the development of eye-gaze trackers, which can accurately measure pupil
Ac
Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015
photographs. In recent time, using sophisticated eye-gaze trackers, researchers found that an
dilation, eye blinks and eye gaze movement in real time in a wide variety of situations and contexts. Modern eye-gaze trackers [5, 11] are mostly used to record and analyse eye gaze movements for various stimuli although research on eye gaze movement dated back to late 18th century when Louis Émile Javal investigated saccadic movements in a reading task [8]. Eye movement can be classified into three main types of movement – Saccade, Smooth Pursuit and Vergence movements [17].In recent time, researchers concentrated on a particular
type of micro-saccadic eye gaze movement termed as Saccadic Intrusion (SI) [1, 18] in relation to detecting mental workload. We have explained Saccadic Intrusion later in the paper, but in short Tokuda and colleagues [18] found velocity of SI was higher in a complex task condition than the simpler one for most participants. However, previous research on cognitive load estimation from ocular parameters did not
ip t
consider gaze-controlled interfaces. Gaze controlled interfaces introduce new challenge for
cr
pointer in a direct manipulation interface. Recent advent in eye gaze tracker makes gaze
us
controlled interfaces common in assistive technology [2, 6, 19], consumer electronic products (like tablets and smartphones) as well as made them popular in automobiles [15] and aircrafts
M an
or more specifically to combat aircrafts [3].
This paper presents two studies conducted on gaze controlled interfaces, which were used to measure and compare different ocular parameters non-intrusively calculated from eye-gaze
d
tracker for estimating users’ cognitive load. The first study was conducted in an automotive
pt e
environment and used a low cost eye gaze tracker that did not accurately measure pupil dilation. The study analysed eye-gaze movements and related saccadic intrusion with users’
ce
perceived cognitive load in terms of TLX scores. The second study used an expensive eyegaze tracker and compared different ways of cognitive load measurement for simple point and selection tasks. Overall, we found that average velocity of saccadic intrusion is most
Ac
Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015
cognitive load detection from eyes as users need to move eye gaze to control an on-screen
indicative of users’ cognitive load compared to pupil dilation and eye-blink based parameters.
2. User study on automotive environment This section reports a user trial on exploring the possibility of gaze control interface for operating a dashboard in an automotive environment. In particular, we evaluated the effect of two different track conditions on drivers' performance with eye-gaze tracking interface.
Previous work [15] has already compared eye-gaze tracking interface with touch-screen control. We took forward that work with a low-cost eye-gaze tracker and intelligent target prediction algorithm [2] that can reduce pointing time. We also computed and compared Saccadic Intrusion (SI) in this study. Saccadic Intrusion is a particular type of eye movement that has been already classified and related to mental
ip t
workload [18]. SI is more robust than pupilometry based method of cognitive load
cr
investigated whether different SI parameters like amplitude and duration can still be related
us
to mental workload even when users were manipulating their eye-gaze to operate different
M an
screen elements.
We have described the user study in the following sections.
2.1. Participants
d
We collected data from 12 participants (age range 19 to 27, 10 male, 2 female). All
pt e
participants were university students and none of them regularly drove cars. Eight participants had driving licenses although the qualities of driving tests were quite different for
ce
them. However all participants were expert users of the driving simulator and used to drive cars in the simulator.
Ac
Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015
measurement as SIs are less sensitive to ambient light condition than pupil dilation. We
2.2. Design
We designed the test to evaluate the effect of an eye gaze-controlled secondary task on the primary driving task with participants with varying level of driving skills. The primary task involved driving a car in the left lane without veering off from the lane. We used two different track conditions -- a simple track consisting of only two turns and a complex track consisting of 20 turns. There were no other traffic on the road and drivers were instructed to
drive safely without veering off the driving lane and simultaneously operating the car dashboard using their eye-gaze. The secondary task was initiated through an auditory cue. It mimicked a car dashboard (figure 1) and participants were instructed to press a button on it after hearing the auditory cue. The auditory cue was set to appear between every 5 and 7 seconds interval. The target button was randomly selected in the car dashboard. The pointing
ip t
was undertaken through eye gaze of users using an intelligent eye gaze tracking algorithm [3]
Track Condition
us
•
cr
The study (figure 2) was a 2 × 2 factorial design where the independent variables were
M an
o Simple o Complex •
Presence of Secondary Task
d
o Driving without Secondary Task
pt e
o Driving with Secondary Task The dependent variables were
Task Completion Time
•
Average deviation from centre of road
ce
•
Ac
Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015
and selection was done through a hardware button on steering.
•
Number of correct selections in gaze-controlled interface
We also measured drivers' cognitive load in terms of pulse rate using an Oximeter 1 and
NASA TLX scores.
1
http://www.patient.co.uk/doctor/pulse-oximetry
2.3. Material We used a Logitech driving simulator hardware and Torque© car simulation software. The hardware was set as an automatic transmission car. We used an Tobii EyeX eye gaze tracker
ip t
and EyeX SDK for the gaze-controlled interface. The primary task was run on a Linux
us
cr
had a dimension of 34.5 cm × 19.5 cm with screen resolution of 1368 × 800 pixels.
2.4. Procedure
M an
Initially participants were briefed about the procedure and trained to use the driving simulator and the gaze controlled interface. Then they undertook the trial in random order of track conditions. After completion of each condition, they filled up the TLX sheet based on their
d
toughest experience during the trial.
pt e
We used logging software that recorded the trajectory of the car with timestamp from the driving simulator and cursor and eye-gaze movements from the secondary task. We also
ce
recorded participants' pulse rate from the Oximeter with timestamp.
2.5. Results
Ac
Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015
desktop while the secondary task was conducted on a Windows 8 Laptop. The Laptop screen
We found a statistically significant correlation between number of correct selections in the secondary task and average velocity of the car (figure 3, ρ = -0.46, p < 0.05). Drivers could make significantly higher number [t (1,21) = -2.2, p < 0.05] of correct selections using eyegaze control while they were driving in the complex track than the simple track (figure 4). In a repeated measure ANOVA, we found
•
Significant main effect of Track Condition on o Task completion time F(1, 11) = 88.24, p
