Comparing Ocular Parameters for Cognitive Load ...

4 downloads 23003 Views 22MB Size Report
Oct 18, 2015 - ISSN: 1044-7318 (Print) 1532-7590 (Online) Journal homepage: http://www.tandfonline.com/loi/hihc20. Comparing Ocular ... Interfaces for Automotive and Desktop Computing ..... two to three training sessions. Regarding ...
International Journal of Human-Computer Interaction

ISSN: 1044-7318 (Print) 1532-7590 (Online) Journal homepage: http://www.tandfonline.com/loi/hihc20

Comparing Ocular Parameters for Cognitive Load Measurement in Eye-gaze Controlled Interfaces for Automotive and Desktop Computing Environments Pradipta Biswas, Varun Dutt & Pat Langdon To cite this article: Pradipta Biswas, Varun Dutt & Pat Langdon (2015): Comparing Ocular Parameters for Cognitive Load Measurement in Eye-gaze Controlled Interfaces for Automotive and Desktop Computing Environments, International Journal of Human-Computer Interaction, DOI: 10.1080/10447318.2015.1084112 To link to this article: http://dx.doi.org/10.1080/10447318.2015.1084112

Accepted online: 14 Oct 2015.

Submit your article to this journal

Article views: 14

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=hihc20 Download by: [Indian Instiute of Technology Mandi]

Date: 18 October 2015, At: 06:23

Comparing Ocular Parameters for Cognitive Load Measurement in Eye-gaze Controlled Interfaces for Automotive and Desktop Computing Environments Pradipta Biswas1, Varun Dutt2 and Pat Langdon1 1

ip t

University of Cambridge, UK

2

cr

Address correspondence to Pradipta Biswas at: [email protected]; [email protected]

us

Abstract

M an

Eye-gaze tracking is traditionally used to analyse ocular parameters for investigating visual psychology, marketing study, behaviour analysis and so on. In recent time, eye-gaze trackers are also being used to control electronic interfaces in assistive technology, automobile control and even in consumer electronic products like smartphones and tablets. However

d

there are not many attempts to combine these two streams of research on active and passive

pt e

uses of eye-gaze trackers. This paper compares a few ocular parameters to estimate users’ cognitive load in eye-gaze controlled interfaces.

We found that average velocity of a

ce

particular type of micro-saccadic eye movement called Saccadic Intrusion is most indicative of users’ cognitive load compared to pupil dilation and eye-blink based parameters. Results from our study can be used to develop new metrics of cognitive load measurement as well as

Ac

Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015

Indian Institute of Technology, Mandi, India

to design intelligent gaze controlled interfaces those respond to users’ cognitive load.

1. Introduction Evolution of eyes from simple photoreceptor cells to an organ supporting complex vision during the evolutionary event known as Cambrian Explosion, has surprised many scientists including Charles Darwin. The complex eyes in human do not only support a complicated

visual system which is yet to be completely reproducible by machine but also work as an excellent medium to express affective states. Charles Darwin in his book The Expression of The Emotions in Man and Animals written in 1872, indicated a correlation between widening and narrowing of eyes with emotional states. Inthe first decade of 19th century, Redlich [16] and Westphal [20]related pupil dilation with physical task demand, or even thinking of

ip t

physical task, while Hess [7] reported change in pupil dilation with respect to viewing of

cr

increase in cognitive load results in a sudden hike in pupil dilation which can be measured by

us

a set of metrics calculated through Wavelet transform of the pupil signal considering driving simulator [12, 13], aviation [14] or map reading [9] tasks.

M an

However analysing pupil signal is not the only way of measuring cognitive load and researchers also investigated eye blinks and eye movement for detecting mental workload. In his 1991 report for Navy Personnel Research and Development Centre, Kramer [10]

d

presented a detailed review of endogenous blink and pupil dilation based techniques for

pt e

mental workload detection. Although the relation between mental workload and rate of eye blinks is debatable but most study found a reliable correlation between average duration of

ce

blinks and mental workload in single and dual task situations. Recent advancement of digital electronics and in particular image processing systems contributed to the development of eye-gaze trackers, which can accurately measure pupil

Ac

Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015

photographs. In recent time, using sophisticated eye-gaze trackers, researchers found that an

dilation, eye blinks and eye gaze movement in real time in a wide variety of situations and contexts. Modern eye-gaze trackers [5, 11] are mostly used to record and analyse eye gaze movements for various stimuli although research on eye gaze movement dated back to late 18th century when Louis Émile Javal investigated saccadic movements in a reading task [8]. Eye movement can be classified into three main types of movement – Saccade, Smooth Pursuit and Vergence movements [17].In recent time, researchers concentrated on a particular

type of micro-saccadic eye gaze movement termed as Saccadic Intrusion (SI) [1, 18] in relation to detecting mental workload. We have explained Saccadic Intrusion later in the paper, but in short Tokuda and colleagues [18] found velocity of SI was higher in a complex task condition than the simpler one for most participants. However, previous research on cognitive load estimation from ocular parameters did not

ip t

consider gaze-controlled interfaces. Gaze controlled interfaces introduce new challenge for

cr

pointer in a direct manipulation interface. Recent advent in eye gaze tracker makes gaze

us

controlled interfaces common in assistive technology [2, 6, 19], consumer electronic products (like tablets and smartphones) as well as made them popular in automobiles [15] and aircrafts

M an

or more specifically to combat aircrafts [3].

This paper presents two studies conducted on gaze controlled interfaces, which were used to measure and compare different ocular parameters non-intrusively calculated from eye-gaze

d

tracker for estimating users’ cognitive load. The first study was conducted in an automotive

pt e

environment and used a low cost eye gaze tracker that did not accurately measure pupil dilation. The study analysed eye-gaze movements and related saccadic intrusion with users’

ce

perceived cognitive load in terms of TLX scores. The second study used an expensive eyegaze tracker and compared different ways of cognitive load measurement for simple point and selection tasks. Overall, we found that average velocity of saccadic intrusion is most

Ac

Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015

cognitive load detection from eyes as users need to move eye gaze to control an on-screen

indicative of users’ cognitive load compared to pupil dilation and eye-blink based parameters.

2. User study on automotive environment This section reports a user trial on exploring the possibility of gaze control interface for operating a dashboard in an automotive environment. In particular, we evaluated the effect of two different track conditions on drivers' performance with eye-gaze tracking interface.

Previous work [15] has already compared eye-gaze tracking interface with touch-screen control. We took forward that work with a low-cost eye-gaze tracker and intelligent target prediction algorithm [2] that can reduce pointing time. We also computed and compared Saccadic Intrusion (SI) in this study. Saccadic Intrusion is a particular type of eye movement that has been already classified and related to mental

ip t

workload [18]. SI is more robust than pupilometry based method of cognitive load

cr

investigated whether different SI parameters like amplitude and duration can still be related

us

to mental workload even when users were manipulating their eye-gaze to operate different

M an

screen elements.

We have described the user study in the following sections.

2.1. Participants

d

We collected data from 12 participants (age range 19 to 27, 10 male, 2 female). All

pt e

participants were university students and none of them regularly drove cars. Eight participants had driving licenses although the qualities of driving tests were quite different for

ce

them. However all participants were expert users of the driving simulator and used to drive cars in the simulator.

Ac

Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015

measurement as SIs are less sensitive to ambient light condition than pupil dilation. We

2.2. Design

We designed the test to evaluate the effect of an eye gaze-controlled secondary task on the primary driving task with participants with varying level of driving skills. The primary task involved driving a car in the left lane without veering off from the lane. We used two different track conditions -- a simple track consisting of only two turns and a complex track consisting of 20 turns. There were no other traffic on the road and drivers were instructed to

drive safely without veering off the driving lane and simultaneously operating the car dashboard using their eye-gaze. The secondary task was initiated through an auditory cue. It mimicked a car dashboard (figure 1) and participants were instructed to press a button on it after hearing the auditory cue. The auditory cue was set to appear between every 5 and 7 seconds interval. The target button was randomly selected in the car dashboard. The pointing

ip t

was undertaken through eye gaze of users using an intelligent eye gaze tracking algorithm [3]

Track Condition

us



cr

The study (figure 2) was a 2 × 2 factorial design where the independent variables were

M an

o Simple o Complex •

Presence of Secondary Task

d

o Driving without Secondary Task

pt e

o Driving with Secondary Task The dependent variables were

Task Completion Time



Average deviation from centre of road

ce



Ac

Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015

and selection was done through a hardware button on steering.



Number of correct selections in gaze-controlled interface

We also measured drivers' cognitive load in terms of pulse rate using an Oximeter 1 and

NASA TLX scores.

1

http://www.patient.co.uk/doctor/pulse-oximetry

2.3. Material We used a Logitech driving simulator hardware and Torque© car simulation software. The hardware was set as an automatic transmission car. We used an Tobii EyeX eye gaze tracker

ip t

and EyeX SDK for the gaze-controlled interface. The primary task was run on a Linux

us

cr

had a dimension of 34.5 cm × 19.5 cm with screen resolution of 1368 × 800 pixels.

2.4. Procedure

M an

Initially participants were briefed about the procedure and trained to use the driving simulator and the gaze controlled interface. Then they undertook the trial in random order of track conditions. After completion of each condition, they filled up the TLX sheet based on their

d

toughest experience during the trial.

pt e

We used logging software that recorded the trajectory of the car with timestamp from the driving simulator and cursor and eye-gaze movements from the secondary task. We also

ce

recorded participants' pulse rate from the Oximeter with timestamp.

2.5. Results

Ac

Downloaded by [Indian Instiute of Technology Mandi] at 06:23 18 October 2015

desktop while the secondary task was conducted on a Windows 8 Laptop. The Laptop screen

We found a statistically significant correlation between number of correct selections in the secondary task and average velocity of the car (figure 3, ρ = -0.46, p < 0.05). Drivers could make significantly higher number [t (1,21) = -2.2, p < 0.05] of correct selections using eyegaze control while they were driving in the complex track than the simple track (figure 4). In a repeated measure ANOVA, we found



Significant main effect of Track Condition on o Task completion time F(1, 11) = 88.24, p