Cognitive Load: Assessment with Visual Behavior Cognitive Load and In-Vehicle HumanMachine Interaction Workshop Joanne L. Harbluk Transport Canada Auto UI 2016, Ann Arbor 24 October 2016
Overview 1. A few words about Cognitive Load 2. What does the impact of Cognitive Load look like? 3. Approaches to Cognitive Load using glances 4. Standard - ISO 15007 (revision & new development) 5. Challenges, future directions
2
Cognitive Load: Mental resources required to perform a task: planning, decision making, error detection, inhibiting habitual actions, utilizing information in working memory and resolving novel and complex situations
Cognitive Distraction: Diversion of mental
resources from driving in dual-task conditions
3
Visual: eyes off the road Manual: hands off wheel Cognitive: mind off driving V V
M
Texting C
M
Hands free phone C
V
M C
Reaching
Cognitive Demand Is Part of Most Tasks 4
[Lee, 2009; Distracted Driving Summit]
Tasks Can Be Very Cognitively Demanding: Email reader
(From Harbluk & Lalande, 2005) 5
Tasks Can Be Very Cognitively Demanding: Email reader
• Slower detection of events in the environment • Poor speech quality resulted in missed events (From Harbluk & Lalande, 2005) 6
Demand on the driver is more than just the mode of presentation “Driving is a visual manual task” – present a task in a non-visual format (e.g., speech)
Motivation: – presentation format vs processing of that information – Might account for conflicting results in literature (Trbovich, 2007) 7
Logic of study: – Driving has strong visual demand – Different formats for presentation (auditory, visual) – Tasks require different processing resources (phonological, visual, spatial)
8
Task Processing Requirements: – Phonological task: 2 words “compute pewter” • Subvocal speech: Common phonetic sound? “pute”
– Visual task: 3 words “LUCKY, FURRY, DICE” • Visualize the words; “yes” if at least 2 words contain a closed letter; else no
– Spatial Task: 2 times to compare (8:30 & 9:00) • Imagine analog clock faces: which of the two times forms the greater acute angle (90deg) 9
10
11
Type of Processing Required for the task (Trbovich, 2007)
12
Visual Presentation Mode: Reduced driving performance (looking away from road)
Auditory Presentation Mode: Reduced driving performance for tasks requiring visual spatial processing
(Trbovich, 2007)
13
2. The impact of cognitive load looks like?
Effects of cognitive load are difficult to observe Video example:
(Harbluk, Noy, Trbovich & Eizenman, 2007)
14
Visual Scanning: No Additional Task
15
Visual Scanning: Cognitive Task
16
3. Approaches to Cognitive Load 1. Changes in where drivers look inside & outside the vehicle 2. Concentration of gaze in forward view
Areas of Interest (AOI)
Measures of Gaze concentration
17
Areas of Interest (AOI) Visual Behavior: • location, duration & frequency of glances Area of Interest (AOI): predetermined area within the visual scene • # of times a driver looks to that area • Driving time spent looking at that area • % of task time spent looking at that area • Fail to look 18
19
Changes seen in AOIs With increased cognitive load: • Look less to relevant places: intersections, traffic lights, instruments etc • Look at the device itself (e.g., NHTSA, 2003) Is visual confirmation needed? Necessary to explore the data; you may not know what those changes are..... A lot to be learned from actually watching what the drivers do; video & audio of the session very helpful 20
Gaze Concentration • Tendency not to look around – consistent with AOI neglect • Looking at the road, but this behaviour needs to be considered in the context of the tasks and the driving situation
21
Gaze Concentration: Visual-Manual
Baseline
Visual Manual Task Fixation Density Plots 22 (Victor, Harbluk, Engström , 2005)
Gaze Concentration: Cognitive Task
Baseline
Auditory Task Fixation Density Plots 23 (Victor, Harbluk, Engström , 2005)
Percent Road Centre Percent Road Centre (PRC): The % of fixations directed towards the road centre during a task (Victor et al., 2005) Intuitive measure Interpretation: Greater cognitive load results in more concentrated visual behavior Variations in the procedure • Definition of centre: circle 16 degree diameter centered around road center point; also 8, 12, 16 degrees (Wang et al., 2014) • May be influenced by driving environment and task (road type, simulator) • Rectangle 20 degree horizontal by 15 degree vertical rectangle • % of time spent looking in that area •
type of data used; ease of analysis, type of analysis packages used; variation in how “center” is defined, area as circle, rectangle, size varies 24
Changes in Gaze Behavior % change in time spent as cognitive demand increases
(Harbluk, Noy, Trbovich & Eizenman, 2007)
25
Standard Deviation of Gaze Standard Deviation of Gaze: is the standard deviation of the combined horizontal & vertical angles Interpretation: The smaller Standard Deviation of Gaze, the more concentrated the visual behavior as a result of increased cognitive workload Variations: • Often only horizontal gaze is used • gaze angle or gaze position (what is available by system)
(e.g., Recartes & Nunes, 2003; Sodhi et al., 2002, Wang et al., 2014)
Standard Deviation of Gaze
27
4. Measurement of Driver Visual Behaviour – Revision & Elaboration • ISO 15007-1 & 2 • Key terms & parameters for analysis of driver visual behaviourglances & glancerelated measures • Common reference, consistency in approach, empirically based methods, best practices 28
• Equipment & procedures for analysing driver visual behaviour • To include:
Technical Specification
• Planning of evaluation trials • data capture equipment • Analysis, interpretation and reporting • add video reduction etc. 29
5. Challenges and Future Directions • • • • •
Glances: looking but attention? Processing? Significance? Decision making with results. Often only have relative data; no redline. What metrics to use & how many do we need? Safety impact, crash relevance? Technical aspects of dealing with the data
• Glance data is very compelling from a safety perspective. • Choice of metric: who is your audience -Policy/science • Assessment of optimum workload, measures of engagement? • Turn distraction around- is enough attention being paid? How do we know that drivers are sufficiently engaged?
Thank you for your attention!
[email protected]
31
