Automatic Event Structure Parsing for Context Modeling: A Role for Postural Orienting Responses Carey D. Balaban, Jarad Prinkey, Greg Frank, Wilhelm Kincses1, James C. Forsythe2, Kevin R. Dixon2, and Mark Redfern
Dept. of Otolaryngology,University of Pittsburgh , EEINS 107 Pittsburgh, PA 15213, U. S. A. DaimlerChrysler AG, Research & Technology, Machine Perception, 096/T728 – REI/AI, 70546 Stuttgart, Germany
1
2
Sandia National Laboratories, P.O. Box 5800, Mailstop 1188, Albuquerque, New Mexico 87185, U. S. A. E-Mail:
[email protected]
During normal living activities, we are immersed in a stream of activities and contexts. However, we perceive these contexts and activities as being divided into temporal distinct parts or sequences, in a process termed event structure perception (Zacks and Tversky, Psychol. Bull. 127 (2001) 3-21). These perceptual event boundaries can be identified operationally by methods such as subjective identification of meaningful boundaries in video clips or monitoring selfpresentation of text on a computer monitor during reading comprehension tasks. This presentation presents evidence that measurement of postural orienting responses in seated subjects can demarcate aspects of the parsing of continuous events into parts. Since the measurements are unobtrusive, they provide objective, subject-based framework for real-time contextual cognitive modeling. Postural adjustments reflect a trade-off between the demands of maintaining balance and the demands of generating voluntary movements. Voluntary movements vary in their degree of automaticity. Orienting responses are automatic voluntary movements that direct processing resources toward events of interest. Detecting these movements is facilitated in seated humans by the constraints imposed by the chair and by the reduced degrees of freedom of body movements for directing gaze, the torso and upper limbs. Hence, seated workplace and vehicular environments are advantageous platforms for using pressure sensor and head tracker technologies to detect orienting responses. Seated postural data have been collected from subjects (1) working at a computer workstation and (2) driving a car during Phases 1 and 2 of the DARPA AugCog program. Pressure data from the seat surface and back are measured at 4-4.5 Hz from a 16 X 16 array of ulthrathin capacitive pressure sensors, spaced at 1 inch intervals (Vista Medical FSA Pads). Head movement data are measured with 6 degrees of freedom at a minimum of 50 Hz with either an electromagnetic (Ascension Technologies ‘Flock of Birds’) or a Logitech ultrasonic head tracker system. The data are collected on a laptop computer with custom software written in LabView (National Instruments). In the computer workstation environment, initial results were obtained from 15 subjects in studies with the Warship Commander Task (WCT) and 7 subjects in studies with a modified Aegis CIC task (ACICT). Both tasks require the operator to select, identify and initiate action toward tracks on a screen representing a battlespace. The initiation of a wave of tracks is cues by a bell for the WCT, while changes in tracks are uncued in the ACICT. An orienting response was detected in either task as epochs (8 second duration for WCT, 12 second duration for
ACICT) where more than 50% of the variance in the data is accounted by a linear relationship between the instantaneous number of tracks on the screen and the instantaneous position of the head or the center of pressure on the chair seat relative to the screen. Epochs of ‘orientation’ invariably appeared when the initial tracks entered the field of view at the beginning of cued waves in the WCT. In the ACICT, these epochs of ‘orientation’ occurred intermittently and the cumulative duration of these epochs in each session was correlated strongly with measures of performance (hook score and ID score, r2 >0.71 for complete data from 5 subjects). Across all subjects, the median probability of a subject action within 2 seconds of orientation responses was 0.62, increasing to 0.84 within 4 seconds of response onset. Analogous results from a reading comprehension task and a structured Microsoft Office work task also argue that measurable orienting responses indicate meaningful task parsing. Postural data from five drivers were collected in Mercedes-Benz S-class under normal driving conditions on limited access highways, two lane highways and an urban environment. Driving contexts were logged from a video recording synchronized with the data collection. In initial analyses, discriminant analysis with a subset of five gauge calculations from the pressure pads produced statistically significant classification (p
