changes in pose and/or illumination produce large changes in objects' visual .... design, utilized software and hardware components, as well as the processing ...
Simulation Driven Experiment Control in Driver Assistance Assessment Andreas Riener and Alois Ferscha Johannes Kepler University Linz, Institute for Pervasive Computing Altenberger Str. 69, A-4040 Linz, Austria Tel. +43/7326/3343-920, Fax. +43/732/2468-8524 {riener,ferscha}@pervasive.jku.at
Abstract Embedded systems technologies and advances in micro electronics have accelerated the evolution of driver assistance systems towards more driving safety, comfort, entertainment and wayfinding. With the technological and quality progress, however, also goes the growing of interaction complexity, information overload, and intricate interface designs. Assessing interaction designs for in-car assistance services is an emerging and vibrant field of research. To avoid situations of possibly fatal danger when assessing driver assistance services in real driving situations, we propose trace-driven simulation to steer the experiments with users in a automotive driving simulator. Based on our own developments of driver assistance systems involving the sense of touch, i.e. exploiting haptics as a communication channel in vehicle-to-driver interactions, we demonstrate how pre-recorded traces of driving situations can control user studies. Our experiments show, that simulated driving journeys is a viable alternative to the more hazardous ”on-the-road” user studies. With respect to haptics as an additional channel of communication we find that vibro-tactile stimuli are a promising means to raise driver attention when the visual and auditive channels fail due to overload. Keywords. Trace-Driven Simulation, Vibro-tactile Feedback, Multimodal Interfaces, Haptic Perception, Driving Performance, User-centered Design, Realtime.
1
Motivation
Vehicle handling gets a more and more challenging task, because of (i) an increasing number as well as complexity of IT services in cars, (ii) excessive or even overstrained use of visual and auditive information channels (capacity should be freed to keep focus on the main activity of driving), (iii) traditional interaction paradigms, which are often not quali-
fied for in-car appliance control (input devices, such as fullfunctional keyboards or mice are not available, output capabilities are limited due to fixed indicator elements in the dashboard or small/low-resolution screens; thus, in-vehicle I/O have to to be reworked or replaced by adequate alternatives [1]), and (iv) person-dependency and environmentalinfluence of auditive and visual information channels (voice is affected by a multitude of user-specific parameters like age, gender, cognitive load or emotional stress, and ambient noise is furthermore responsible for distortion of spoken instructions [2, p.368]; face or image detection often lacks on illumination and background variation problems; slight changes in pose and/or illumination produce large changes in objects’ visual representation [3], which results in performance loss in visual identification tasks [2, p.21f], [4]). In general, less than 20% of overall information is perceived with the sense of touch; this is quite few regarding to 70% to 80% of information gathered via visual and auditive sensory modalities (approximately 80% of all sensory input is received via the eyes [5], [6], and another 15% via the ears [7, p.41]), but however opens new perspectives for future vehicle-interaction in assisting todays permanently charged eyes and ears with an additional information channel. With this paper we propose a haptic display for vehicledriver notifications, integrated in the car seat and back. Experiments for investigating the potential and performance of haptics, and it’s dependency on human parameters, such as reaction time and age driven, have been processed in a stopped car by using a trace-driven simulation approach in order to avoid road accidents and protect test persons from hazardous situations. The aims of this paper are: (i) A comparison of the adequancy and accuracy of haptic stimulations in contrast to vision and sound, considering following assumptions: (a) A test driver reacts on haptic sensations as fast as on heard or seen stimulations. If this expectation turned out to be correct, we can recommend the transmission of notifications from (at least secondary) tasks in vehicles by haptics.
(b) Mean deviation times for the three interaction modalities are strongly person-dependent, and therefore the modality best suited to use is respecting the individual persons. Furthermore, we believe that specific driving situations probably could be faster resolved with a dedicated modality.
of speech-interfaces, e.g. environmental noise, drivers constitution, interference with conversations, etc.) [10, p.32]. This publications strengthen our resolve in experimentation with different (combinations of) sensory modalities.
(c) Reaction time on haptic stimulation decreases with the progress of the experiment, caused by the fact that haptic feedback is unusally and the user therefore needs to get trained on it.
In [12], Bengtsson et al. reported on using haptic interfaces for improving human-machine interfaces (HMI) without increasing the visual load on the driver (stressed by the necessity of dealing with rising interaction-complexity in vehicles, caused by an increasing number of in-vehicle comfort functions). Amditis et al. have presented design aspects for future automotive environments, focusing on the optimization and adaptation of HMI on driver, vehicle, and environment [13]. One objective of the project was to find the best way to provide information to the driver, concentrating on available information sources in cars (vision, sound, and touch sensory modalities) and drivers’ workload at a specific time. An adaptive integrated driver-vehicle interface concept has been proposed as early result. Information about an event takes different amounts of time to be processed, depending on which sensory channel the event was activated. Harrar and Harris did reaction time experiments with visual, auditive and haptic stimuli (presented in random order). They found, that reaction times were stable following repeated exposures [14]. These results have been used in the definition of our experiment. Jones et al. [15] investigated on tactile cues of visual spatial attention. Test persons had to perform a visual change detection task, following the presentation of a tactile spatial cue on their back (locations corresponds to one of the four quadrants of a computer monitor). They had shown that the cross-modal attentional link between touch and vision is natural and easily established, and that this link is largely a learned strategic shift in attention (it can be broken, e.g. when participants are verbally informed that they should ignore the cues). This seems to be an indicator for the implicit, not distracting perception of haptic stimuli. In [16], Ho et al. reports about experiments on vibrotactile warning signals for presenting spatial information to car drivers. Results show that the presentation of such stimuli on the torso can lead to a shift of visual attention (e.g. time-critical response to visual events seen in the distal space) and confirms our assumption of the potential of vibro-tactile cues for attracting added attention (and it’s use for at least secondary tasks in vehicle-handling).
(ii) A qualitative assessment of driver-vehicle interface performance and reality-connection when replacing on-theroad driving experiments with trace-driven simulation. Outline The paper is organized as follows: The next section gives a detailed overview of related work in human-vehicle interaction, influencing the presented work. Section 3 describes the experimental setting, as well as utilized hard- and software components. Furthermore, experiment execution and regulations are stated here. Section 4 presents and discusses the findings of the conducted experiments, concluding section 5 summarizes the paper.
2
Related Work
A constantly rising cognitive load is caused by factors like (i) increasing number of cars and road signs on the streets, (ii) advanced driver assistance systems, advising the driver about dangerous situations, but also notifying him/her on relatively unimportant system messages, (iii) increasing number and complexity of in-car infotainment systems, requiring drivers’ attention on vehicle- and trafficindependent messages. Arising information needs on the street and in the vehicle increases the risk of driver distraction by taking drivers’ eyes off the road and it’s hands off the steering wheel [8], [9], [10]. The development of safer driver-vehicle interfaces (DVI), without comprising the primary function of driving, and considering the full range of operator behaviour (age, reaction time, etc.), has become a ever important challenge in vehicle design. Vilimek et al. stated, that a single modality does not allow for efficient interaction across all tasks, while multimodal interfaces enable the user to (implicitly) select the best suited [9]. According to Erdogan et al., every modality has it’s own importance and improves special parts of the recognition system [11]. McCallum et al. reports an increase in driving performance, and a dimishing cognitive workload of the driver when using speech interfaces instead of ”normal” interaction paradigms for in-vehicle device control (but without considering the well-known constraints
Haptic Sensations
Human Reaction Times The investigation on human reaction times is of scientific interest since 70 years, but only in the last decades increasing effort has been attempted on driver reaction times in the
automotive field. Much of the research in this filed stems from the early work of Robert Miller (and his 1968-paper on performance analyzing). Exemplarily, in [17] he proposed that a ideal response time, which is essentially to know when designing convenient user interfaces, should not be longer than 2 seconds (later, in [18, p.63], this result was confirmed by Testa et al.). Furthermore, it has been investigated, that mean simple reaction times for light and sound stimuli are below 200msec and that light stimuli are approximately 20% higher than that of sound stimuli1 . We agree to the definition of human reaction time given by Dix [19] as: ”human reaction time is defined as the time it takes a human to react to an event such as a light or a sound”. Shneiderman et al. [20] and Teal et al. [21] defines reaction time from the opposite side as system response time: ”The computer system’s response time is the number of seconds it takes from the moment users initiate an activity until the computer begins to present results”, see „Model response Figureof1.system In [22, p.4] time“ and [20] it was assumed that sysAdapted from Shneiderman, 1998 / Teal, 1992 tem delay has an effect on user performance, and that this effect can be evidenced through increased user productivity at decreased system response times. Triggs and Harris
SYSTEM RESPONSE TIME
(Computation, Overhead, Delay)
HRT
(human reaction time)
USER DELAY
EXECUTION TIME
brake pedal of a car when the traffic light turns to red) when processing ordinary stimuli gains with increasing age. Analysis of traffic accidents in finland shows a drastic ageproportional increase, caused by declining speed of information processing and response time [25]. But on the other hand it has been determined, for instance by L. Breytspraak, that experience with a specific task apparently compensates for the decline with age [26]. In [27], Shaffer and Harrison confirms that (i) human Pacinian Corpuscles (PC) decrease in number with advanced age, (ii) vibro-tactile sensitivity involving PC pathways becomes impaired with age, and (iii) older adults (x=68.6 years) required significantly greater amplitudes of vibration to achieve the same sensation-perceived magnitude as younger subjects (x=23.5 years). Likewise, Smither et al. found that older people experience a decline in the sensitivity of skin, and also have more difficulty in discriminating shapes and textures by touch [28]. Measuring and interpretation of human reaction times has a long history, but was particulary investigated in traditional human-computer interfaces. Due to the fact, that traditional interaction paradigms are mostly not suitable for vehicle handling, evaluations on reaction times have to be repeated in cars, considering available interaction modalities, and facts such as overaging population (as there is evidence that stimulations are age-dependent). In order to prevent road casualties and protect test participants and other drivers from accidents, a trace-driven simulation should be used instead of classical on-the-road studies.
t User initiates system activity
System starts activity response
System completes activity processing
User prepares system input
User initiates system activity
Figure 1. Model of system response time (adapted from Shneiderman [20], Teal [21]).
mentioned that human reaction time depends (linearily) on the number of possible alternatives that can occur [23, p.4]. As one result, cognitive activity is limited to a small number of items at any one time; Testa and Darie discovered in [18], that this number is between five and nine (grouping, sequencing, or neglecting on more items). Age-related Impact on Performance It has been evidenced that the accuracy of perception is affected by the age, exemplarily for haptic stimuli in [24]: They found, that threshold mediated by the Pacinian Mechanoreceptors increases 2.6dB per 10 years (measurements on the fingertips). Response time (e.g. hitting the 1 Reaction Times, URL: http://biae.clemson.edu/bpc/bp/ Lab/110/reaction.htm\#Kinds, retrieved July 29, 2008
3
Experimental Design
Convincing evaluation of human reaction times in the automotive domain, considering visual, auditive and haptic stimuli, is normally carried out using real driving experiments. We propose a trace-driven simulation approach, mainly for two reasons: (i) Repeatability: When performing real driving journeys for each test person, the ride could not be reproduced because of high road traffic dynamics. (ii) Equality: Similar experiment-realization for any test attendee can only be guaranteed with simulation, using a pre-recorded trace of a real trip. In order to prevent road accidents and casualties, and furthermore to secure test participants from injuries, the simulation-driven approach provides additional benefit against on-the-road studies. In the next paragraphs an overview about experimental design, utilized software and hardware components, as well as the processing of the experiment itself, is given.
Signal
Visual
Auditory
Haptic
Turn Left
Symbol ”Left” ”Turn Left. . . ” All 8 left are (Superimposed to the (Spoken instruction) activated simultaneously video) Turn Right Symbol ”Right” ”Turn Right. . . ” All 8 right are (Superimposed to the (Spoken instruction) activated simultaneously video) Lights On∗) Symbol ”Lights on” ”Lights On. . . ” All 6 tactors on the (Superimposed to the (Spoken instruction) seat are oscillating video) Lights Off∗) Symbol ”Lights off” ”Lights Off. . . ” All 6 tactors on the (Superimposed to the (Spoken instruction) seat are oscillating video) ∗) The light switch is a binary item, therefore the same (haptic) patterns could be used for switching on and off. Table 1. Evaluated activities and corresponding feedback signals. Taping: Prior experiment processing we recorded a driving scenario cross-town the city of linz, austria with controlled and uncontrolled crossings, road tunnels and freeway components. Waiting times on crossings and unsubstantial or pointless driving sections had been excluded from the taped run, so that we finally got a video of 11min. 22sec. in length.
„Turn left...“
Perception Time
„Respected sir, at the next crossing please prepare to turn left...“
Visual
Haptic
Auditory (Misconfigured)
Auditory (Revised)
Figure 2. Perception times of individual feedback channels have to be aligned one to each other in order to get meaningful results.
Tagging and Instruction Editor: After that, the video had been integrated into our evaluation application by using the Java Media Framework (JMF), version 2.1.1e. 44 points in the time line had been tagged as initial positions for later triggering specific user actions – initially, only the four ac-
tivities ”turning left”, ”turning right”, ”low-beam lights on”, and ”low-beam lights off” had been differentiated and evaluated. For each of these activities we defined a visual, auditory, and haptic signal (see Table 1 for details). For the tagging task we have internal discussed and commited only to tag valid actions for specific points in time (this means e.g. that a test participant cannot receive a ”turn left”-request at a situation when left turns are impossible in the corresponding video-section). For the assignment of vehicle-control instructions to specific points in the video (on which test participants should later in the experiment react), we have implemented the ”instruction editor”. It is a component of the software framework which enables us to define, modify, and store individual instruction sets per video in a .eli-file (=event list). For the current trace-driven simulation experiment we used this editor only to inspect and adjust the individual trace lines, recorded during the journey as described above. Parameter list of the instruction editor is extendable, and facilitate us up to now the selection of visual, auditive, and haptic notifications. Duration time for user notifications could be set individually, or can be assigned automatically from the software. It is possible to choose a specific interaction modality for each instruction-point (e.g. for testing one driving situation only with haptic instructions), or to let the application select one randomly (which is preferred and has been used in these experiments). Corresponding to the behaviour of the modality, different visuals, audio files, or vibro-tactile patterns could be declared. Additional parameters can be specified for each modality, for example vibration intensity, frequency, duration, occurance, etc. for the haptic feedback channel. Principle structure and parameters of a .eli-file (event list), as well as a short example
Listing 1. Valid task identifiers and their parameters ASSIGNED IDs: 0...Visual Task 1...Auditive Task 2...Haptic Task 3...Random Task (one out of 0, 1 or 2) STRUCTURE OF INDIVIDUAL TASKS (0,1,2,3): ID;Task Name;Stop Sign;Trigger Time (ms);Serial ID;Task Name;Stop Sign;Trigger Time (ms);Serial ID;Task Name;Stop Sign;Trigger Time (ms);Serial ID;Task Name;Stop Sign;Trigger Time (ms);Serial
ID;Image ID;Sound ID;Touch ID;Image
Path;Label Path Path Path;Sound Path;Touch Path;Label
EXAMPLES (SINGLE TASKS ONLY): 0;Right1;s;22484;Turn Right;C:\\driveSim\\images\\turnRight.jpg;right 2;Left1;a;40030;Turn Left;C:\\driveSim\\haptics\\turnLeft.bis 1;Right2;s;69722;Turn Right;C:\\driveSim\\sound\\turnRight.wav
composed with the instruction editor, are shown in Listing 1 below. Sequence Creator: A toolset for defining and organizing a library of individual vibro-tactile patterns, the so-called ”tactogram alphabet”. A ”Tactogram” specifies the dynamic behaviour of all vibro-tactile elements (tactors) in a system. Tactograms could be loaded from the file-system, changed and stored again. Pattern files are indicated by the file extension .bis (=board instruction set). This application is predestinated to be used as rapid-prototyping tool for haptic patterns. Individual templates could be defined, with the possibility for varying in following parameters: (i) arbitrary selection of tactors at each step in the instruction set, (ii) vibration frequency is selectable in the full tactor range (in 10Hz-steps from 10 to 2500Hz, with a nominal center frequency of 250Hz), (iii) four discrete gain levels are selectable, (iv) activation and pause periods are free configurable in ms-resolution (known as pulse-pause ratio). Additionally there is no limit in the length and complexity of a instruction list. A set of on-the-fly defined (or loaded) instructions can be directly transmitted to the tactor system and evaluated instantly. If the perception of the tested pattern is unpleasant, it could be changed immediately at runtime. We have integrated a functionality to verify tactograms without a connected tactor system, simply by inspecting patterns visually on the ”visual tactor board”. Mapping: For the definition of activities we attempted to find intuitive mappings, and furthermore looked after the rule that each of the three signals for a specific activity is recognizeable in approx. the same amount of time (as explained in Figure 2). We discussed our mapping suggestions in detail, and changed them several times; finally we specified the mappings as stated in Table 1. It is quit plain that time for unique identification of patterns would increase
when the number of possible samples is raised (see Testa and Dearie, [18]). In the present case, the number of samples is kept constant for the entire experiment, and across the different feedback channels. Hardware Portion and Experiment Processing: The present study has been processed in a parked car; it was the first in a series of data acquisition experiments in vehicles. The system itself has been designed universally, so that further simulated or on-the-road experiments could be conducted with the same setting. A autonomous power supply system has been provided, transforming the 12V on-board DC voltage to 230V alternative current (required from notebook computer, vibro-tactile controller, etc.). The utilized vibro-tactile actuators (tactor driver) have been selected according to their capability of providing a strong, pointlike sensation that can be easily felt and localized from persons on their body, even through clothings. The experiments were conducted in a comfort station wagon (type Audi A80) which was parked in a single garage near the university campus. In order to prepare for this experiment, following pre-arangements had been made (see images in Figure 4 as well as the second picture of Figure 5 for an overview of experimental setup and placement): (i) the software framework for experiment processing and data acquisition was executed on a notebook computer; (ii) sensors and actuators were interconnected to this notebook computer by using standard USB-ports; (iii) a video-beamer with high luminous intensity had been mounted on the roof of the car, projecting the pre-recorded road journey on a 2x3 meter-sized projection screen, placed ahead the front windshield, so that test participants could see the entire video while sitting on the driver seat; (iv) auditive feedback was delivered with stereo headphones (to prevent distractions from the influence of unintentional environmental noise); (v) visual instructions were displayed on the pro-
No Vibration
Low Vibration
High Vibration
Vibro-tactile Elements TURN RIGHT
SE TUP OF T HE VIBRO -TAC T ILE S EAT (2 stripes of 8 elements each)
(8 tactors vibrating simultaneously)
SWITCH LIGHTS (same signal for on/off)
Figure 3. Setup of the vibro-tactile seat used in the exeriment and visual representation of two patterns (or ”tactograms”) for right turns and switching the lights on/off.
jection screen, as superimposition to the video of the road journey; (vi) vibro-tactile feedback was given with 16 C2 linear tactors (arranged in two strips of eight, as shown in Figure 3); (vi) Users’ reaction times from the directionindicator control or the light switch were captured as electrical signals with a Atmel AVR ATmega8 microcontroller (placed on a STK500 development board, and extended with a voltage regulation circuit; see the left picture in Figure 5) and passed over to the computer, running the playback and evaluation application (see the left picture in Figure 5); (vii) during video-playback, the synchronized traceengine processes events, passes them over to a random generator which selects one out of the three feedback modalities and transmits associated notification signals to the test person in the car (by now activating either the visual, auditory, or haptic channel). Simultaneously, a timer is started, measuring the delay between the notification and corresponding users’ (re)action. The latter is captured from real turn indicators or light switches. The experiment procedure itself, which takes about 15 minutes in time, is fully automated (a supervisor only ensures correct processing of the experiment). For each event which is enqueued by the trace engine, a dataset containing notification time, the channel used for the feedback, user’s reaction time, and the switch firstly activated by the user (to determine if the driver has activated a incorrect switch instead of the right one, e.g. a ”light on”-switch instead of the ”left-turn indicator”-switch) is stored.
4
Evaluation and Results
For the present study, position of vibro-tactile actuators in the seat (respectively on drivers’ body), as well as activation frequency and intensity, has been configured to provide optimal stimulation for Pacinian corpuscles (this kind of receptors are that with highest sensitivity on vibrations). As discussed in the Related Work-section, there is evidence
Trait
Min Max Mean Median Std.Dev. xmin xmax x x e σ All (18 subjects, 15 male, 3 female) Age 18.00 38.00 25.00 25.00 5.12 Weight 50.00 120.00 81.06 75.00 19.29 Size 167.00 197.00 178.94 178.00 6.87 DEY∗) 1.00 20.00 7.11 6.00 4.92 ∗) ”DEY” stands for ”Driving experience in years”.
Table 2. Relevant personal statistics of experiment participants.
that perceived vibration intensity as well as reaction accuracy remains not constant over lifetime. Considering these issues, we selected test participants according to their age – a narrow range of age (and thus, a little standard deviation σ) should improve simulation-results by eliminating age-related distortions. Finally, the experiment has been conducted on 18 test persons (15 male, 3 female) which were university students, research staff and friends in the age-range from 18 to 38 years, all with a valid drivinglicence. Table 2 gives a summary about test participant personal statistics. Briefing: Before starting the simulated driving journey for a test attendee, he/she has been briefed shortly about the requirements, expectations and goals of the experiment. Afterwards, the test has been started immediately, without a run-in test. This possibly causes longer reaction times for the first few stimulation events, and furthermore has an influence to the linear trend lines (which expects a correlation of decreasing reaction times with progress of the experiment), as depicted in Figure 6). Results: Figure 7 shows the reaction times separately for each of the three utilized notification channels vision, sound, and touch (5% confidence interval, 752 datasets).
Figure 4. The garage with projection screen, data-acquisition vehicle, and processing equipment.
Figure 5. ATmega8-16PU microcontroller, placed on a STK500 development system, with external connectors and voltage regulation circuit (left image), schematic representation of the experimental setting for trace-driven simulation in a garage (middle image), video frame with superimposed control window for showing visual notifications (right image).
The channel of sense used for a particular notification has been selected randomly and thus, implicates that the number of tasks for the three modalities is not uniformly distributed (depending on the quality of the random number generator). Tasks in x-axis are sorted according to their occurance over all test attendees (e.g. for the sense of touch: initially, the reaction times of all eighteen ”first” haptic stimuli are selected, after that the reaction times of all ”second” haptic notifications, and so on). Comparing this three stem diagrams only visually, allows already to identify that fastest response has been given on haptic stimulation, followed by visual and auditive notifications. Furthermore, the linear trend lines of reaction time faces downwards for all three modalities – meaning that learning improves reaction performance (the significance of improvement from training can be deduced from the gradient of the trend line; auditive sensory modality performs a little better than haptic, decline of the visual channel is nearly zero). Based on our evaluations (summarized in Table 3), we can state results in more detail (considering the 5% confidence interval): Mean reaction time from haptic notifications (xh = 690.62ms) is 13.56% faster than from visual (xv = 784.25ms), and 63.56% faster than from auditive
Trait
Min Max Mean Median Std.Dev. xmin xmax x x e σ Confidence Interval 3%(768 Datasets, 96.97%) ALL 281.0 4,829.0 931.4 828.0 466.4 Visual 391.0 4,829.0 843.5 703.0 478.3 Auditive 641.0 3,532.0 1,149.0 1,078.0 325.0 Haptic 281.0 4,156.0 705.4 641.0 341.3 Confidence Interval 5%(752 Datasets, 94.94%) ALL 281.0 1,985.0 889.2 812.0 349.9 Visual 391.0 1,922.0 784.3 703.0 295.8 Auditive 641.0 1,984.0 1,129.6 1,078.0 269.6 Haptic 281.0 1,625.0 690.6 641.0 255.9
Table 3. Statistics on reaction times for two conf. intervals, separated for each modality.
(xa = 1, 129.61ms) stimulation. Improvement in response times when using haptic notifications are very promising and support our position of using the sense of touch for providing system feedback to the user for time-critical tasks (in a first step at least for less-important tasks, with the aim to reduce workload on auditive and visual sensory modalities).
60
2000
1800
50 1600
1400
Proportion [Number of counts]
Reaction Time [ms]
40 1200
1000
800
30
20 600
400
10 200
0
20
40
60
80
100
120 Number of Task
140
160
180
200
0
220
0
200
400
600
800
1000 1200 Reaction Times [ms]
1400
1600
1800
2000
0
200
400
600
800
1000 1200 Reaction Times [ms]
1400
1600
1800
2000
0
200
400
600
800
1000 1200 Reaction Times [ms]
1400
1600
1800
2000
60
2000
1800
50 1600
1400
Proportion [Number of counts]
Reaction Time [ms]
40 1200
1000
800
30
20 600
400
10 200
0
50
100
150
200
0
250
Number of Task
60
2000
1800
50 1600
1400
Proportion [Number of counts]
Reaction Time [ms]
40 1200
1000
800
30
20 600
400
10 200
0
50
100
150
200
Number of Task
Figure 6. Reaction times for auditive, visual and haptic stimuli for a 5% confidence interval (from top). Mean reaction time is lowest for haptic stimuli; the linear trend line on reaction times faces downwards for all three notification modalities.
250
0
Figure 7. Histograms shows reaction times for the three notification modalities sound, vision, and touch (confidence interval as in the left column). Reaction on haptic notifications performs best, followed by visual and auditive sensations.
5
Conclusions
We have implemented a novel vehicle-control system based on vibro-tactile feedback, and evaluated it in a simulated navigation scenario against the common used interaction modalities vision and sound. To get an all-purpose test environment with similar conditions for each test participant and reproduceable results as well as to prevent casualties and secure drivers’ from accidents, we used a tracedriven simulation approach to conduct the steering experiments for vehicle-navigation instead of on-the-road evaluation. The simulation trace had been recorded prior the experiment, together with a movie from the driving trip. During experiment execution in a parked car the trace engine processes the pre-recorded trace file, and executes driving tasks precisely and synchronized to the displayed video, so that test participants reported a ”closed to reality” driving behaviour. Results of our excercises on trace-driven simulation can be summarized as follows: (i) Purpose of Haptic Stimuli: Simulation results confirmed our assumptions that haptic feedback is eligible to support interaction based on vision or sound. Reaction times on navigational tasks such as turn or light signals from vibro-tactile stimuli are rather similar to that from visually presented or spoken ones, and often performs better. As a major consequence we can suggest the usage of vibro-tactile interfaces in vehicles, e.g. for relieving the driver from distraction (which contributes to 25% in crashes [p.28][8]), increasing driving comfort, or reducing cognitive load. As vibro-tactile stimulation is age-dependent, this highlights the necessity of compensating impaired proprioception and vibration to establish a universal haptic interface. The maximum measured response time over all modalities was 1,985ms (5% confidence interval) – this value goes along with the recommendation of a maximum response time of 2 seconds (as proposed for example in [18, p.63], [17], or [20, Chap.11]). (ii) Efficiency: As shown in this paper, utilization of tracedriven simulation is viable in human computer interaction (HCI) studies and experiments. Up to now, user experiments have often been designed spontaneously, processing models with random scripts – and therefore with no possibility of repeating it. Results from the trace-driven simulation experiment support our assumption, that the class of trace-driven applications has great potentials in simulating HCI problems. For the purpose of comparing data and for a qualitative proof of the here presented results, we are currently
working on a on-the-road experiment, using the same data acquisition system.
Acknowledgements We would like to acknowledge the valuable help of Martin Mascherbauer and Martin Weniger, both computer science students at the Johannes Kepler University in Linz, for their technical assistance in experiment design and implementation as well as for supporting us in the processing of the experiment.
References [1] A. Wilson and N. Oliver, “Multimodal Sensing for Explicit and Implicit Interaction,” in In Proceedings of the 11th International Conference on HumanComputer Interaction (HCII’05). Mahwah, NJ: Lawrence Erlbaum, Las Vegas, Nevada, 2005. [2] E. Erzin, Y. Yemez, A. M. Tekalp, A. Ercil, H. Erdogan, and H. Abut, “Multimodal person recognition for human-vehicle interaction,” IEEE MultiMedia, vol. 13, no. 2, pp. 18–31, 2006. [3] M. Osadchy and D. Keren, “Image detection under varying illumination and pose,” Proceedings of Eighth IEEE International Conference on Computer Vision (ICCV’01), vol. 2, pp. 668–673, 2001. [4] L. Torres, “Is there any hope for face recognition? (position paper),” in International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2004), Lisboa, Portugal, 21-23 April 2004. [5] G. Mauter and S. Katzki, “The Application of Operational Haptics in Automotive Engineering,” Team for Operational Haptics, Audi AG, Business Briefing: Global Automotive Manufactruing & Technology 2003 pp. 78–80, 2003. [6] B. L. Hills, “Vision, mobility, and perception in driving,” Perception, vol. 9, pp. 183–216, 1980. [7] M. Dahm, Grundlagen der Mensch-ComputerInteraktion, 1st ed., ser. Pearson Studium. Pearson Education, December 2005, 368 pages, ISBN: 978-3-8273-7175-1. [8] N.N., “Driver distraction trends and issues,” IEE Computing & Control Engineering Journal, pp. 28–30, Feb.-March 2005, volume 16, pages 28–30. [9] R. Vilimek, T. Hempel, and B. Otto, “Multimodal interfaces for in-vehicle applications,” in HCI (3), 2007, pp. 216–224.
[10] M. McCallum, J. Campbell, J. Richman, J. Brown, and E. Wiese, “Speech recognition and in-vehicle telematics devices: Potential reductions in driver distraction,” International Journal of Speech Technology, vol. 7, no. 1, pp. 25 – 33, January 2004.
[20] B. Shneiderman and C. Plaisant, Designing the User Interface: Strategies for Effective Human-Computer Interaction. Pearson Education, Inc., AddisonWesley Computing, 2005, 4th edition, ISBN: 0-32119786-0.
[11] H. Erdogan, A. Erc¸il, H. K. Ekenel, S. Y. Bilgin, I. Eden, M. Kirisc¸i, and H. Abut, “Multi-modal person recognition for vehicular applications,” in Multiple Classifier Systems, 2005, pp. 366–375.
[21] S. L. Teal and A. I. Rudnicky, “A performance model of system delay and user strategy selection,” in CHI ’92: Proceedings of the SIGCHI conference on Human factors in computing systems. New York, NY, USA: ACM, 1992, pp. 295–305.
[12] P. Bengtsson, C. Grane, and J. Isaksson, “Haptic/graphic interface for in-vehicle comfort functions,” in Proceedings of the 2nd IEEE International Workshop on Haptic, Audio and Visual Environments and their Applications (HAVE’2003). Piscataway: IEEE Instrumentation and Measurement Society, 2003, pp. 25–29, ISBN: 0-7803-8108-4. [13] A. Amditis, A. Polychronopoulos, L. Andreone, and E. Bekiaris, “Communication and interaction strategies in automotive adaptive interfaces,” Cogn. Technol. Work, vol. 8, no. 3, pp. 193–199, 2006. [14] V. Harrar and L. R. Harris, “The effect of exposure to asynchronous audio, visual, and tactile stimulus combinations on the perception of simultaneity,” Experimental Brain Research, vol. 186, no. 4, pp. 517–524, April 2008, ISSN: 0014-4819 (Print) 1432-1106 (Online). [15] C. M. Jones, R. Gray, C. Spence, and H. Z. Tan, “Directing visual attention with spatially informative and spatially noninformative tactile cues,” Experimental Brain Research, vol. 186, no. 4, pp. 659–669, April 2008, ISSN: 0014-4819 (Print) 1432-1106 (Online). [16] C. Ho, H. Tan, and C. Spence, “Using spatial vibrotactile cues to direct visual attention in driving scenes,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 8, no. 6, pp. 397–412, November 2005. [17] R. B. Miller, “Response time in man-computer conversational transactions,” in Proceedings of AFIPS Fall Joint Computer Conference, vol. 33, 1968, pp. 267– 277. [18] C. J. Testa and D. B. Dearie, “Human factors design criteria in man-computer interaction,” in ACM 74: Proceedings of the 1974 annual conference. New York, NY, USA: ACM, 1974, pp. 61–65. [19] A. Dix, “Closing the Loop: Modelling action, perception and information,” in Advanced Visual Interfaces (AVI’96), T. Catarci, M. F. Costabile, S. Levialdi, and G. Santucci, Eds. ACM Press, 1996, pp. 20–28.
[22] J. A. Hoxmeier and C. DiCesare, “System Response Time and User Satisfaction: An Experimental Study of Browser-based Applications,” in In Proceedings of the Fourth CollECTeR Conference on Electronic Commerce. Breckenridge, Colorado, USA: Collaborative Electronic Commerce Technology and Research (CollECTeR), April 2000. [23] T. J. Triggs and W. G. Harris, “Reactions Time of Drivers to Road Stimuli,” Human Factors Group, Monash University (Accident Research Centre), Department of Psychology, Monash University, Victoria 3800, Australia, Human Factors Report HFR-12, June 1982, ISBN: 0-86746-147-0. [24] A. J. Brammer, J. E. Piercy, S. Nohara, H. Nakamura, and P. L. Auger, “Age-related changes in mechanoreceptor-specific vibrotactile thresholds for normal hands,” The Journal of the Acoustical Society of America, vol. 93, no. 4, p. 2361, April 1993. [25] K. W. Kallus, J. A. J. Schmitt, and D. Benton, “Attention, psychomotor functions and age,” European Journal of Nutrition, vol. 44, no. 8, pp. 465–484, December 2005, ISSN: 1436-6207 (Print) 1436-6215 (Online). [26] L. Breytspraak, “Center on Aging Studies, University of Missouri-Kansas City,” Website, last retrieved July 09 2008, http://missourifamilies.org/quick/agingqa/ agingqa18.htm. [27] S. W. Shaffer and A. L. Harrison, “Aging of the Somatosensory System: A Translational Perspective,” Physical Therapy, vol. 87, no. 2, pp. 193–207, February 2007. [Online]. Available: http://www. ptjournal.org/cgi/content/abstract/87/2/193 [28] J. A. Smither, M. Mouloua, P. A. Hancock, J. Duley, R. Adams, and K. Latorella, Human performance, situation awareness and automation: Current research and trends. Erlbaum, Mahwah, NJ, 2004, ch. Aging and Driving Part I: Implications of Perceptual and Physical Changes, pp. 315–319.
