Using Artificial Intelligence to Automatically

Int'l Conf. Artificial Intelligence | ICAI'16 |

151

Using Artificial Intelligence to Automatically Customize Modern Car Infotainment Systems Ashraf Gaffar and Shokoufe Monjezi School of Computing, Informatics, & Decision Systems Engineering Ira A. Fulton Schools of Engineering Arizona State University, Phoenix, AZ, USA

Abstract –In the last decade the automotive industry added digital features to their new cars, including advanced interactive infotainment system. Drivers execute several secondary tasks simultaneously besides the essential driving tasks. More complex infotainment system needs more cognitive attention causing more driver distraction. A consensus from multiple driver distraction research is that multi-tasking causes impaired performance. Although drivers know that distraction is a major reason of car accidents, they continue doing distracting activities during driving. In this paper we introduce a customized user interface instead of a standard user interface to reduce response time and hence driver distraction. While it is hard to directly measure cognitive-related distraction, events that take longer response time by the driver indicate a higher cognitive load as the user needs more time to perceive, analyze and comprehend when compared to simple events. Our hypothesis is that a reduction in the driver’s response time while using infotainment system can indicate a reduction in driver distraction Keywords: human car interaction, response time, driver distraction, cognitive load, cognitive distraction

1

Introduction

The primary task of driving can be challenging for the drivers and requires full attention and focus on the road, continuously watching other cars, street and traffic signs, and pedestrians. Drivers also need to keep an eye on their own car’s instrument panel as well as side and rear views. Additional tasks include operating necessary car controls to turn on lights, windshield wipers or turn signals. While not essential for proper car operation, driver entertainment was added early on due to reduce the boredom of driving. A simple car radio evolved rapidly in features then a CD was added, and eventually a complete digital system with advanced screen display was emerging. As modern cars become predominantly digital in most of its systems, with a lot of information available, industry needed to communicate some of this information to the driver using an information system. Combining this “Information” system with a

complex entertainment system and a significantly larger screen, it is now commonly known as the “Infotainment System, occupying the mid console of many cars. In the last few years, the infotainment system of modern cars saw a rapid growth in complexity as manufacturers kept adding advanced interactive ditgital features to the cockpit [1]. With the avilability of a large screen, a full computing architecture slimila to Von Neuman maching, including multi-core microporcessors, memory, data bus, I/O devices and advanced networking capabilitiy as well as an operating system and software, the infotainment system opened the door to bringing virtually every application that was originally developed for the desktop computing environment into the car cockpit. This migration was seen before as the same trend took place with smart phones evolving into an advanced computing platform with similar features as a desktop. We call this the “First Digital Migration Wave, Wave I”. The “Second Digital Migration Wave, Wave II” is now happening as the infotainment system is evloving in the same way. Unfortunately, the context of use in both waves is dramatically different. While users can generall afford to have full focus on their smartphone screens, with few exceptions, drivers can not. This Wave II evolutionary trend is therefore causing complications not seen in Wave I. Besides the essential driving tasks, some drivers execute several secondary tasks such as texting, listening to the music or checking email simultaneously. These secondary tasks need cognitive attention while some of them also need visual or physical attention, requiring drivers to take their eyes off the road, or their hands off the wheel, causeing distraction. Modern cars with advanced infotainment system often need more cognitive attention, causing more distraction. A growing trend in the research community is to find better ways to manage distraction by improving the interaction between the car and the driver using design principles from Natural User Interfaces (NUI) and Human Computer interaction (HCI) [1]. NUI systems have to adapt to the human behind the wheel instead of forcing people to adapt to the system. Although some of these systems are less

ISBN: 1-60132-438-3, CSREA Press ©

152


cognitively demanding, they all cause driver distraction [2]. Driver distraction is broadly defined as “The diversion of attention away from activities critical for safe driving toward competing for activities” [3]. Several known brain areas are activated during drivingrelated tasks such as visual area, auditory, and spatial working memory. Theta is a brain wave that has a direct correlation with increase drivers errors. When the amount of theta increases in any brain’s areas the result is more driver distraction and more errors during driving. Using devices such as mobile phone increases theta activity so it causes more driver distraction [4][5]. According to fatal accidents report issued by NHTSA [6], invehicle distraction was an important factor in fatal accidents from 1985 to 1995 in England. National Highway Traffic Safety Administration (NHTSA), estimates that in 25% of accidents in the United State of America driver distraction is the main reason of accident. This means 1.2 million incidents each year happen because of driver distraction.

Rank: Driving tasks are inherently distracting. Even a perfect driver cannot focus on the road ahead 100% of the time. A good driver needs to distribute attention wisely between the road, the car instrument panel, the management of car driving-related controls (gas and break pedals), utility controls (lights, turn signals, wipers, windows…), as well as the side view and the rearview mirrors. We call these items to cause “essential distraction”. Other distractions like interaction with a navigation system, radio/CD, or managing additional information or features on the infotainment system or other interactive gadgets in the cockpit are considered non-essential or “dispensable distractions”. There is clearly a complex and intertwined relationship between tens of distractors and their effect on driver distraction. What makes it harder to come up with an objective evaluation is that the driving activity inherently has varying demand of user attention, and rarely requires 100% attention 100% of the time. At certain moments, drivers can partially distribute their attention between different tasks with no harm. In other driving situations, the same behavior can lead to a major accident.

Four kinds of distraction were identified by NHTSA [7]: I. Visual Distraction: Happens when the driver takes his eyes off the road, for example when reading text messages II. Manual Distraction: Happens when the driver’s one or two hands are not on the steering wheel III. Audio Distraction: Any sound such as music, engine’s sound and passengers’ conversation cause audio distraction. IV. Cognitive Distraction: Anything that absorbs thoughts and reduces the driver’s attention. In general, these four kinds of distractions rarely happen in isolation. A deeper analysis of distraction allows us to provide a simple model with specific attributes and relationships. Composition: Most of the time, distraction takes place as a combination of two or more of these four types happening together. We call this the “composite” distraction. Association: Visual, Manual, and Audio distraction is often associated with cognitive distraction. This “Association” can be weak as in the case when a noise is common to the driver. The car’s own engine noise would have weak association to cognitive distraction as the driver is probably used to it. However, a strange or a sudden sound would have a strong cognitive association as the driver tries to think if that was a blown tire, a broken muffler, or something else.

The common conclusion is that multi–tasking during driving a car causes impaired performance. The researcher found that doing second activity during driving causes a competition for cognitive resources among different tasks, the result of this competition is driver distraction. For example listening to sentences while driving reduces the spatial processing by 37% [8]. Although driver distraction is a major contributor to car accidents, car drivers continue doing distracting activities. For example, they regularely use mobile phones, navigation or other features of infotainment system [2]. 75% of drivers believe that hands-free devices are safe to use during driving, they believe this kind of devices doesn’t make distraction but according to the American Automobile Association (AAA) they have a false sense of security [8]. The basis for all distraction detection algorithms is measured and registered during driving. They include driver behaviour (e.g eye movement), driving behaviour (e.g speed) and other data. After collecting these data, some features such as gaze direction are extracted. Driver distraction detection algorithms use these data to estimate driver distraction. For example, “glance away” from the road is a glance put outside the place relevant to driving. When the driver looking away from the road for too long or too often the driver is distracted [2]. An experiment was made by AAA to measure cognitive distraction. In this experiment, a ‘one’ rating means there isn no distraction and ‘five’ means workload is a maximum workload that a human can handle, indicating maximum distraction. They found out that even if the perfect



technology is used in car’s infortainment system it ranked 3 for cognitive distraction [8]. Development of driver assistant systems can reduce the number of car accidents that happen because of driver distraction but sometimes these driver assistant systems don’t work efficiently in the real driving environment. For example, using nonvisual and multi-sensing collision warning system reduces the number of rear-end collisions but In the real driving environment different kinds of noise such as passengers’ conversations , music, and the sound of car’s engine cover the warning noise and sometimes it can make an audio distraction for the driver [9]. The workload system in a car can reduce the distraction that is made by driver assistant system. For example, the lane changing warning is unnecessary when the driver is aware it only causes an audio distraction in this situation. The workload can reduce false alarms or unnecessary alarms by sensing driver inattention to the road and it uses warning about the situation when the driver is truly unaware [10].

153

user interface. The experiment was conducted in a Drive Safety research simulator DS-600s (Figure 1). The DS-600c is a fully integrated, high performance, high fidelity driving simulation system which includes multi-channel audio/ visual systems, a minimum 180° wraparound display, full-width automobile cab (Ford Focus) including windshield, driver and passenger seats, center console, dash and instrumentation, and real-time vehicle motion simulation. It renders visual imagery at 60 frames per second on a sophisticated out-the-window visual display with horizontal field-of-view. It also includes three independently configurable rear view mirrors. An android application was developed to display the user interface. The application was hosted on the Android v4.4.2 based Samsung Galaxy Tab4 8.0 which was connected to Hyper Drive simulator.

Reducing the response time is an appropriate solution to reduce the driver distraction. In this paper, we discuss the details of an experiment that aim to reduce the response time of the driver while she is using infotainment system. In the experiment, the effect of customized user interface in reducing driver distraction was measured.

2

Customized User Interface

The use of application in the car center stack differs from person to person. Six features has been used in our design and each feature includes several commands. Although diversity kinds of commands are available the modern’s car center stack, most of the users only use few commands so a customized center stack would be really helpful since the most frequent commands has been bolded in the customized interface so the user can find them easily and the response time of users that use customized user interface is less than users that use standard user interface. Using setting to customize the user interface is not convenience for all drivers. Some drivers don’t know how to change the setting of their car’s system so it is better to use AI methods to customize the user interface since it is more convenience also the driver doesn’t need to learn how to use setting of his car’s infotainment setting. Each driver has a specific behavior’s pattern during using his car’s infotainment system. This pattern can be found and used to customized the user interface.

2.1

Experiment

We conducted this experiment to show customized interactive interface has a better response time compared to standard

Figure 1: Hyper Drive Simulator in the Experiment 45 people belonging to all sectors participated in this experiment. We asked them to drive on a previously programmed route and we gave them special instructions. Response time and the number of driver’s errors have been measured throughout the experiment. I. Response Time: This is the total time taken by a person to do the navigation operation. Specific action (like play FM) was prompted to the user and the response time was recorded. II. Numbers of Error: This is the number of errors that has been done by each driver during the experiment. Errors can be as simple as not following traffic rules or accidents that has been made by each volunteer driver. Combination of these two factors should help us quantify the distraction level.

2.2

Steps of Execution

Each participant was given few minutes of using the simulator to get familiar with the system. Once everyone completed their preparatory test drives, actual experiment was started. They were asked to drive two times. The first time, when they are driving, which is the primary task, they were asked to do the secondary task like operating the button on customized UI. Response time was recorded. They were


154


asked to drive the second time and interact with the buttons in the same way by using the standard UI. Response time was noted again. The average was calculated. The sequence of execution of different steps: I. User would take a test drive in which he would be briefly informed about actions he is expected to perform. II. User starts driving and the user uses customized user interface in this step. III. He was prompted to play FM when he reached four different locations. (Table1) he should navigate to specific FM page and play a song. IV. He repeats the operation three more times. During the execution of experiment the response time would be recorded and tabulated for each subject. Table I User prompt Play FM

Play FM Play FM

Play FM

2.3

Location Very first left turn when the cars would be coming from different directions. When the cars suddenly change on the freeway On the last left turn of the simulation when the cars bump into each other on the side When pedestrians are crossing the road

If c1 0, then the measurements are statistically insignificant. If c1 and c2 >0 or c1 and c2 < 0, then we can conclude the measurements are statistically significant and if c1 and c2 are positive then we can say alternative 2 is better than alternative 1 and vice versa. The second method that has been used to analysis this experiment is analysis of variance (ANOVA). Analysis of variance is a technique to divide the total variation observed in an experiment into different meaningful components. This technique assumes that the errors in the measurements for different setting are independent with normal distribution. Using ANOVA we can statistically compute: I. The variation observed due to the effect of changing the alternatives. II. The variation observed due the errors in the measurement. Ideally the measurements are significant if the variation observed due to the change of alternatives is greater than the variation due to errors. The variation due to alternatives SSA and variation due to errors SSE are calculated using equation 2 and equation 3. Where (K) being the number of alternatives. In our case k = 2 and n is the number of users in the experiment.

SSA =

S2 a =

-

(3)

(4)

S2 e =

(5)

F value is computed using equation 6: F = S 2 a / S2 e (6) This calculated F value is compared with the value obtained from the table of critical F values. Then we can say the variation due to alternatives is greater than variation due to errors with a confidence level of (1 -α). We will use this statistical analysis to prove that the variation of response time is significantly due to the change in alternative (changing the UI setting of Center stack) rather than the variation due the different users. This also gives a statistical estimate of the effect on the response time due to change in alternative and change in user.

2.4

(1)

(2)

The estimate of variance of alternatives and errors are calculated using equation 4 and equation 5:

Analysis Methods

We used two methods to analysis this experiment. The first method is “comparing two alternatives”. The statistics “comparing between two alternatives” is to compute the mean of differences of the paired measurements. The confidence interval C1 and C2 are determined with a confidence level, say 95%. Equation 1 shows the confidence interval. This is computed using the information of the mean of the differences and standard deviation. If the interval has zero, then the measurements are not statistically significant else we can statistically say that the mean lies within C1 and C2 with 95% confidence. This statistics helps us to estimate if the results of the experiment are statistically significant.

-

SST =

Results

The results of using “computing two alternatives” method (1) to analysis the measured response times in this experiment are as follows (7)(8): C1 = 1366.64581


(7)


C2 = 1700.420857

(8)

Since C1 and C2 are positive, and ‘0’ doesn’t lie in between C1 and C2, the measurements are statistically significant. With 95% confidence level we can statistically say that response time with customized user interface is better than response time with standard user interface. Figure 2 shows the response time with two different kinds of user Interface. You can see in this figure that the response time of driver with customized interface is less than the response time of this driver with standard interface. The results of using “computing two alternatives” method to analysis the number of errors in this experiment are as follows (9),(10):

155

measurements of different user. Since F-computed is greater than F-tabulated, at 95% confidence we can say that the differences seen in the response time by changing the alternatives is statistically significant. We similarly applied ANOVA on the errors measured and we found that 26.9% (SSA/SST) of total variation is due to difference among the two alternatives options and 73.1% (SSE/SST) of total variation is due to noise in the measurements of different user(Table 6). Since F-computed is greater than F-tabulated, at 95% confidence we can say that the differences seen in the response time by changing the alternatives is statistically significant (Table 5). Table 4

C1 = 0.168580822 (9) SSA SST SSE SSA/SST SSE/SST F-computed F-tabulated

C2 = 0.720308066 (10)

381919524.6 512608213.3 130688688.7 0.745051512 0.254948488 257.1677663 3.9846 Table 5

Figure 2: Response Time with Two Modes of UI in Milliseconds Figure 3 shows the number of errors in standard user interface and customized interface. It is obvious that the number of driver’s errors with customized interface was less than the number of driver’s errors with standard user interface. Since C1 and C2 are positive, and ‘0’ doesn’t lie in between C1 and C2, the measurements are statistically significant. With 95% confidence level we can statistically say that the number of errors with customized user interface is less than the number of errors with standard user interface.

SSA SSE SST SSA/SST SSE/SST F-computed F-tabulated

According the results of our experiments we can say that using customized user interface reduced the driver distraction significantly since it decrease the number of errors also it reduced the response time. Using customized interface causes less cognitive load because the most frequent commands has been bolded and driver can find them easier so it makes less cognitive distraction. In addition, it reduces the response time that means driver’s eyes off-the-road time reduces by using customized user interface so the visual distraction reduces too.

3 Figure 3: Number of Errors in Two Modes of UI We applied ANOVA on response time measured (Table 4). Thus we can say 74.5% (SSA/SST) of total variation is due to difference among the two alternatives options and 25.5% (SSE/SST) of total variation is due to noise in the

24.19432099 65.7997037 89.99402469 0.268843638 0.731156362 32.35729231 3.9846

Conclusions

We conducted the experiment with two scenarios, one with Standard user interface found in many cars and another with customized user interface. Response time was measured in both scenarios. In the first case average response time to operate the user interface turned out to be 3.8s to do a specific operation. When we made the participants to drive the car using customized user interface, average response


156


time was reduced to 1.7s which is less than half of the first scenario. The response time is directly proportional to the distraction caused since it is the amount of time the driver keeps working on the secondary task rather than focusing on the road. Reducing the response decreases the driver distraction. We also measured number of errors made during driving in both scenarios. We found that the ratio between numbers of errors made by the driver when operating with the standard user interface to customized user interface is 7:3. There was a reduction of errors by 57% in customized user interface. From the analysis of readings taken during experiment, we can say with 95% confidence level statistically that response time with customized user interface is better than standard user interface. Also errors with customized user interface are less than standard user interface. Hence Customized UI would be easier to interact and reduces driver distraction drastically so the driver distraction with customized user interface is less than standard user interface.

4

References

[1] Bastian Pfleging, Tanja Doring and Ignacio Alvarez. “Auto NUI : 2nd workshop on Automotive Natural User interface ”, AutomotiveUI '12, pp.37-38, October, 2012.

[5] Mustapha Mouloua, Daniela Jaramillo, and Janan Smither.”The effect of iPod use on driver distraction”, proceedings of the human factors and ergonomics society 55th annual meeting, Vol No.41,pp.5886-5888,2011.

[2] Christer Ahlstrom and Katja Kricher. “Review of real time visual driver distraction”, 7th International Conference on Methods and Techniques in Behavioral Research, article No 2,August, 2010.

[6] Nicola Dibben and Victoria J Williams.”An exploratory survey of in-vehicle music listening”, Sychology of music society for education ,music and sychologyresearch, Vol No. 35(4),pp. 571‒589,2007.

[3] Michael A.Regan, John D.Lee and Ignacio Alvarez. ”Driver distraction theory, effects and mitigations”,CRS press , 2009.

[7] www.esurance.com/info/car/3-types-of-distracteddriving, accessed April 16th, 2015.

[4] Mustapha Mouloua, Amber Ahren and Edward Rinalducci.”the effect of text messaging on driver distraction: A bio behavioral analysis ”, proceedings of the human factors and ergonomics society 54th annual meeting ,Vol No. 19,pp.1541-1545,2010.

[8] www.speechtechmag.com , accessed April 10th, 2015. [9] Fanxing Meng, Rob Gray, Cristy Ho, Mujthaba Ahtamad and Charles Spence. ”Dynamic vibrotactile signals for forward collision avoidance warning systems”, Vol.57,No.2,pp.329-346,March 2015. [10] Paul Green. ”Driver distraction telematics design and workload manages: safety issues and solutions”, SAE International,No. 2004-21-0022, 2004.


Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.