2012 Intelligent Vehicles Symposium Alcalá de Henares, Spain, June 3-7, 2012
Driver Behavior Monitoring System based on Traffic Violation Nourdine Aliane, Javier Fernández, Sergio Bemposta and Mario Mata
Abstract²This paper describes the overall framework and components of an experimental platform for driver behavior monitoring based on GULYHU¶V traffic violation records. This platform is composed of two separate subsystems: a driver assistance system based on road sign detection and recognition, and a traffic violation recording unit in which the vehicle is involved. The system provides drivers with their traffic violation records allowing them to visualize the spatial and temporal information of their traffic violation using the standard Google Earth tool. This feedback can be used to persuade drivers in changing their driving styles by instilling improved behavior. The paper covers firstly the description of the hardware architecture and then presents the developed functionalities. I.
INTRODUCTION
R
oad safety is one of the major policy subjects within the transport policy of the European Union Commission. In 2009 around 35,000 people were killed and more than 1.5 million injured in about 1.15 million traffic accidents on roads in the European Union. This represents approximately 160 billion¼ of cost for society [1]. The majority of traffic accidents is caused by driver inattention [2], [3], distraction due to in-vehicle activities and fatigue [4], [5], and in this aspect, drivers are not fully aware of their inattention or the distracting effects of invehicle tasks on their driving performance. Along these same lines, the majority of traffic violations, such as speeding or ignoring stop signs, are unintentional and they occur due to a lack of concentration rather than because drivers deliberately break the law. These unintentional mistakes could occur as a consequence of variations in performance level rather than through a lack of respect for traffic laws. Thus, driving assistance systems for alerting drivers about their negligent behavior on the road and warning them to be more careful should be considered a primary solution for preventing crashes. It is important to stress that a device that can warn drivers about the speed limit at sites where driving at the wrong speed may result in an accident, and therefore prevent them from doing so, is a crucial safety solution. These warnings should be issued with sufficient notice so that driver has enough time to react to the on-coming traffic situation. Nevertheless, proper Manuscript received 15 January, 2012. This work was supported in part by the Regional Government of Madrid under the S2009/DPI-1509SEGVAUTO grant, and the PN I+D+i under the TRA2010-20225-C03-02 SAMPLER grant $OO WKH DXWKRUV DUH ZLWK ³'SWR 6LVWHPDV ,QIRUPiWLFRV \ $XWRPiWLFD´ DW Universidad Europea de Madrid, Villaviciosa de Odón, Madrid (28670), Spain, (+34-91-211-5671, e-mail:
[email protected]). 978-1-4673-2118-1/$31.00 ©2012 IEEE
driving relies on GULYHUV¶ DFFXUDWH YLVXDO LQIormation and their appropriate reactions. Traffic sign detection and recognition is critical for aiding drivers to understand the road conditions. This situation is particularly severe driving during nighttime, and therefore, road sign detection and recognition is also necessary to reduce the likelihood of missing traffic signs in dark environments. In this aspect, the use of computer vision is a valuable instrument to implement such systems, and some realizations can be found in [6], [7], [8], [9], [10] and [11]. Another approach to enhance safety is by improving driver behavior. This may be achieved by a device able to monitor the driving [12], [13], [14], and [15]. In a recent survey on driving style [16], drivers were asked about a number of risky behaviors and whether they had done these during the last year, and the most prevalent of these behaviors reported were speeding 89% and driving when tired 56% among other issues. Moreover, a research carried out in [17] shows that drivers involved in serious accidents where they are at fault have a traffic violation record with a number of offences that is clearly above the average. Thus, it seems that drivers with a record containing several traffic violations have a much likelihood of becoming involved in a traffic accident. Thus, a system for recording traffic violations can be considered as an alternative for deterring drivers from their aggressive driving rather than conventional punishments. To enhance the driving, and thereby safety, our approach consists in complementing a driver assistance with a system for monitoring driver behavior in the road. This system may contribute to prevent drivers from committing traffic violations and infringements by instilling improved behavior, and can be used to persuade drivers in changing their driving styles. So, this paper presents an experimental platform, hereafter referred to as SACAT, which combines a driver assistance system, based on the use of computer vision for traffic signs recognition able to operate during day and nighttime, and a system for traffic violation recording which is mainly used for monitoring driving. Although, it is far from being commercial, SACAT system offers real and promising framework for experimentation. The rest of the paper is organized as follows: First, system hardware architecture is described. Then, the traffic sign detection and recognition during day and nighttime, and the system for traffic violation recorder are presented. Afterwards, some preliminary results are discussed. Finally, conclusions and future works are drawn.
1096
II. SYSTEM OVERVIEW Developing a test-bed able to capture the vehicle surround and the vehicle state is not an easy task given the excess of hardware and software choices. Besides being affordable, the different alternatives create a challenge for selecting the right equipment. The experimental platform is based on a Nissan Note car model (See Fig. 1), and the embedded hardware consists of a Mini-ITX motherboard as a host computer, and a PC as slave computer for computer vision tasks. The vision system PRXQWHG RQ WKH YHKLFOH¶V URRI is equipped with a panoramic camera for video recording, two IEEE 1394 digital color video cameras for traffic sign detection, and an IR filtered spotlight for night vision. The Mini-ITX motherboard integrates various readily available chipsets, such as a compact-flash memory for data recording, a cardbus slot connected to reading smart card, and a number of connectors such as fire-wire, MPEG2/MPEG4 decoding, or serial ATA, to cite a few. The Mini-ITX motherboard is used not only to centralize the information received from the vision system, but also from external elements such as the GPS and CAN interface. All the electronic devices are powered by a dc/ac inverter, transforming 12V from the vehicle battery to 600W of 220V AC. )URP XVHU¶V SRLQW RI view, SACAT is also equipped with a tactile display located on the dashboard. It permits access to system information as well as performing administrative tasks through an interactive GUI. The hardware architecture, summarized in Fig. 2, is designed with the purpose of making the management of the distributed components with easy reconfiguration, the deployment of different functionalities simpler, and finally the presented architecture permits managing most of the possible applications of on-board computer vision systems in a flexible way. SACAT software is developed using the Matrox Imaging Library (MIL) and the visual C++Builder environment. MIL is a comprehensive set of optimized functions for developing machine vision and imaging software applications. It is a hardware-independent solution, meaning that a developer does not require an in-depth knowledge of the underlying hardware.
Fig. 2. System hardware architecture.
Fig. 3. The snapshot of the SACAT software GUI.
Fig. 1. SACAT: The experimental platform.
A) User profiles As far as system use is concerned, SACAT supports up to three types of user profiles which are as follow: driver as end users, a transport company agent acting as a system administrator, and a traffic enforcement agent (reserved for IXWXUH XVH 8VHU¶ LGHQWLILFDWLRQ LV FDUULHG RXW WKURXJK personal smart cards, which are granted with different permissions to access the different system functionalities. For example, drivers can only view their personal profile and 1097
1098
Another way to analyze driver¶V behavior provided by SACAT is by generating a geographical map that permits to explore and visualize the spatial and temporal information associated with a given trip using the standard Google Earth Map. This tool is graphical and gives divers a precise idea of the vehicle path, its speed, and marks the GPS locations of
the recorded traffic violations. An example of such visualization is shown in Fig. 5 and Fig. 6. By using this map, drivers can retrieve hidden data such as photographs of the surroundings where a traffic violation are committed by just moving the mouse around the map.
Fig. 5. Snapshot of a Google Earth view showing the journey information.
From an implementation point of view, the raw data stored in the database is translated to Keyhole Markup Language (KML) file used for modeling geographic features such as points, lines, images, polygons, and models for display in Google Earth. Like HTML, KML has a tag-based structure with names and attributes used for specific display purposes, and a KML file is processed by Google Earth in a similar way as HTML or XML files are processed by web browsers. IV. DISCUSSION The SACAT traffic sign detection and recognition module is capable of processing up to 4 frames-per-second (fps). So, at normal speed (i.e 100 km/h), the system offers 2 opportunities to identify vertical signs at distances of 50 and 30 meters respectively. Normally, more opportunities are possible at low speed. For daytime processing, the system is fully operational and works in real-time mode. The system was tested under different daytime conditions, such as sunny and cloudy, on highways as well as in an urban area at normal driving
speeds. It is also tested in a night driving scenario. A total of 2000 kilometers of tests have been performed under various illumination conditions. The detection rate (true positive) reaches 90% in clear weather conditions and it performs even better under cloudy conditions. The rate does not depend on the type of road (highway or urban road). However, the aforementioned rate falls slightly during rainy conditions. There are many other negative factors such as shadows caused by trees, the presence of fog, the low contrast of vertical signs due to bad lighting conditions, the VXQ¶V SRVLWLRQ LQ IURQW RI WKH FDPHUD RU when the vehicle is over the brow of a hill. The experiments performed for night vision are conducted in an off-line fashion using a record of 1800 images collected over a number of tests in approximately 3-hours driving, and about 200 images correspond to true traffic signs. The detection rate is about 92%, slightly better than the daytime score. False positive detection rate is less than 1%, and are mainly caused by objects that have similar shapes to some traffic signs.
1099
Fig. 6. Snapshot of a Google Earth view showing an example of a traffic violation of over speeding.
As far as traffic violation recorder is concerned, the time elapsed from sign recognition and issue the alerts to registering the traffic violation obeys different policies depending on the traffic signs. In the case of speeding, the actual vehicle velocity, obtained from the CAN bus is FRQVWDQWO\ FRPSDUHG WR WKH LGHQWLILHG ³6SHHG /LPLW´ VLJQ When driving at 100km/h, and if the speed limit sign is detected at (50 and 25 meters), the system then offers a reaction time of about (2 to 1 seconds). Furthermore, an extra time-out is also provided before registering the traffic violation, (approximately, 25 meters later). In the case of ³6WRS 6LJQ´ DQG ³)RUELGGHQ WXUQLQJ´ ZKLFK QRUPDOO\ WDNH place within urban areas, the typical driving speed is now about 50-70km/h or less. In this case, drivers have a reaction time of about (4 to 3) seconds. In all cases, warnings can be issued with sufficient notice to provide the driver with enough time to react to the on-coming traffic situation. Giving the driver early information will give him the possibility of directing his attention to the most important traffic situations. Finally, it is worth mentioning that the database unit can be used in other avenues to the ones discussed in this paper. For example, it can be used for analyzing movement patterns, journey times, for acquiring more specific GULYHUV¶ parameters, study of why drivers make some decisions in
certain scenarios, and thereby by use it for training and prevention V. CONCLUSION In this paper an experimental platform for monitoring drive r behavior has been presented. The system is aimed at assisting drivers, and more particularly for reminding them of the presence of some specific traffic signs on the road. The warnings come in the form of acoustical messages emitted through the vehicle loudspeakers, and they are issued with sufficient time to provide the driver with enough notice to react to the on-coming traffic situation. Despite the alert and allowed reaction time, if a traffic violation is committed, it is finally recorded. The violation record consists of indicating the type of traffic sign, its GPS location and a photograph of the surroundings, and the YHKLFOH¶V VSHHG SACAT platform focuses at present only on ³6SHHG /LPLW´ ³6WRS 6LJQ´ DQG ³)RUELGGHQ 7XUQLQJ´ violations. From driving behavior monitoring point of view, drivers can be provided by their own traffic violation register as a feedback, and can visualize the spatial and temporal information of their traffic violation register using the standard Google Earth Map. This system may contribute to prevent drivers from committing traffic violation and
1100
1101
