CCIS 395 - Security Increasing Trends in Intelligent ... - Springer Link

Security Increasing Trends in Intelligent Transportation Systems Utilising Modern Image Processing Methods Emília Bubeníková, Mária Franeková, and Peter Holečko University of Žilina 010 26 Žilina, Slovakia {emilia.bubenikova,maria.franekova,peter.holecko}@fel.uniza.sk

Abstract. The paper is focused on a description of modern image processing methods in intelligent vehicles with emphasis on an LDWS (Lane Departure Warning System). The authors describe possible solutions for finding the horizontal road marking using a segmentation method based on a Hough’s transformation. In practical realisation the image data have been recorded in the. AVI format using a camera located near the inner rear mirror of the vehicle. A Sobel’s detector is used for the edge detection, which obtains a binary image containing pixels representing the borders of objects. Keywords: intelligent transportation system, digital image processing, LDWS system, security focused applications, Hough’s transformation, Sobel’s detector.

1

Introduction

The programme to ensure the road safety for the period 2011-2020 established a number of areas that will be subject to the priority of innovation and technological progress in the EU [1]. Its main objective is to build a European strategy to accelerate research, development and use of intelligent integrated safety systems IISS (Intelligent Integrated Safety Systems) including advanced driver assistance systems ADAS (Advanced Driver Assistance Systems) in order to increase the road safety in Europe. The increasing number of accidents is the reason as well as the fact that contributions to attainable standard security measures already reached their limits. Car manufacturers are developing new systems focused on minimisation of accidents occurrence. Collision warning and prevention of collisions are important components of ADAS systems, such as driving in the lane, adaptive cruise control, night vision, etc. These systems are often used as input image data obtained from cameras placed at appropriate locations such as car rear-view mirrors and windshield. Algorithms running in real time can warn the driver in time about an emerging dangerous situation and possibly subsequently react, for example by communication through the inter-VC (Vehicular Communications) [2]. LDWS systems are important representatives of ADAS systems. These are vehicle electronic systems based on computer vision methods [3], [8] that track the vehicle position in the lane and warn the driver if the vehicle deviates or is about to depart out J. Mikulski (Ed.): TST 2013, CCIS 395, pp. 353–360, 2013. © Springer-Verlag Berlin Heidelberg 2013

354

E. Bubeníková, M. Franeková, and P. Holečko

of the road. Many systems were originally developed for truck drivers, who are particularly susceptible to fatigue and sleepiness due to long driving hours.

2

Methods of Lines Detection in the Image

For a software implementation, we used the international standard ISO 17361:2007, which specifies the definition, classification, functions, human machine interface and test methods alerting against unintentional lane departure [4]. When a vehicle is travelling close to the centre of the lane, it is within the system’s “no warning zone”, the system does not issue any position warnings. The lane boundary is defined by lane road markings (Fig.1). Around the lane boundary there is an alert zone with the earliest and latest warning lines. If the vehicle leaves the safe zone and enters the alert zone, the system generates a warning to the driver. The LDWS does not take any automatic action to avoid the lane departure or to control the vehicle; therefore, drivers remain responsible for a safe operation of their vehicles.

Fig. 1. LDWS warning thresholds and warning threshold placement zones

2.1

Line Detection via a Hough’s Transformation

A Hough’s transformation (HT) is a segmentation technique often used in cases, where objects have to be detected with a known shape of the boundary (line, circle and so on) [5], [6]. This method searches for the required shapes in a class of shapes using a polling procedure. The procedure is performed in a corresponding parametric space from which the candidate objects are obtained as local maxima in the accumulation space. It uses the principle that an infinite number of lines can pass a point, each in different direction. The transformation tries to identify one of these lines, which best fits the image data by crossing the most pixels. Its utilisation for lines searching using HT results from a normal line equation:

Security Increasing Trends in Intelligent Transportation Systems

y=

x. cos Θ r , + sin Θ sin Θ

355

(1)

where r represents the smallest distance of line (at point X[x, y]) from the beginning of coordinate system, Θ is the size of oriented angle from the positive x-axis to the half-line guided from the beginning of the coordinate system upright to the searched line (Fig.2). The use of a normal form of line equation has the advantage that it is independent of the axes orientation.

Fig. 2. Hough’s transformation a) Cartesian image space, b) polar Hough-space

Coordinates of pixels x, y forming a line are used as the input data. The values of r and Θ are unknown. The coordinates of pixels are stepwise inserted into the equation and the set of r and Θ solutions forms a continuous curve in the Hough’s space. The results are stored in a so called accumulator 2.2

The Line Search Algorithm

Image brightness values 0 and 1are the input to the algorithm. We are interested in points with brightness value logic 1.If we assume that the total number of such points is K and a line search algorithm (or line) is very simple and can be summarized in words: 1. 2.

3.

We set the accumulator value to zero A[Θi, ri]=0 for all values of Θi,ri, .....i.e. i=1,2, ..., M, j=1,2,..., N. For each pixel in the image (xk,yk) k=1,2,..., K, which brightness value is equal to 1and each value Θi, i=1,2, ...M is calculated: Θ Θ and we increase the contents of accumulator A[Θi,rj] at the position (Θi, rj), i.e. A[Θi,rj]= A[Θi,rj]+1. After processing all the image pixels, the value in the accumulator A[Θi, rj] determines the total number of points nij, that lie in the line with parameters (Θi,rj). We find the maximum value in the accumulator and values Θi, rj, where the maximum occurs, define the line, which contains the highest number of points.

356


The accuracy, i.e. the maximum deviation from the straight line is given by the fineness of space allocation, i.e. the number of cells M and N, which determine the dimensions of accumulator. The minimum and maximum values Θ and D are in the range -90 ≤ Θ ≤ 90 and -D ≤ r ≤ D, where Θ is the size of oriented angle and D is the distance between opposite corners of the image. In most cases before the very transformation a pre-processing method is used, which emphasises the edges of objects. Well known edge detectors can be used (Canny’s, Sobel’s) to obtain a binary image containing pixels representing the borders of objects. The input parameter for line searching using the Hough’s transformation is a colour section of the image being scanned. This colour image section is transformed from an RGB model into a greyscale image (YUV) according to the relation: 0.299 0.147 0.615

0.587 0.289 0.515

0.144 0.436 . 0.100

.

(2)

Next, the image is processed according to the calculated threshold by the edge operator. The authors utilised the Sobel’s method to find edges in the image because of its satisfying results in a relatively short time [7]. 2.3

Algorithm for Line Crossing Detection

The lane and the path of the vehicles are captured by cameras. Then from the video sequences the system estimates the vehicle position on the road and the lane width. Detection of line crossing in a software implementation works on the principle of seeking intersection of line segments. The position of found road marking lines(with HT) is cyclically compared to the segment, which has precisely defined values and represents the car in the image (Fig. 3).

found left line

line representing

found right

vehicle width

line

Fig. 3. Principle of line crossing detection

The intersection of two line segments on a plane can be determined in a general plane using the following mathematical analysis: d ( A, B, C ) = x A ( y B − y C + y A ( x C − x B ) + x B y C − y B x C ),

where A, B, C are points of lines with coordinates A[xA, yA], B[xB, yB], C[xC, yC].

(3)


357

Function d(A,B,C) takes different signs according to the position in the half-plane defined by line BC. So if d(A, P, Q).d(B, P, Q) 0 or s p .sQ > 0.

(5)

That means they do not intersect and the line has not been crossed.

3

Software Implementation and Its Testing

The software implementation of a moving object (vehicle) detection on a road, with a focus on the detection of lane departure detection has been performed in MatlabR2010a using the Signal Processing Toolbox. The whole process of data processing was carried out in a number of logically related steps, which are based on a standard chain of image information processing (data acquisition, digitization, pre-processing, segmentation, object detection and description, understanding the content). The software implemented detection of road markings and crossing lines is schematically shown in Fig.4.

Conversion to YUV Image segmentation Hough‘stranformation

Lane detection

Image acquisition

Lane crossing detection

Block alert Fig. 4. Process of crossing lane detection system implementation and its links to block alert

The authors have chosen a segmentation method based on a Hough’s transformation, which is a standard function of the software in which also the algorithm for line crossing detection from the captured colour video has been implemented. The input data for the algorithm testing were recorded in advance and video sequences of the area in front of a car stored on a hard drive. The image data were first captured using a digital camera, with video recording function with 640x480 pixels

358

E. Bubeníková, M. Franeková, F and P. Holečko

resolution, which was placed under the windscreen in the interior mirror of a mootor vehicle [6]. The algorithm sequentiaally reads individual video frames. Because the video recordings also contain data irrelevant i for line crossing detection (horizon, outer areaa of vehicle’s bonnet) each fram me had to be cropped. The region of interest has been aallocated as a road segment loccated in front of the vehicle and it was defined empiricaally with dimensions of 580x100 pixels. The colour section of thee image of interest is transformed to a greyscale and ussing an automatically determineed threshold it is processed by the Sobel’s edge detecctor. The output of edge operatorr application is a binary image. HT functions (the principle is described above) are applied to the binary imaage. The output is a data structture containing positions of starting and ending pointss of lines, the distance of lines and a the beginning of the coordinate system and the sizee of the oriented angle between n the positive x-axis and the half-line guided from the beginning of the coordinate sy ystem upright to the searched line. If there are broken liines on the road, many lines aree found. Fig.4 presents in red colour the lines found ussing HT. Their parameters are in nput to the algorithm for connection of similar lines, whhich compares the size of the orriented angle between all lines. The comparison finds sseveral favourites, which anglle orientation corresponds to the largest number of liines found at a defined adjustab ble toleration (default value is 5 degrees) and determinees a single line. The outputs are new linees parameters entering a so called poller. The poller’s ttask is to determine validity off the line found from the last 10 frames. The poller ratio value (n out of 10) can be adjusted a before the program start. These data progress to th he next processing using the algorithm for line crossing detection (the principle was mentioned m earlier) and the fact is indicated by an opttical information in a form of lettter „A“ (alert) on the image (Fig. 5).

Fig. 5. Display of alarm in image

To find optimal values of parameters of some input variables, like sensitivityy of lines joining, votes ratio in the poller, alarm threshold setting, many combinationss of input variables settings have been tested. The correctness of detecction has been validated based on the comparison of in advance determined frame numbers in which a subjectively determined line crossing has been determined in the exaamined .AVI record. Consequently these values were coompared with the values deterrmined on the same record using the designed algorithhm. Three line crossings occurrred in the selected testing record. For some rangess of


359

testing input parameters thee line crossing has not been detected at all or the detecttion was delayed by several ten ns of frames. Fig. 6 shows the comparison of subjectivvely determined values of vid deo frames (red colour) and violet values of fram mes determined using the algoriithm.

Fig. 6. Comparison of subjecctively determined frames and frame values when an alarm has been triggered

4

Conclusion

For a practical utilisation of o the algorithm it is desirable to detect line crossing and issue an alert in a visual form even before physical line crossing. Based on this requirement the input param meters values for a correct detection have been selectedd as follows: j (values range 3-5), • sensitivity of lines joining • votes ratio in the po oller (values range 4-6), • alarm threshold settting (values range 10, 20). These values were obtaiined as optimal based on a subjective line crossing coomparison in frames numbered d 50, 168, 409 and the designed algorithm, as it is shownn in Fig. 6. Currently the authors aree working on an extension of line detection to detect teemporary traffic signs, which on o Slovak roads is realised using an orange colour. Tem mporary markings shall have priority p over the original markings, and so their detecttion and recognition could be ussed to more precisely interpret the situation on the road. Acknowledgments. This work w has been supported by the European Regional Devvelopment Fund and the Ministry M of Education of the Slovak Republic, IT TMS 26220220089 “New metho ods of measurement of physical dynamic parameter and interactions of motor vehiccles, traffic flow and road” and by the Educational Grrant Agency of the Slovak Republic (KEGA) Number: 024ŽU-4/2012: Modernizationn of technology and education methods orientated towards the area of cryptography for safety critical applications.

360


References 1. European Strategies, http://ec.europa.eu/transport/strategies/ 2011_white_paper_en.htm (date of access: July 03, 2013) 2. Beňuš, J., Bubeníková, E., Franeková, M.: Effectiveness of digital signature schemes for use in vehicular communication within intelligent transportation system. In: TRANSCOM 2013, June 24-26, pp. 17–20 (2013) ISBN 978-80-554-0692-3 3. Hlaváč, V., Sedláček, M.: Signal and Image Processing (Zpracování signálu a obrazu), ČVUT Praha (2009) 4. ISO 17361: 2007, Intelligent transport systems – Lane departure warning systems – Performance requirements and test procedures (2007) 5. Hough, P.: Methodand Means for Recognizing Complex Patterns, U.S. Patent 3069654 (1962) 6. Zolotová, I., Karch, P.: Contribution to modification of Graph cut method and its implementation in the image segmentation. International Journal of Circuits, Systems and Signal Processing, 49–52 (2012) ISSN 19984464 7. Bubeníková, E., Muzikářová, Ľ., Halgaš, J.: Application of Image Processing in Intelligent Transport Systems. In: 11th IFAC/IEEE International Conference on Programmable Devices and Embedded Systems, Brno, pp. 53–56 (2012) ISBN 978-3-902823-21-2 8. Šimák, V., Hrbček, J., Pirník, R.: Traffic flow video-detection. In: International Conference SSKIVyšná Boca, p. 4 (February 2010) ISBN 978-80-227-3241-3 9. Mikulski, J.: Using telematics in transport. In: Mikulski, J. (ed.) TST 2010. CCIS, vol. 104, pp. 175–182. Springer, Heidelberg (2010) 10. Mikulski, J.: The possibility of using telematics in urban transportation. In: Mikulski, J. (ed.) TST 2011. CCIS, vol. 239, pp. 54–69. Springer, Heidelberg (2011) 11. Mikulski, J.: Telematic Technologies in Transportation. In: Janecki, R., Sierpiński, G. (eds.) Contemporary Transportation Systems. Selected Theoretical and Practical Problems. New Culture of mobility, pp. 131–143. Publishing House of the Silesian University of Technology. Monograph no. 324, Gliwice (2011)