Enhancing Road Signs Detection Rate Using MultiScale Retinex Karim AFDEL Laboratory of Computer System & Vision -LabSivFaculty of Sciences, Ibn Zohr University Agadir, Morocco
[email protected]
Abstract— In this paper, a detailed conception of an embedded system of road sign recognition algorithms based on color segmentation, shape analysis and template matching has been made. These techniques are poorly adapted to the Arab context, since some signs are written with Arabic letters. The illumination changes are the greatest obstacle in our work. Therefore, in the first module of the system there is a pre-processing of the image which uses Multi-Scale Retinex and a color model based on normalized RGB color space due to its superior performance in illumination changes such as cloud, fog and dark. The regions of interest are detected using color segmentation. This preprocessing is followed by a Hough transform in order to detect the existing forms. In the second module, the output of the last processing is compared, using the normalized cross correlation function, with our reference database. Experimental results show that the proposed algorithm increases the detection rate of traffic signs. Keywords-component: Road sign; detection; recognition; Normalized Cross-Correlation; Multi-Scale Retinex.
I. INTRODUCTION A road and traffic sign recognition system could in principle be developed as part of an Intelligent Transport Systems (ITS) that continuously monitors the driver, the vehicle and the road in order, for example, to inform the driver in time about upcoming decision points regarding navigation and potentially risky traffic situations [1]. Also, this recognition of the panels is very useful as driver assistance in connection with robotics, aids to navigation in addition to the GPS or diagnostic infrastructure. Generally, there are two phases for road signs identification: first, the traffic signs detection and second, the traffic signs recognition. The quality of detection is necessary to get a good recognition [2]. Signs identification can be classified into three categories:
Color segmentation: interest based on color.
Geometry Methods: The analysis of image contours using mathematical transforms (Hough Transform).
Detection
of regions
of
Abdelah AMGHAR Laboratory of Metrology and Information Processing -LMTIFaculty of Sciences, Ibn Zohr University Agadir, Morocco
[email protected]
Learning Method: a SVM ...) on a database.
classifier training (GMM,
In the identification of signs field, some strategies use a segmentation based on HSV space [3] [4] because it contains no information about the luminance. In the same way is used the normalized RGB [5]. Furthermore, the use of Hough transform, as a robust geometric method for circles detection (speed limit signs or stop) or lines (triangular panels) [6] is useful. Our main idea is to create an embedded system allowing, for instance, for the control of speed according to the limits shown on signs. We plan to improve the performance of some algorithm of signs identification, and adapt it to our cultural context (Arabic signs). II.
APPROACH
Our general strategy, shown in figure1 below, comprises of two main steps. The first is to detect shapes by combining geometric and colorimetric method, which is a candidate to recognize. The second stage seeks to correlate the detected image, after local correction, with reference images.
Input image Detection phase
Mourad BOUSSAID Laboratory of Computer System & Vision -LabSivFaculty of Sciences, Ibn Zohr University Agadir, Morocco
Recognition phase
*Tarik AYAOU Laboratory of Metrology and Information Processing -LMTIFaculty of Sciences, Ibn Zohr University Agadir, Morocco
[email protected]
Pre-processing
Image rectification
Shape analysis
Color segmentation
Canny filter
Characteristic extraction
Cross Correlation with database
Rate?
Data base
Low
Not recognized
High Recognized sign
Figure1. The general strategy adopted for this work.
A. Multi-Scale Retinex Retinex technique was developed by Land and McCann [7].
r=
R R+G+B
It tries to describe how the human visual system interacts with the brightness of natural scene
g=
G R+G+B
b=
B R+G+B
The single scale Retinex [8] is given by: ( , )=
( , )−
[ ( , ) ∗ ( , )]
Where ( , ) is the obtained output, ( , ) is the image distribution in the spectral band, “*” represents the convolution operation, and ( , ) is the surround function: F(x, y) = Kexp −
r c
Where c is the width of the Gaussian, And K is chosen such that: ( , )
=1
While multi-scale Retinex (MSR) is the sum of multiple outputs from the single scale (SSR):
The three components r, g and b are called pure colors. The normalized RGB space was chosen for different reasons; it contains no information about the luminance, we can also work with only two components, which will help to improve the speed of image processing, in addition we use a simple and quick processing to convert the standard RGB to normalized RGB space. After this normalization of the RGB color space, a threshold was used, on red channel, to detect the regions of interest: ( , )
( , ) ( , )
>
&&
( , ) ( , )
>
= ( , )= Where N is the number of scales, R is the ith component of the nth scale, R is the ith spectral component of the MSR output, and w is the weight associated with the nth scale. Gaussian function is also such that:
0
is the threshold. We determinate it using a statistical methed.
r F (x, y) = Kexp − c
3.a 2.a
otherwise
2.b
Figure2. (2.a) the original image, (2.b) The resulting image after the application of Multi-Scale Retinex.
B. Color segmentation In such method, the selection of the color space is very important. There are several works on the comparison of different color spaces [10]. The obtained results show that the normalized RGB, which does not contain information about luminance [11], is one of the most effective spaces for the detection of the regions of interest. The normalized RGB space is obtained using a simple normalization of standard RGB [12]:
3.b
3.c
Figure3. (3.a) The enhanced image, (3.b) the resulting image after converting into normalized RGB, (3.c) the resulting image after thresholding of red component.
C. Shape detection The segmented regions of interest (ROI) are also treated to eliminate false detections and to identify the geometric shapes that represent the real signs. 1) Canny filter In order to detect edges in an image, the canny filter algorithm consists of five separate steps [13]: a) Smoothing To reduce noise in the image, the smoothing step is crucial. In the Canny algorithm, a symmetric twodimensional Gaussian filter is used:
G(x, y) =
1 x +y exp − 2πσ 2σ
Where σ is the standard deviation of the Gaussian. b) Finding gradients and orientations The canny algorithm can detect gradients and orientations of an image by calculating the variance of the first order of each pixel. Detected gradients and orientations can be used for edge detection.
Figure5. Using three points and tangents to detect ellipse centre.
c) Non-maximum suppression This step aims to detect the true contours of the gradients of the image. Generally, all local maxima of the image gradients amplitudes are preserved whereas non-maxima are removed. d) Double threshold For the Canny algorithm distinction, two thresholds were used. If the pixel amplitude is greater than the upper threshold, the pixel is labeled ''high''. If the pixel amplitude is smaller than the low threshold, the pixel is removed. If the value of the pixel amplitude is between the two thresholds, the pixel is labeled ''low''. e) Edge tracking In the Canny algorithm, the strong pixels are caused by real contours, which must be preserved, while the weak pixels may be caused by noise must be removed.
Figure6. (6.a) image after application of Canny operator, (6.b) the resulting image after application of Hough transform.
D. Image correction Validation and identification of the form detected are made by matching it with a database of reference signs. However, the signs are detected under the effect of distorted perspective and therefore they are not in the same geometric of the panels of reference. This makes the matching complex. We therefore propose to rectify the image locally to put it in the same geometry as the reference template [9]: =
4.a
a x′ + b y′ + c a x′ + b y′ + 1
a x′ + b y′ + c a x′ + b y ′ + 1
A B
2) Hough transform The application of the Hough transform on contours, detected by the canny filter, allows us to detect the existing forms in the processed image. We used in our work the Hough transform based ellipse detection algorithm.
=
A
4.b
Figure4. (4.a) the segmented image, (4.b) the result image after application of Canny operator.
6.b
6.a
D C
D
B C
Figure7. Using of four points for the estimation of the required correction
Our approach is based on using three points and tangents to find the parameters of the ellipse. The characteristic equation of an ellipse is on the form: ( − ) + 2 ( − )( − ) + ( − ) = 1 Figure8. Example of image correction
E. Sign recognition The correlation based template matching is the simplest way to find or to validate an object in the image. Using the normalized cross correlation, we can detect good results about the similarity between the histogram of the image detected f and the histogram of the reference image t: ∑
( , )= ∑
∑
∑
[ ( , ). ( , ( , ). ∑
∑
)] ( ,
)
The range of NCC is [0.1], where the highest value indicates the best match. F. Experimental results The performance of the proposed technique is measured by testing it with several images which contains several Arabic stop with different position and scale. Our proposed method is implemented using C++ and OpenCV library.
III.
A robust algorithm for detecting road signs is designed in this paper. For the identification of road signs, detection, which is sensitive to illumination changes, represents a crucial step. These variations of illumination, such as fog and cloud, significantly decrease the detection rate and also increase the rate of false detection. The application of Multi-Scale Retinex, that improves the distribution of color in the image, allows us to obtain a good segmentation and also improves the edge detection. Subsequently, we increased the recognition rate of road signs from 75% to 87.5%. The experimental tests have validated our technique and shown its robustness. In the near future we plan to improve our system by using a GMM classifier and apply it on video frames to recognize the Arabic signs. REFERENCES [1] [2]
The following definitions are applied: Success Rate: (Total of Success/ Total Number of Input Sample) x100 %. Failure Rate= (Total Number of Failure/ Total number of Input Sample) x100 %. TABLE I.
Before using MSR Using MSR
Experimental detection of traffic signs
Total number of sign
Total number of Arabic stop extracted
Success rate
40
30
75%
25%
40
35
87.5
12.5
[3]
[4]
[5]
Failure rate [6]
[7] [8]
[9] [10]
[11]
[12]
[13]
Figure9. Results of correlation between the histogram of the detected image and that of the reference image.
CONCLUSION
H. Fleyeh, “Traffic and Road Sign Recognition”, Thesis, 2008. Rachid Belaroussi and Jean-Philippe Tarel, “Détection des panneaux de signalisation routière par accumulation bivariée” , Traitement du signal. Volume 27–n˚ 3, pages 265 à 297, 2010. A. De la Escalera, J. Armingol, M. Mata, “Traffic sign recognition and analysis for intelligent vehicles”, Image and Vision Computing, vol.21, n˚3, p.247-258, 2003. D. Shaposhnikov, N. Lubov, L. Podladchikova, A. Golovan, Shevtsova N., Hong K., Gao X., “Road Sign Recognition by Single Positioning of Space-Variant Sensor Window”, Proceedings of 15th International Conference on Vision Interface, Calgary, Canada, p.213-217, 2002. G. Dutilleux, P. Charbonnier, “Métaheuristiques biologiques pour la détection de la signalisation routière”, in P. Siarry (ed.), Optimisation en traitement du signal et de l’image,Traité IC2, série traitement du signal et de l’image, Hermes, chapter 10, p.271-294, février, 2007. M. Garcia-Garrido, Sotelo M., Martin-Gorostiza E., “Fast traffic sign detection and recognition under changing lighting conditions”, Proceedings of IEEE Intelligent Transportation Systems Conference (ITSC’06), Toronto, Canada, p.811-816, 2006. E.H. Land and J.J. McCANN. “Lightness and retinex theory”, J. Opt. Soc. Am., 61(1):1-11, January 1971. Daniel J. Jobson, Zia-ur Rahman, and Glenn A.Woodell, “A Multiscale Retinex for Bridging the GapBetween Color Images and the Human Observation of Scenes”, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL.6, NO.7, JULY 1997. B. Soheilian, A. Arlicot et N. Paparoditis, “Extraction de panneaux de signalisation routière dans des images couleurs”, 2010. J. Terrillon and S. Akamatsu, "Comparative Performance of Different Chrominance Spaces for Color Segmentation and Detection of Human Faces in Complex Scene Image", 1999 S. Kannumuri and A.N. Rajagopalan, 2002, Human Face Detection in Cluttered Color Images Using Skin Color and Edge Information, Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP'02), (Ahmedabad, India). L. Mostafa and S. Abdelazeem, “Face Detection Based on Skin Color Using Neural Networks”, GVIP 05 Conference, CICC, Cairo, Egypt, 2005,. Long Xiang and Wu Xiaoqing. Motion Segmentation Based on Fusion of MSRF Segmentation and Canny Operator. Procedia Engineering, 15 1637 – 1641, 2011.
