Indian Traffic Sign Recognition Using HSV Color ...

International Journal of Printing, Packaging & Allied Sciences, Vol. 4, No. 5, December 2016

3381

Indian Traffic Sign Recognition Using HSV Color Model and Kernel Extreme Learning Machine W. Devapriya and C. Nelson Kennedy Babu Abstract--- In Advance Driver Assistance System(ADAS), Traffic Sign Recognition (TSR) is one of the significant area where endless research works are carried out. In the proposed method, TSR system involves two modules: 1) extraction of deep perceptual feature using HSV color model and 2)classification of traffic signs under different categories using Kernel Extreme Learning Machine (KELM) algorithm.HSV color model stands a step toward positive direction than other color model by measuring discriminative ratio. The comparison of KELM under various kernel function like linear, polynomial, RBF, sigmoidal are analysed and it is concluded that the wavelet kernel function gives superior result than other kernel function. For classification purpose, the proposed algorithm has excellent generalization performance, classification accuracy and computational cost when compare to existing methods like ELM, SVM, Conventional Neural Network (CNN),random Forest etc. For analysing the performance two set of traffic sign databases are consider namely GTSDB and own Indian TSDB. Keywords--- Extreme Learning Machine, Kernel Extreme Learning Machine (KELM), Kernel, HSV, Traffic Sign, Wavelet, RBF.

I.

INTRODUCTION TSR is one of vital area on Intelligent Transport System (ITS) for both safety and efficient transportation. A well

programmed TSR framework is very useful for drivers and it is basic for independent automated cars. Traffic difficulties are city traffic congestion, safe driving, detection of sign board and speed bump and transportation efficiency. Though the traffic signs are clearly visible, at some time they are not noticed due to driver distraction or some other disturbances. This can be overcome by the new TSR systems. To be sure, they warn the driver perhaps about the missed traffic sign. The present challenging issues in TSR are changing viewpoint of the roadside, partial occlusion, motion blur, contrast degradation, colour distortion and etc. Traffic signs and street markings are quiet speakers to the street customers. Each road customers ought to know about the signs and their significance. A wide range of traffic signs are seen on the streets. They give proper and required information about street conditions ahead. A street marking additionally gives requests, cautioning or direction to drivers or riders. Basically there are four categories of traffic sign in India as shown in Table 1. Compulsory Sign -These kinds of signs are used to alert driver about the laws and regulations they have to follow for safety and avoid congestion of traffic. These include prohibition or restrictions sign for which the road user must obey. They are circular in shape with red circumference may also contains cross line in black. The abuse of these signs is a legal offence. Sign like no U turn, no entry, no Horn, no left entry will fall under this category. Stop sign which have hexagon shape with red color is also include under this category. W. Devapriya, Electronics and Communication Engineering, KSR Institute for Engineering and Technology, Tiruchengode, Namakkal, Tamilnadu, India. C. Nelson Kennedy Babu, Computer Science Engineering, Dhanalakshmi Srinivasan College of Engineering, Coimbatore, Tamilnadu, India.

ISSN 2320-4387 | © EDITOR IJPPAS


3382

Table 1: Indian Traffic Sign under Each Category

Compulsory Sign

Regulatory Sign

Warning signs

Informatory signs

Regulatory Sign-These signs are an extension to compulsory signs and similarly violation of these is a legal offence, which makes them most important signs. They are circular in shape with blue background and the sign information in white. Give sound horn, turn left, go straight are the sign belongs to this type. Warning signs- These category of signs are used to alert the drivers about the existence of certain risky condition either on or adjacent to the roadway, so that the motorists are careful and take the preferred action. They are triangular in shape with red color circumference with white background. Violation of these is not a legal offence. Informatory signs – These kind of signs are used to provide the information to the road users about Km to reach the destination , nearby Petrol bank, Hotel ,ATM, Hospital, tourist spot and provide other information that will easy the road travel in terms of safe and pleasant. They are most seen as white images which refer the information with blue background. The shape of the sign board is square.

II.

RELATED WORK TSR involves two stages, first determine the feature extraction to represent traffic sign and secondly classify the

traffic sign. Normally the traffic signs features are shape, size, color, statistics data and Transformation data in terms of gradient or orientation energy are commonly used to represent traffic signs, e.g., HOG [1]–[3],scaleinvariant feature transform (SIFT) [4], [5], and Gabor features [6] ,[7] and Feature Based [8] and also in paper [9] feature are determine

using K-d Tree method. Existing TSR scheme consists of hand-crafted features and

conventional classifiers. Even though many hand-crafted features have been created and integrated with classifiers like Neural network (NN) [10]-[12], random forests [9], support vector machine (SVM) [13], [14]. Extreme Learning Machine is the recent classification methods which reduce the computational cost and improve the performance rate also. (ELM) [15]– [17], etc., it is still difficult to deal with the increasing diversity and variability of traffic signs.



III.

3383

FLOWCHART OF THE PROPOSED METHOD

The flowchart of the proposed method consists of pre-processing stage followed by color transformation in HSV and Lab. For the result apply average image subtraction and determine deep features. From the feature extraction result the classification is achieved based on KELM classifier. The flowchart of the proposed method is shown in Fig 1. Preprocessing & Color Transformation

Input Traffic sign RGB image

Average Image Subtraction

KELM classification

Traffic sign Recognized output

Linear

Extract Deep Perceptual Feature

Sigmoidal

RBF

Wavelet

Polynomial

Fig. 1: Flowchart of the Proposed Method a.

Pre-processing& Color Transformation Pre-processing prepare the image before doing the real process. The different formats of input images are jpeg,

bmp, ppm, mpeg, etc. All the formats are converted to a common jpeg format. In addition, this stage include image resizing and enhancement for better analysis. Resize will modifying the image to a fixed size of 25*25 and enhancement is to sharpen the image to highlight the edge of the image. (i) Lab Color Transform The Lab space is made up of three dimensions, that is, L for lightness a and b for the specialized color components. It is clearly stated in paper [17], comparing to other color representation like RGB, Grayscale and



3384

YUV, Lab color performs better in terms of discriminative ratio but HSV model is not considered in the papr[17] .So here the comparison are done for HSV color model. For converting RGB to Lab space Equation(1 to 4) are used, to do this first images should be converted into XYZ space

(1)

(2)

(3) where

(4) Where Xn, Yn and Zn are the tristimulus CIE XYZ values of the reference for white point ii) HSV HSV refers three values namely Hue, Saturation and Value. HSV color model is preferred than other models, because its color representation is similar to human color identification system. The RGB image are converted to HSV color model using equation (5,6 7) The input RGB image and its equivalent Lab and HSV image are shown in Table 2. Hue Calculation

(5) Saturation Calculation

(6)



3385

Value Calculation V =Cmax

(7)

Table 2: Input Image and its Lab and HSV Color Model Traffic Sign Type

Original Input Image

Lab Color model output

HSV color model output

Compulsory Sign

Regulatory Sign

Warning signs

Informatory signs

b.

Average Image Subtraction Before forwarding the HSV and Lab color image to the actual feature extraction, average image subtraction is

done to maintain illumination invariance to alevel. c.

Extracting Deep Perceptual Features Feature extraction is the most important stage where the performance begins. Since HSV color model has good

similarity to human vision. The feature considered are hue, saturation, value , geometric parameter like shape, mean, median, variance, standard deviation, area, correlation, contrast, energy, homogeneity, covariance etc. d.

(i) Extreme Learning Machines (ELM) It is a kind of feedforward neural network method with a single hidden layer node especially for classification

or recognition, where the weights linking inputs to hidden nodes are chosen randomly and not at all updated[15-17]. The weightage between hidden nodes and outputs are acquire in a single step, which helps to learn about the linear model. The architecture diagram of ELM algorithm is shown in Fig 2.These technology gives good overall performance and learning is 1000 times faster compare to backpropagation neural network The input data is mapped to L-dimensional ELM random feature space, and the network output is given by equation (8) fL (x) = ∑Li=1 βi hi (x) = h(x)β ISSN 2320-4387 | © EDITOR IJPPAS

(8)


3386

whereβ = [β1, . . . , βL]T is the output vector weights lies between the hidden layer and output node ℎ(𝑥𝑥)called as a feature mapping since it maps the data from the d input space to the L hidden-layer feature space. h(x) = [h1(x), . . . , hL(x)]is the output vector along row of the hidden layer with respect to the input x.

(9)

(10) Hw,b,x β = T

(11)

The algorithm of ELM contains the following four steps. i.

Generate Hidden layer parameter: The weights wj and biases bj of hidden nodes are selected randomly, for j = 1, 2… M

ii.

Calculate output matrix of hidden layer Hw,b, s and its Moore-Penrose Generalized Inverse H† w,b,s

iii.

Find out the output weight matrix using the equation β = H†w,b,s T.

iv.

Finally the output discriminative function f (x) = Hw,b,xβ, is computed where x is the data input.

Advantages of Using ELM 1.

Compare to back propagation and support vector machine ELM require very less time for training

2.

The performance also shows a good sign compare to BP and SVM

3.

Non-linear activation function works well in ELM

4.

Need to change only the parameter of hidden layer nodes.

Fig. 2: The ELM Architecture Diagram



3387

(ii)Kernel Based Extreme Learning Machine In ELM, the initial factors of hidden layer are not varied and almost all kind of non-linear piecewise functions can be used as the hidden neurons. For reducing the training error and to improve the overall performance of neural networks, the training error and the output weights should be minimized at the same time, which is shown in equation 12. Min:‖𝐻𝐻𝐻𝐻 − 𝑇𝑇‖, ‖𝛽𝛽‖

(12)

The least squares solution of the equation (13) is based on KKT conditions can be written as 1

−1

𝛽𝛽 = 𝐻𝐻 𝑇𝑇 �𝐶𝐶 + 𝐻𝐻𝐻𝐻 𝑇𝑇 � T

(13)

Where H- refer the output hidden layer matrix

C –refer the regulation coefficient, and T - refer the expected output matrix of samples. Then, the output function of the ELM learning algorithm is defined in equation (14) 𝑓𝑓(𝑥𝑥) = ℎ(𝑥𝑥)𝛽𝛽

(14)

If the feature mapping function ℎ(𝑥𝑥) is not known then the ELM based kernel matrix must satisfy the Mercer’s

conditions as defined in equation (15)

𝑀𝑀 = 𝐻𝐻𝐻𝐻𝑇𝑇

(15)

Further, the kernel based extreme learning machine output function (KELM) can be described as shown in equation (16) 1

𝑓𝑓(𝑥𝑥) = [𝑘𝑘(𝑥𝑥, 𝑥𝑥𝑖𝑖 ), … … . 𝑘𝑘(𝑥𝑥, 𝑥𝑥𝑁𝑁 )] �𝐶𝐶 + 𝑀𝑀�

−1

𝑇𝑇 (16)

where 𝑀𝑀 = 𝐻𝐻𝐻𝐻 𝑇𝑇 and 𝑘𝑘(𝑥𝑥, 𝑦𝑦)is the kernel function of hidden neurons of single hidden layer feed-forward neural

networks.

Mercer's condition: Areal-valued functionk(x, y) is said to satisfy Mercer's condition if for all square integral

functions g(x) one has is given in equation (17)

∬ 𝑔𝑔(𝑥𝑥)𝑘𝑘(𝑥𝑥, 𝑦𝑦)𝑔𝑔(𝑦𝑦)𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑 ≥ 0

(17)

Many kernel functions obey the Mercer condition, such as linear kernel, polynomial kernel, Gaussian kernel, hyperbolic kernel, wavelet kernel and RBF. 5 kernel functions are considered for simulation and performance analysis and the kernel functions and its equation are defined in (18 to 22) Linear Kernel:

Polynomial kernel:

Sigmoidal kernel:

𝑘𝑘(𝑥𝑥, 𝑦𝑦) = 𝑥𝑥 𝑇𝑇 𝑦𝑦

(18)

𝑘𝑘(𝑥𝑥, 𝑦𝑦) = (𝑏𝑏. 𝑥𝑥 𝑇𝑇 𝑦𝑦 + 𝑐𝑐)𝑑𝑑 𝑏𝑏 > 0

(19)

(𝑏𝑏𝑥𝑥 𝑇𝑇 𝑦𝑦 + 𝑐𝑐) 𝑘𝑘(𝑥𝑥, 𝑦𝑦) = tanh⁡

(20)



3388

RBF Kernel:

Wavelet kernel:

𝑘𝑘(𝑥𝑥, 𝑦𝑦) = 𝑒𝑒𝑒𝑒𝑒𝑒 �− 𝑘𝑘(𝑥𝑥, 𝑦𝑦) = 𝑐𝑐𝑐𝑐𝑐𝑐 �𝑑𝑑

‖𝑥𝑥 −𝑦𝑦‖2 2𝜎𝜎 2

‖𝑥𝑥 −𝑦𝑦‖ 𝑒𝑒

�

� 𝑒𝑒𝑒𝑒𝑒𝑒 �−

(21)

‖𝑥𝑥−𝑦𝑦‖2 𝑓𝑓

�

(22)

Compared with the ELM learning algorithm, the hidden layer feature mapping prerequisite not be known and no need to select the number of hidden neurons in the KELM. The KELM learning algorithm attains better generalization performance and is steadier compared to traditional ELM and it is faster than support vector machine. The KELM system is trained with various type of kernel function, regularization coefficient, and number or vector of Kernel Parameters.

IV.

EXPERIMENTAL RESULTS

To analyse the performance of the suggested method, a data collection of traffic sign from “German Traffic Sign Detection Benchmark (GTSDB)”are considered. GTSDB consist of more than 50,000 image belonging to 40 classes. But as per Indian road system concern the traffic sign are classified under 4 categories, so a collection of sample from the GTSDB are selected and classified into 4 classes which obeys the Indian road regulation. The notation and representation of color and shape for each traffic sign varies from country to country. Apart from that, some real time sample are collected which are taken on the road side of India. The number of input image under each category from different dataset is shown in the Table3.Tocompare the performance of the proposed method with the most well-known methods described in the literature a parameter called discriminative ratio are measured and tabulated. Table 3: Images in Each Database of 4 Categories Dataset GTSDB Indian DB

Regulatory Sign 512 224

Compulsory Sign 354 108

Warning Sign 489 218

Informatory Sign 519 245

The Discriminative ratio formulae are shown in equation [23] and its value for various color space are tabulated in the Table 4. The table values are compared with the color model like RGB, grayscale, XYZ, Lab color space. The HSV color model has a good discriminative ratio comparing all other color model. Higher the value of Discriminative ratio implies that there is huge inter-class deviation and a lesser the inner-class deviation Table 4: Dr of Various Color Space Color space name RGB Gray scale XYZ Lab HSV

Dr =

Discriminative ratio 0.384 0.366 0.190 0.398 0.405 ∑ni=1 n i ‖μ i −μ‖2 n ∑i=1 ∑x ∈C ‖x −μ i ‖2 i

(23)



3389

where x refers the learned CNN feature of an image in the specified color space, belonging to traffic sign class set. The image class set is denoted as Ci of the i -th traffic sign class, and ni represented the number of images in each class. And μi andμ are the mean image of CNN-learned features of the class Ci and entire training dataset that contains n images respectively. The traffic sign recognition accuracy depends on two major factor that is feature extraction and classification. The following Table 5 and Fig 3 show the comparison between different kernel and database or dataset with respect to recognition rate. While viewing, it shows that performance of wavelet function is higher while comparing to the other type of kernels. In Table 6 the comparisons of KELM with other existing methods are done. For all these analysis the regularization coefficient C is assumed as 1 and the kernel parameter value is also assigned as 1. The default ending epoch is set to 35. The initial weights of hidden nodes in KELM are drawn randomly from a uniform distribution in [−0.01, 0.01].Using the MatConvNet toolbox [20] KELM classification is done. Matlab 2014a is used to conduct all the operations, running on a system with Intel(R), Core(TM), i3-4010U CPU, @1.70 GHz, 1700 MHz,2 Cores(s) Table 5: Recognition Rate for Various Kernel Kernel Linear Polynomial Sigmoidal RBF or Gaussian Wavelet

Dataset GTSDB DB (Indian) GTSDB DB (Indian) GTSDB DB (Indian) GTSDB DB (Indian) GTSDB DB (Indian)

Recognition rate 89.45% 85.78% 91.82% 89.78% 92.48% 90.49% 94.01% 92.08% 97.89% 94.48%

Fig. 3: The Comparison of Recognition Rate for Various Kernel and Dataset Table 6: Training Time and Recognition Rate Comparisons for Various Methods Method SVM CNN- ELM CNN – HLSGD single Deep CNN- KELM

Training Time ~ 142 h ~ 35 h ~ 102 h ~2h


Recognition rate 90.16% 92.48% 94.18% 94.47%


V.

3390

CONCLUSION In this paper, various KELM algorithms are applied on Indian and German (Benchmark) database for TSR. In

KELM, the deep features are learned from the Lab color space and HSV instead of straight away using RGB space. From the analysis it shows that the parameter discriminative ratio is improved by HSV than other color models. It has been verified that the KELM-based feature learning process and the discriminability of the KELM-learned features are influenced by the color space. The recognition rate is improved due to wavelet kernel type. The proposed method uses a relatively simple architecture that decreases the computation cost in term of time and achieve a competitive recognition performance. Future work includes the learning of number of hidden nodes and extending KELM-based classifier for traffic sign detection. Misclassifications due to shadow and motion blurring images are need to be enhanced.

REFERENCES [1]

[2] [3] [4]

[5]

[6] [7] [8]

[9] [10] [11] [12] [13] [14] [15] [16] [17]

I.M. Creusen, R.G.J. Wijnhoven, E. Herbschleb and P.H.N. de, “With. Color exploitation inhog-based traffic sign detection”, 17th IEEE International Conference on Image Processing(ICIP), Pp. 2669 –2672, 2010. Y. Xie, L.F. Liu, C.H. Li and Y.Y. Qu., “Unifying visual saliency with HOG feature learning for traffic sign detection”, Intelligent Vehicles Symposium, Pp. 24–29, 2009. Nadra Ben Romdhane, HazarMliki and Mohamed Hammami, “An Improved Traffic Signs Recognition and Tracking Method for Driver Assistance System”, IEEE, 2016. Y. Aydin, G. TümüklüÖzyer and D. Özdemir, “Traffic sign recognition using scale invariant feature transform and bagging based ensemble”, Signal Processing and Communication Application Conference (SIU), Pp. 605-608, 2016. XiaoguangHUa, Xinyan ZHU, Deren LI and Hui LI, “Traffic Sign Recognition Using Scale Invariant Feature Transform And SVM”, A special joint symposium of ISPRS Technical Commission IV &AutoCarto in conjunction with ASPRS/CaGIS 2010 Fall Specialty Conference, 2010. F. Măriuţ, C. Foşalău, M. Avila and D. Petrişor, “Detection and recognition of traffic signs using Gabor filters”, International Conference on Telecommunications and Signal Processing (TSP), Pp. 554-558, 2011. F. Mˇariut, C. Fo¸salˇau, M. Avila and D. Petri¸sor, “Detection and recognition of traffic signs using Gabor filters”, Proc. Int. Conf.Telecommun. Signal Process, Pp. 554–558, 2011. Dilip Singh Solanki and Dr. GireeshDixit, “Traffic Sign Detection Using Feature Based Method”, International Journal of Advanced Research in Computer Science and Software Engineering, Vol. 5, No. 2, 2015. F. Zaklouta, B. Stanciulescu and O. Hamdoun, “Traffic sign classification using K-d trees and random forests”, Proc. Int. Joint Conf.NeuralNetw., Pp. 2151–2155, 2011. D. Cire san, U. Meier, J. Masci and J. Schmidhuber, “A committee of neural networks for traffic sign classification”, Proc. Int. Joint Conf.NeuralNetw., Pp. 1918–1921, 2011. ArunNandewal, Abhishek Tripathi, Satyam Chandrra, “Indian Traffic Sign Detection and Classification Using Neural Networks”, 2nd International Congress of Technology, Management and Social Sciences-16. J. Jin, K. Fu and C. Zhang, “Traffic sign recognition with hinge loss trained convolutional neural networks”, IEEE Trans. Intell. Transp. Syst., Vol. 15, No. 5, Pp. 1991–2000, 2014 S. Sathiya, M. Balasubramanian and S. Palanivel, “Pattern Recognition Based Detection Recognition of Traffic Sign Using SVM”, International Journal of Engineering and Technology, Vol. 6, No. 2, 2014. Jack Greenhalgh and Majid Mirmehdi, “Real-Time Detection and Recognition of Road Traffic Signs” IEEE Transactions on Intelligent Transportation Systems, Vol. 13, No. 4, 2012. Y.J. Zeng, X. Xu, Y.Q. Fang and K. Zhao, “Traffic sign recognition using deep convolutional networks and extreme learning machine”, Proc. IScIDE, Pp. 272–280, 2015. J. Tang, C. Deng and G.B. Huang, “Extreme learning machine for multilayer perceptron”, IEEE Trans. Neural Netw. Learn. Syst. Yujun Zeng, Xin Xu, Dayong Shen, Yuqiang Fang and Zhipeng Xiao, “Traffic Sign Recognition Using Kernel Extreme Learning Machines With Deep Perceptual Features”, Intelligent Transportation Systems.



[18] [19] [20]

3391

Bin Li, XuewenRong and Yibin Li, “An Improved Kernel Based Extreme Learning Machine for Robot Execution Failures”, Hindawi Publishing Corporation Scientific World Journal, 2014. Huda Noor Dean and K.V.T Jabir, “Real Time Detection and Recognition of Indian Traffic Signs using Matlab”, International Journal of Scientific & Engineering Research, Vol. 4, No. 5, 2013. MatConvNet toolbox.:https://github.com/almazan/matconvnet.git.
