Proceedings of the 2010 IEEE International Conference on Information and Automation June 20 - 23, Harbin, China
License Plate-Location using Adaboost Algorithm Xiangdong Zhang, Peiyi Shen, Yuli Xiao and Bo Li
Yang Hu, Dongpo Qi, Xiao Xiao and Liang Zhang
Xidian University, Xi’an,Shaanxi Province, China {xdchen, pyshen}@xidian.edu.cn
Xidian University, Xi’an, Shaanxi Province, China {xiaoxiao, liangzhang}@xidian.edu.cn
is reached. In this project, we select features like variance[2], derivative, texture features and so on. We optimize the performance by introducing a cascade of strong classifiers, which greatly speeds up the scanning process, as not all classifiers need be evaluated to rule out most non-license plate sub-regions.
Abstract— License Plate Recognition(LPR) is a very important research topic in computer vision of ITS. License plate location is the key step of LPR. Though numerous of techniques have been developed, most approaches work only under restricted conditions such as fixed illumination, limited vehicle license plates,and simple backgrounds. This paper attempts to use the AdaBoost algorithm to build up classifiers based on various features. Combining the classifiers using different features, we obtain a cascade classifier. Then the cascade classifier which consist of many layers of strong classifiers is implemented to locate the license plate.The training speed of the traditional AdaBoost Algorithm is slow. In order to increase the training speed, different features like derivative, texture are included. The classifiers based on the features we selected decrease the complexity of the system. The encouraging training speed is achieved in the experiments. Compared with other LPR method, for instance, color-based processing methods, our algorithm can detect the license plates with accurate sizes, positions and more complex backgrounds. Index Terms— Adaboost Algorithm, License Plate-Location, Feature selection
II. A LGORITHM The basic idea of AdaBoost is to get a strong classifier by selecting a combination of weak classifiers. The weak classifiers are weak because their correct rates are not very high. We get the set of weak classifiers by the training procedure which selects the best weak classifier with the lowest error rate after each round of training. The following steps show the training procedure: 1)The training data set(x1 , y1 ),....,(xl , yl ) must be prepared carefully, you had better choose the images which often appear in the normal circumstance. xi ∈ X..... stand for the sample and each yi ∈ Y = {0, 1}, Yi = 0 means this sample is a positive one, Yi = 1 with the opposite meaning, l is the total number of the samples in the set. 2)At beginning, the weight of every sample is initialized with the value 1/2m or 1/2n, m indicates the number of positive samples while n for negative ones, thus each positive sample will get the weight 1/2m , the negative will get a 1/2n as well. 3)The training will be done for t = 1, ..., T rounds, after each round, we will get a weak classifier. To start the nprocess the weights must be normalized as wt,i = wt,i / j=1 wt,j respectively, to make a wt,i probability distribution. Then for each feature(weak classifier), the error rate is evaluated with respected to wt : εj = wi |hj (xi , yi )| (1)
I. I NTRODUCTION Although many LPR systems had been developed, we must admit that lots of them only work efficiently in certain environments. Like the techniques based upon combinations of edge statistics and the mathematical morphology can not work well with complex images, color-based processing method do not provide a high degree of accuracy because of the color is not stable when the lighting conditions change. So the detection of license plates under various circumstances is still a challenging task. The AdaBoost[1] Algorithm has been adopted in many LPR systems, it has the advantage of being invariant to the color, size and the position of license plates, that is the reason why we chose it. This paper focus on explaining the features we used in adaboost. Features we used in our program are given and how the features are chosen and the differences between plate feature values and non plate feature values are explained in detail. The number of features we selected for AdaBoost training is more classical and efficient than the traditional one so that a improvement in the time cost of training procedure
4)Find the weak classifier ht with the least error rate. 5) Get the weights updated according to: wt+1,i = wt,i βt1−ei
∗ Contact
Auther Peiyi Shen.This work is partially supported by NSFC Grant #60672117. This work is aslo supported by the Open Projects Program of National Laboratory of Pattern Recognition
978-1-4244-5704-5/10/$26.00 ©2010 IEEE
(2)
Where ei = 0 if xi is classified correctly, otherwise ei = 1,and βt = t /1 − t . It forces the weak classifiers
2456
to concentrate on the ”harder” examples that are often misclassified. 6)Finally we get the strong classifier which combines the weak classifiers’ respective votes in a weighted manner: T
1,
t=1 C(x) = {0,other
αt ht (x)≥(1/2)
T
αt
t=1
αt = log 1/βt
(a)
(b)
Fig. 2. Gray level features used in our program. In(a), the feature value is the image pixels that fall within red rectangle subtracted from the image pixels that within the blue rectangle. In(b),the feature value is the image pixels that fall within red rectangle subtracted from the image pixels that fall out of the red rectangle.
(3)
7)The strong classifier we have got above will then be applied to the scanning of a whole image to find the subregions covers a license plate.
positive samples’ feature value and negative samples’ feature value were demonstrated. Similarly, as shown in Fig.2 (b), another gray level feature which calculate over two blocks can be got.The image pixels that fall within red rectangle are subtracted from the image pixels that are fall out of the red rectangle takes as the value of this feature. Since the variance of brightness in plate-area is remarkable, in a binary image, the change of pixel value is very frequent than elsewhere. So several features can be concluded from the binary images. Fig.3 gives the RGB image of a vehicle and its binary image is shown in Fig.4. In Fig.4 the plate area is marked with red rectangle and we can see that variance of brightness in this area is remarkable.
III. F EATURES This paper focus on explaining the features we used rather than adaboost algorithm itself. In this project, considering the certain features we know which have a good discrimination between positive samples and negative samples like derivative, texture features, and so on, we will use a variation of the Haar-like features used by Viola and Jones[1]. We divide the window into several rectangles and compute values for certain features in each rectangle[3], the difference of the calculation between certain rectangles is the result we want. Some features we selected and the reason why they were chosen are given as follows . As we have already normalized the size of all plates into 20 × 70 pixels, it is easy to be recognized that there must be a distinction in the value of the some blocks’ sum of pixel intensities. For example, in Fig.2 (a), the red block’s value will be greater than the blue one if the window hit plates, while in a non-plate-area, this value could be either positive or negative. The image pixels that fall within red rectangle are subtracted from the image pixels that are within the blue rectangle. The resulting number is considered to be the value of this feature. 4
20000
6
x 10
4
15000
2 10000 0 5000 −2
0
−5000
−4
0
200
400
600
(a)
800
1000
1200
−6
0
1000
2000
3000
4000
5000
6000
Fig. 3.
7000
(b)
RGB image of a vehicle
For example, the number of the vertical changes has some relations with the horizontal one, so their difference and quotient can be computed as a feature. The number of horizontal changes of pixel value itself also could be treated as a single feature, another feature is the number of vertical changes. As is shown in Fig.5. To get some new kinds of features, a variation of the Haarlike features is used here, the window is divided into two blocks and for each block the number of vertical changes of pixel value is calculated, then the difference of the two value is introduced as the value of this feature. We use this variation to rule out the window which has half a plate in it, as shown in Fig.6 (a). In order to get the
Fig. 1. Comparison between the values of positive samples and negative samples under the same gray level feature that shown in Fig.2(a). In fig.1(a) The x-axis represents different positive samples, The y-axis represents gray level feature values of the samples. while In fig.1(b) represents negative samples.
Fig.1 illustrates the comparison between positive samples’ gray level feature values and negative samples’ gray level feature values.In fig.1(a) The x-axis represents different positive samples ,The y-axis represents gray level feature values of the samples. In fig.1(b) The x-axis represents different negative samples ,The y-axis represents gray level feature values of the samples. From Fig.1, the different distribution between
2457
variance might be a qualified choice. Several features can be concluded if using the same ideas we created features above. Fig.7 shows two features: 0.4
0.2 0.18
0.35
0.16 0.3 0.14 0.25
0.12
0.2
0.1 0.08
0.15
0.06 0.1 0.04 0.05 0 −8000
0.02 −6000
−4000
−2000
0
2000
4000
0 −1
−0.5
0
0.5
1 5
x 10
(a)
Fig. 7. Distribution of the feature valus of variance of X-derivative (a) and the variance of Y-derivative (b), where the blue curve is for the positive samples and the red curve is for the negative samples.
Fig. 4. In a binary image, the changes of brightness in plate-area yields a wealth of information. 0.2
0.2
0.18
0.18
0.16
0.16
0.14
0.14
0.12
0.12
0.1
0.1
0.08
0.08
0.06
0.06
0.04
0.04
0.02 0
Though many efficient features we have, the error rate is still too high to put the system into use. The samples which could not be classified correctly often appears to be the lights, trees, and the radiator region in the front view of the vehicle. To effectively eliminate unwanted regions, new kind of features must be introduced. So we select texture features to achieve this scope. Texture[4] is a rich source of visual information. The complex nature of texture has resulted in a large number of representations and an even greater number of texture features that may be derived from these representations. Empirical features and perceptual features are the two types of textures we introduced. Empirical texture features are those features that do not have any corresponding humanvisual interpretation. The Grey-Level Co-occurrence matrices (GLCM)[4] method is one of the most extensively used representations of texture. We select some of the common texture features that are often derived from the GLCM, they are energy, entropy, contrast, mean, homogeneity, variance, correlation and uniformity[4]. As shown in Table.1, we list the calculation of each feature derived from GLCM(P stands for the GLCM). Fig.8 displays distribution of two features which derived from GLCM. For perceptual features, the Tamura features are the most widely acknowledged set of that. The Tamura features used in this system are coarseness, contrast, direction, line-likeness, regularity and roughness. The calculation of Tamura features is very computationally expensive thus it should not be included in the first or second layers of the cascade classifier. But it really has an impressive ability to eliminate unwanted regions mentioned above. If the number of areas to be classified is very small, these features would not increase the computation time too much. The calculation of the Tamura features is complex. Here we give calculation in detail. Coarseness is the most fundamental of Tamura perceptual features as it is probably
0.02 0
50
100
150
(a)
200
250
0 −300
−200
−100
0
100
200
300
(b)
Fig. 5. Distribution of the feature values of the number of the horizontal changes (a) and the difference of the number of the vertical changes and the horizontal ones’ (b), where the red curve is for the positive samples and the blue curve is for the negative samples.
accurate location in vertical direction, another technique has been developed. As shown in Fig.6 (b), the top and bottom area of a plate may have less horizontal change of pixel value, thus divide the window into three blocks, noticed that the area of the middle one is the twice of either the top or the bottom, the calculation of the number of vertical changes of pixel value is carried on each of the blocks, the value of this feature is the sum of the changes in the red rectangle subtracted from the other two blocks. For a similar reason, a new feature could be found by substituting the horizontal change of pixel value for the sum of pixel intensities. For the features about the variance of brightness above, the image is a binary one thus we could count the number of changes in either direction. If we use a gray-level image to represent the change of the intensive of pixel values,
(a)
(b)
(b)
Fig. 6. Hopping features based on brightness changes in binary images. These features’ values are the differences of the number of the changes in red blocks and that of remainders.
2458
Texture Feature
Equation N −1
Uniformity
i,j=0
√
Energy
U nif ormity
N −1
Entropy
i,j=0
Fig. 9.
Pi,j (− ln Pi,j )
N −1
Contrast
2 Pi,j
i,j=0
Pi,j (i − j)2
moving averages on opposite sides of the point is calculated. This calculation is computed for both the horizontal and vertical directions as shown in following equations respectively.
i,j=0
Pi,j N −1 1+(i−j)2
Homogeneity
i,j=0
Correlation
N −1
Pi,j [
σi2 =
Variance
(i−μi )(j−μj ) ] σi2 σj2
N −1
μi =
Mean
i,j=0
N −1
i,j=0
GLCM
0.06 0.05 0.04 0.03 0.04 0.02 0.02 0.01
2
3
4
5
0 0.4
0.5
0.6
(a)
0.7
0.8
0.9
1
(b)
Fig. 8. Distribution of the feature valus of entropy (a) and homogeneity (b), where the blue curve is for the positive samples and the red curve is for the negative samples.
n−1 n−1 PDd (i,j) cos[(i−j) 2π ] j=0 i=0 n ,if n−1 PDd (i,j) i=0
the most visually meaningful. The distribution of the feature valus of Tamura Coarseness is given in Fig.9. Following steps show the calculation thoroughly. First we compute the moving averages for the various sized windows. The various sized windows are taken to be powers of two, i.e. 2k × 2k . In this system we calculate windows for k = 0, ..., 3. The average over the neighbourhood of size 2k × 2k at the point (x, y) is given as: Ak (x, y) =
k−1 k−1 −1 x+2 −1 y+2
i=x−2k−1
f (i, j)/22k
n
(8)
To calculate Tamura line-likeness a directional GLCM is constructed such that it contains data indicating the orientations of the line-like edges. The scalar value of the displacement vector is chosen to be 4. For a given edge we must select another edge in the image in order to determine whether they satisfy the line-likeness condition. In order to select the corresponding edge we take the orientation of the edge in question and decompose it into its x and y components as illustrated in the Fig.10. If the orientation of the corresponding edge is within a certain range of the orientation of the first edge then the edges are determined to be line-like and their orientation values are updated in the GLCM. The equation given in Tamura paper for the calculation of the line-likeness feature is shown below.
0.08
1
(7)
1 Sbest (i, j) m ∗ n i=1 j=1 m
Fcrs =
0.08
0
(6)
The final coarseness value F crs is calculated by averaging Sbest over the entire image.
Pi,j (i − μi )2
0.07
0
Ek,v (x, y) = |Ak (x, y + 2k−1 ) − Ak (x, y − 2k−1 )|
Sbest (x, y) = 2k
0.1
0.06
(5)
i(Pi,j )
A LISTING OF THE MOST COMMONLY DERIVED H ARALICK FEATURES
0.12
Ek,h (x, y) = |Ak (x + 2k−1 , y) − Ak (x − 2k−1 , y)|
At each pixel we choose the value of k that maximizes E in either direction in order to get the highest output value:
TABLE I FROM THE
The orientation of the edge decompose into x and y components
Flin = {0,otherwise
n−1 n−1 i=0
j=0
PDd (i,j)>t
(9)
Tamura regularity feature is intended to capture variations in the placement rule observed in a texture. A texture with no variation in the placement rule would be observed as regular, whereas a texture with large variation in the placement rule would be observed as irregular. It was noted however that natural textures reduce the regularity in general because of their inherent qualities so difference may appears between plates and natural areas. The regularity feature is calculated using the equation:
(4)
j=y−2k−1
Where f (i, j) is the grey-level value at the point (x, y). At each point, the differences between pairs of non-overlapping
Freg = 1 − γ(σcrs + σcon + σdir + σlin )
2459
(10)
The σ values represent the standard deviation of each of the features. As previously mentioned, the standard deviation will be calculated using the feature values from each of the sub-images. The value γ is a normalizing factor which is necessary if not all of the images are of the same size, of course here we use the same size window. Roughness can be calculated as follows. Frgh = Fcrs + Fcon
of the positive samples are shown in Fig.11 while some of the negative samples are shown in Fig.10. A set of 100 images was used for testing, each image containing a license plate, and the evaluating to the performance of the trained classifier will be shown at section result. V. A DABOOST TRAINING
(11)
In this project, we build a cascade classifier for LP detection with four layers. The number of weak classifiers in each layer is about 6-8 and the first layer’s is the least thus the time cost can be reduced rapidly. Since the speed of the computation in the first layer is very important, several features that compute easily are selected for the training of the first layer. While for the training of other layers in the cascade classifier, the computationally expensive features like textures might be included. In our system the first layer of the cascade classifier has the features like variance, X-derivative, Y-derivative, the gray level features as shown in Fig.2 and hopping features shown in Fig.6 (b) and Fig. 5 (b). The second layer has features like X-derivative, the number of the vertical changes and horizontal changes, the gray level feature shown in Fig.2 (b), variance, hopping features shown in Fig.6 (a). The third layer and the fourth layer contain features like hopping features in Fig.6 (a), Fig. 5 (b), variance, Y-derivative, Mean, Correlation, Energy, Tamura Coarseness, Tamura Roughness and Tamura Regularity. Noticed that keep the number of positive samples much smaller than the negative samples for the training can give each positive sample a much higher weight, as a result, during the training the features which would not misclassified positive samples to a high degree would be chosen firstly, thus each layers will has a lower rate of the missing of license plates.
A weak classifier is a function which can produce a binary value which stands for yes or no by compare the value calculated above to a threshold. IV. DATA S ET We manually extract license plate areas in the images captured from the camera by streets. All the images have a resolution of 1040 × 1392, The license plates are normalized to the size 20 × 70 and all negative samples are randomly sampling 20 × 70 windows of parts of images known not to contain any parts of license plates.
Fig. 10. Some negative samples normalized used to train the weak classifiers as data sets which are collected randomly.
Fig. 11. Some positive samples normalized of the license plates used to train the weak classifiers as data sets. Fig. 12.
At last, 1011 plates were put into training as positive samples while the number of negative samples is 6074. Some
2460
More than one plates are detected near the license plate.
VI. R ESULT 100 images with visible plates were prepared for testing, each one contains a license plate. 96 plates were detected correctly while 8 non-plate-areas were classified as plates. It is emphasized that we may detected more than one plates near the real license plate, as shown in Fig.12. In this project, all the matches constitute a rectangle area which include all of themselves, we simply select this region as plate area.
Fig. 15. License plate location results of some vehicle images in cloudy conditions, where the license plates are marked with red boxes.
in this project usually about 20 minutes for 1011 positive samples and 6074 negative samples, and the cascade classifier constructed can also achieve a good accuracy. Compared to the other realizations of Adaboost, the new kinds of features, such as texture features, are introduced to increase the accuracy but, at the same time, the time cost increase rapidly. In this project, the pictures used are of a high resolution of 1040 × 1392, it means a much greater number of regions to calculate than that of ordinary resolutions. For this reason, this project is a little slow to fulfill the requirements of real time. Though more works need to be done, we have a good starting.
Fig. 13. License plate location results of some vehicle images in day conditions, where the license plates are marked with red boxes.
ACKNOWLEDGMENT The work is supported in part by Xi’an Desheng Scientific Tech. Inc., Xi’an, P. R. China. The web of the company can be referred as: http://www.xaxdkj.com,for the further information about the instrument please contact Prof. Xiangdong ZHANG, Email:
[email protected]. This work is also supported by the Open Projects Program of National Laboratory of Pattern Recognition. R EFERENCES [1] Viola P., JonesM. Rapid object detection using a boosted cascade of simple features. Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on , Volume: 1, 8-14 Dec. 2001 Pages:I-511 - I-518 vol.1 [2] Louka Dlagnekov License Plate Detection Using AdaBoost Department of Computer Science Engineering UC San Diego La Jolla, CA 920930114 [3] Huaifeng Zhang, Wenjing Jia, Xiangjian He and Qiang Wu LearningBased License Plate Detection Using Global and Local Features Computer Vision Research Group, University of Technology, Sydney E-mail: hfzhang, wejia, sean,
[email protected] [4] Nicholas Wells An Investigation into Texture Features for Image Retrieval Bachelor of Science in Computer Science with Honours The University of Bath May 2007
Fig. 14. License plate location results of some vehicle images in night conditions, where the license plates are marked with red boxes.
Fig.13-15 shows some of the detection results, where the license plates are marked by red rectangles. From the examples, we may see that our algorithm can detect the license plates with accurate sizes, positions and more complex backgrounds. the training procedure is faster in this project,
2461
