A New Feature Extraction Method for License Plate Recognition Salah Al-Shami Dept. of Math and CS Beirut Arab University Debbeye, Lebanon
[email protected]
Ali El-Zaart Rached Zantout Dept. of Math and CS Elect. and Comp. Eng. Beirut Arab University Rafik Hariri University Debbeye, Lebanon Meshref, Lebanon
[email protected] [email protected]
Abstract—In this paper, character recognition found in license plates is described. The developed procedure is based on real license plates. The numbers are limited to ten classes (0-9). The character recognition problem is a very important problem and many people worked on implementing different methods. One of the successful set of methods to recognize characters from a closed set are the methods which uses lines, but this method suffers from the fact that the number of lines and the thresholds for each feature in each line are selected manually for each set of characters. Our goal is being able to develop the optimal recognition tree in the classification process automatically. Several phases are needed in order to recognize a character. In the feature extraction phase, we introduce two new features; the first feature is related to the quantization process on a specific feature, and the second feature is the combination of several features to form new features. The developed algorithm was applied to different datasets in license plates from KSA; and the recognition rate was above 95%. In this paper, we are concerned on the English Numbers in the KSA license plates. Keywords—license plate recognition; feature extraction; quantization; feature selection; recognition tree
1. INTRODUCTION License plate recognition system is an automatic process to recognize the characters in the license plate images. Character recognition has become one of the most successful applications of Information Technology in the field of pattern recognition and artificial intelligence. The need for image analysis became an essential technique in data mining and image processing. It is not efficient to compare an image with all views in a memory. Printed numbers may be recognized, but the performance is directly dependent upon the quality of the input. The application of license plates recognition is used for law enforcement in traffic violation systems by using special types of sensors and cameras in the streets. Searching manually for closed-set characters in a video scene is time consuming, and human could
Ahmed Zekri* Dept. of Math and CS Beirut Arab University Debbeye, Lebanon
[email protected]
Khaled Almustafa College of Engineering Prince Sultan University Riyadh, KSA
[email protected]
commit mistakes while recognizing the image characters. In this paper, we propose an improvement to the algorithm to recognize characters in the KSA license plates. The algorithm is based on an earlier version [1] on KSA license plates. We used new combined features to ease the automatic recognition of the target license plates (KSA) with high accuracy. The algorithm searches for the line containing a feature or a set of combined features after applying quantization, which uniquely identifies a certain character in all the training datasets. Throughout this paper, we discuss in section 2 a brief literature review. In section 3 the methodology of extracting features taken from previous work is discussed. In section 4 we propose a new improvement to feature extraction phase. Then in section 5 we propose a method to apply feature selection phase. In section 6 we propose a method that will classify the optimal tree automatically. In section 7 we include the results of our study, and finally a conclusion and future work suggestions in section 8. 2. LITERATURE REVIEW In order to locate a license plate in an image, researchers usually used one or a combination of several techniques. Some researchers used the line detection or edge detection [5], through the Hough Transform, which is superior in finding lines out of edges; however it is sensitive to noise. Another technique researchers used in recognition is through histogram processing [6] by studying the variation of grey value in the image pixels. Other histogram processing methods use information about valleys and peaks to localize a license plate. Histogram processing is sensitive to orientation of the license plate. Other researchers used the blob analysis [7] that primarily works with regions instead of individual pixels. Blob analysis requires a segmentation phase in order to extract the regions.
*A. Zekri author is with Beirut Arab University, Debbeye, Lebanon, on leave from Alexandria University, Alexandria, Egypt (email:
[email protected])
ISBN: 978-1-4799-4130-8 ©2015 IEEE
64
In [8], a technique was presented for Egyptian vehicle’s license plate recognition system using template matching. The failed identification came from mostly motion blurred or overlapped by other vehicle’s body or dirty in the plates. In [1], the researchers used the line processing method. They extracted the features using three algorithms [2], [3], [4], which rely on processing information from 6 lines strategically drawn vertically and horizontally through a character, and then they applied manual grouping designs using decision tree classification. The reason for introducing all three algorithms is that each algorithm has its limitations in recognizing some of the characters. In [2], it was shown that the “number of Peaks” algorithm can successfully recognize English numbers; however, it cannot distinguish between Arabic numbers 0, 1 and 6. Also the algorithm cannot distinguish between English letters T and L for some factors. In [3], it was shown that the “percentage algorithm” can correctly recognize all English and Arabic numbers and letters if different factors are used for a given group of characters. In [4], it was shown that the “position of peaks” algorithm, it cannot distinguish between Arabic letters اand م. Moreover, Arabic numbers 2 and 3 cannot be distinguished unless different factors are used.
we are able to extract features from h—horizontal lines and extract features from v—vertical lines. In section 3.1.1, we will discuss the methods used to extract features from horizontal lines. The methods used to extract features from horizontal lines are the same applied on vertical lines. For this reason we only present the method of feature extraction for horizontal lines. 3.1.1 Extracting each Horizontal Line Features Our objective is to extract all 3 types of features for each horizontal line: a- Feature 1: it counts the number of transitions in each horizontal line [2]. In our case, a transition is by a passing line from black to white bit, or by starting the line by a white bit starting from the left region; In Fig. 1, we have one transition in each of the Lines 1 and 2, while two transitions in each of the Lines 3, and 4.
This paper presents an improvement of research work [1], thus we concentrate on line processing through the image. 3. METHODOLOGY The algorithm goes through 5 fundamental phases: Starting with data preparation and image preprocessing techniques for the region containing the characters needed for recognition. In the second phase, we apply the segmentation process for the binary images inside bounded boxes. In the third phase, we apply the feature extraction belonging to all possible horizontal and vertical lines in a standard image size, where 3 different features are extracted from each horizontal and vertical line. In the fourth phase, the training and feature selection processes are applied to different number of training datasets. Finally, in the fifth phase, the recognition tree is developed.
Fig. 1. The Transitions are indicated by an arrow
b-Feature 2: it quantifies the percentage of white pixels in each horizontal line [3]; In Fig. 2, the percentage of white pixels in Line 1, 2, 3, and 4 are 0.7611, 0.2116, 0.8878 and 0.4117 respectively.
The discussion in this section will be dedicated about the feature extraction phase. 3.1 Feature Extraction Phase Each character is in an individual .jpg (or .bmp) file labeled with the character identification. Since the segmented images are of different sizes, thus we have to resize the images into a standard size “h x v”, thus Fig. 2. Percentage of White Pixels crossing the Line
ISBN: 978-1-4799-4130-8 ©2015 IEEE
65
c- Feature 3: it is a binary feature that divides the image into 3 identical regions vertically (Left, Middle, and Right), and checks for the occurrence of transitions in each region [4]. • Left_Region = 1, if a transition occurs in the Left region, otherwise Left_Region = 0 • Middle_Region = 1, if a transition occurs in the Middle region, otherwise Middle_Region = 0 • Right_Region = 1, if a transition occurs in the right region, otherwise Right_Region = 0 In Fig. 3, it is assigned by an arrow all the transitions crossing the three lines crossing the image. The results of the 4 lines are as follows: • • • •
Line 1 has Left = 1, Middle = 0, Right = 0 Line 2 has Left = 1, Middle = 0, Right = 0 Line 3 has Left = 1, Middle = 1, Right = 0 Line 4 has Left = 1, Middle = 0, Right = 1
Our algorithm applies the quantization process, where quantization is the procedure of constraining something from a continuous set of values (such as the real numbers) to a relatively small discrete set (such as integers). The value of Q could range between 2 and 100. If we apply quantization Q on the each horizontal line, then: (1)
Percentage_of_White (Q) = ⌈ (Percentage_of_White * Q)
For example, if we apply Equation (1) with Q = 20 to the lines 1, 2, 3, and 4. Then the results of Percentage_of_White feature for the four lines would be 16, 5, 18, and 9 respectively. In section 4.1, we start discussing the way to represent the features in the database. In section 4.2, we will model the extracted features in a cube. 4.1 Feature Extraction Representation in the Database We have five types of features that are stored in the database as follows: Feature 1: represents the Transitions feature, and they are stored in the database as an integer number greater than zero. Feature 2: represents the Percentage_Of_White feature, and they are stored in the database as a floating point numbers greater than zero, and as an integer number if the quantization process is applied.
Fig 3. The occurrence of transitions is indicated by an arrow
4. IMPROVEMENT OF FEATURE EXTRACTION In our example, Lines 1 and 2 contain one transition, but we can differentiate between them by the percentage of white pixels. To solve this problem, we propose to apply the combination of several features, in order to produce higher diversity between the features, and detect if we can classify better results than the previous work [1]. The extracted feature of Percentage_of_White gives us a set of values that are in floating points, and the same feature could quantify different values in the same line number with other images related to the same class (example: 0.7611 is different from 0.7612). To solve this problem, I propose to use the quantization process.
ISBN: 978-1-4799-4130-8 ©2015 IEEE
Feature 3: represents the Position_of_Transition feature in a new representation by storing them in the database as a code of three binary bits in one attribute, instead of three independent attributes. Feature 4: it is represented by combining Feature 2 and Feature 1 in a decimal format as follows in Equation (2). Feature “4” = Feature “2”
. Feature “1”
(2)
For example, the feature value is 16.3, then this means that the value of Feature 2 after quantization is 16, and the line holds three transitions. Feature 5: it is represented by combining all the 3 features together in a decimal format as presented in Equation (3). Feature “5” = Feature “2”
. Feature “1” Feature “3”
(3)
66
For example, the feature value is 8.3111, then this means that the Feature 2 after quantization is 8, and the line holds three transitions which occur in the three regions, since Feature “3” is coded as “111”. 4.2 Feature Extraction Modeling The general model for feature extraction is proposed to be two cubes, one for the horizontal lines extracted features and the second for the vertical lines extracted features. The cube constructed from horizontal line feature is composed with “h” horizontal lines, while the cube constructed from vertical line feature is composed with “v” vertical lines. The horizontal cube is composed by three dimensions: Dimension 1: The set of 5 features (f1, f2, f3, f4, f5), defined in section 4.1 Dimension 2: The set of horizontal lines (H1, H2, ……., Hh), where “h” is the height of the image Dimension 3: The set of 10 classes (C1, C2, …, C10), since we are applying our method to license plates numbers (0-9) We will start constructing the first and second dimensions for each image belonging to a single class (see Fig. 4).
Fig. 5. A cube representing a complete dataset for horizontal lines
Similarly, we can construct the cube from the vertical extracted features, and then the total number of pillars in the vertical cube is 5*v. 5. IMPROVEMENT OF FEATURE SELECTION We apply the feature selection method to all the pillars in the cube, and thus we obtain a cube that contains only the selected features values that differentiate all the classes from each other. A dataset is formed by 10 classes (1, 2, 3, 4, 5, 6, 7, 8, 9, and 10). We consider one pillar from the cube (see Fig. 6). In this figure the left pillar represent a specific feature for a specific line for all 10 classes. The feature values 18 and 16 are not repeated in the pillar, so these two features can differentiate classes C1 and C5 from other classes. Our proposed method of feature selection is to search for dissimilarity of extracted features. In the resulting pillar to the right (see Fig. 6), we keep the dissimilarity feature values and change the other feature values to any negative number that will be ignored in classification (-1 for an example).
Fig. 4. A plane representing one class belonging to horizontal lines
For a set of 10 classes, we construct the third dimension (see Fig. 5). The total number of pillars in the horizontal cube is 5*h.
ISBN: 978-1-4799-4130-8 ©2015 IEEE
67
Fig. 7. Recognition tree for the English numbers in the KSA license plates with Q=7, h=120, and v=120
Line 548 belongs to extracting Feature 5 from the Horizontal Line Number 68. Line 298 belongs to extracting Feature 3 from the Horizontal Line Number 58. Line 595 belongs to extracting Feature 5 from the Horizontal Line Number 115. Line 1182 belongs to extracting Feature 5 from the Vertical Line Number 102. Line 851 belongs to extracting Feature 3 from the Vertical Line Number 11. Fig. 6. Updating the values in a pillar
6. CLASSIFICATION PHASE Our contribution is about retrieving the least number of pillars to classify all the classes, thus we will start classifying with the pillars selecting the highest number of classes in the same pillar. In the classification step, we have to check if we are able to classify all the classes from the selected features. In case we could classify all the classes, then we have to retrieve the following: [Ci, pillar_number, value], where i = 1 till 10 h v Q The result of this process is the classification rules or the recognition tree, as shown in Fig. 7.
ISBN: 978-1-4799-4130-8 ©2015 IEEE
7. RESULTS We will apply the classification rules on characters belonging to the same close-set in order to check the accuracy of recognition. Upon applying an experimental result using MATLAB implementation, we already captured 46 KSA Vehicle License Plates, formed by 4 English Numbers, resulting in 184 extracted objects. In the training dataset, we had 11 training sets (110 images). The accuracy of recognition was 98.37% (181/184), at h=120, v=120, and Q=7. We used 5 different pillars to classify all the 10 classes, as shown in Fig. 7. Our method guarantees automated classification of all different classes. This algorithm can be adapted to work on any close-set pattern. In fact, the algorithm was adapted on other datasets related to Arabic characters found in the Lebanese license plates and English characters in the Croatian license plates, and the accuracy of recognition was above 95%.
68
The tree developed in Fig. 7 is optimal, with a depth of 2, where the maximum number of checks to retrieve a character is 5 checks. Our algorithm guarantees the development of a recognition tree, with a maximum depth of 2 and the maximum number of checks to recognize a character is “i”, where “i” is the number of classes in a certain closed-set. The time complexity of my proposed method is proportionally dependent on the number of checks till the character is recognized in the testing phase. The time earned in the training phase is proportionally dependent on several factors, such as the number of training datasets, the width and height of the image, and the range of quantization taken (2≤Q≤100). 8. CONCLUSION In this paper, an algorithm to recognize the English numbers in the KSA license plate was presented. The algorithm searches for the horizontal or vertical line containing a specific feature or a set of derived features that uniquely identify a certain character in all the training datasets. Moreover, we proposed the process of quantization for the Percentage_of_White feature in order to solve the problem of floating points in this feature, also to constraint something from a continuous set of values to relatively small discrete set.
Exhibition for Sustainable Ubiquitous Technology, Dubai, United Arab Emirates, pp. 186-189., February 19-22, 2011. [5]H. Kwasnicka, B. Wawrzyniak, "License Plate Localization and Recognition in Camera Pictures", Faculty Division of Computer Science, Wroclaw University of Technology, Artificial Intelligence Methods (Gliwice, Poland), November 13-15, 2002. [6]S. Ozbay and E. Ercelebi, "Automatic Vehicle Identification by Plate Recognition”, Transactions on Engineering, Computing and Technology, version 9, November 2005, ISSN 1305-5313. [7]J.R. Parker, P. Federl, “An Approach to License Plate Recognition”, Laboratory for Computer Vision Computer Graphics Laboratory University of Calgary. Vision Interface '97, Kelowna, B.C., May 20-22, 1997. [8]M.A. Massoud, M. Sabee, M. Gergais, R. Bakhit. Automated new license plate recognition in Egypt. Alexandria Engineering Journal 52, 319326, September 2013.
Future research should focus on identifying the number of training sets needed to reach the optimal recognition tree with high recognition accuracy. Another enhancement is to find a method that discover the standard size of images “h” and “v” in order to be treated in the feature extraction phase. Another challenging enhancement could be applying different quantization values in the same recognition tree. REFERENCES [1]K. Almustafa, “On the Automatic Recognition of Saudi License Plate”, International Journal of Applied Information Systems (IJAIS) vol. 5– No.1, January 2013. [2]H. Obeid and R. Zantout, “Line Processing: An Approach to ALPR Character Recognition”, 2007 ACS/IEEE International Conference on Computer Systems and Applications, Amman, Jordan, May 13-16, 2007. [3]K. AlMustafa, R. Zantout, H. Obeid, “Pixel Density: Recognizing Characters in Saudi License Plates”, 10th International Conference on Intelligent Systems Design and Applications, Cairo, Egypt, November 29 – December 1, 2010. [4]K. AlMustafa, R. Zantout, H. Obeid, “Peak Position, Recognizing Characters in Saudi License Plates”, 2011 IEEE GCC Conference and
ISBN: 978-1-4799-4130-8 ©2015 IEEE
69
