Software based Automatic Vehicle Identification aided by OCR for License plate Recognition using ANN and Multilayer Perception Akshit Mahajan B.E. (CSE), School of Engineering and Technology, Chitkara University, Baddi INDIA Email:
[email protected] ABSTRACT – Vehicle license plate recognition is an image-processing technology used to identify vehicle by their license plate. This technology can be used in various security and traffic applications, such as finding stolen cars, traffic flow statistics etc. The study is carried out using vehicle license plate detection (VLPD) system which works on an optimal algorithm for license plate localization and recognition. The proposed algorithm consists of four major parts: Image Capturing, Extraction of plate region from the captured image, segmentation of characters and recognition of plate characters. For extraction and localization, a simple texture-based approach based on edge information is used. Horizontal and Vertical projections are used to separate plate numbers in VPL Segmentation. To recognize the characters on the plates, Multi layer Perception (MLP) technique is implemented. The system as tested, works better even in case of low-quality images and images with illumination effects and noise. The system is integrated with Database Query Engine and GUI on the front end, for effective user handling and data management with overall accuracy. Keywords— Optical Character Recognition (OCR), VLP Detection, LPR, plate region extraction, segmentation, ANN, Multilayer Perception, Database Query Engine. I. INTRODUCTION Automatic Vehicle Identification Systems are used for the purpose of effective controlling. It has many applications in traffic monitoring systems, such as Highway electronic toll collection, Red light violation enforcement, Border and custom checkpoints, Vehicle surveillance theft control and Intelligent Parking strategies. A number of commercial softwares have been developed in this area. However, they cannot be readily used when vehicle image is provided in different styles and formats. These tasks become more sophisticated when dealing with plate images taken in various inclined angles or plate images with noise. Because this problem is usually encountered in real-time systems, it requires not only accuracy but also fast processing. This paper focuses to overcome these constraints and enhance the recognition process of
the plate by building an Artificially Intelligent OCR (Optical Character Recognition) control engine using an optimized Vehicle License Plate Detection and Recognition technique. Most VLP recognition applications reduce the complexity by establishing some constrains on the position and distance from the camera to vehicles, and the inclined angle which can reduce the detection rate of the VPL Detection system. We can gain more accuracy by using some specific features of local VLPs at the time of image capture such as the number of characters, the number of rows in a plate, or colors of plate background, or the ratio of width to height of a plate. Another challenge is to make the system capable of handling huge data with security and easiness. II. RELATED WORK The problem of automatic VLP recognition has been studied since 1990s. The first approach was based on characteristics of boundary lines. The input image was first processed to enrich boundary lines’ information by some algorithms such as the gradient filter, and resulted in an edging image. This image was binarized and then processed by certain algorithms, such as Hough transform, to detect lines. Eventually, couples of 2-parallel lines were considered as a plate-candidate [3][4]. Another approach was morphology-based [1]. This approach focuses on some properties of plate images such as their brightness, symmetry, angles, etc. Due to these properties, this method can detect the similar properties in a certain image and locate the position of license plate regions. The third approach was texture-based [2]. In this approach, a VLP was considered as an object with different textures and frames. The texture window frames of different sizes were used to detect plate-candidates. Each candidate was passed to a classifier to confirm whether it is a plate or not. This approach was commonly used in finding text in images tasks. In addition, there have been a number of other methods relating to this problem focusing on detecting VLP in video data (objects appear in a chain of sequent images)[5][6].
III. PROPOSED SYSTEM The main objective of this paper is to compose a system that can extract the license plate number from the complex-scene images with no assumption of image quality, distance between camera and the plate, angle in which the plate has been captured relative to the camera and storing the valid data in the database.
A. Pre-processing The input image is initially processed to improve its quality and prepare it to next stages of the system. First, the system will convert RGB images to gray-level images using the NTSC standard method:
Our system consists of four modules: i. Capturing the vehicle’s image ii. Extracting the License plate iii. Extracting characters from license plate image. iv. Recognizing the license plate characters.
In the second step, a median filter (5x5) is applied to the gray-level image in order to remove the noise, while preserving the sharpness of the image. The median filter is a non-linear filter, which replaces each pixel with a value obtained by computing the median of values of pixels in this case, 5x5 neighborhood of original pixel.
An overview of the proposed number-plate detection and recognition system can be seen in Fig 1. After the vehicle’s image is captured by the camera, it will be passed to a pre-processing unit which prepares the image for further processing by the system. Its main operations are to eliminate the noise caused by the image-acquisition subsystem and enhance the image features used by the other two subsystems. The image will be scanned by the plate extraction module to locate the vehicle’s license plate. The next phase is responsible for segmenting the individual characters of the plate. Finally each character will be passed to the Optical Character Recognition (OCR) module to be identified and the final results care ASCII characters and the numbers of the plate. These ASCII results are stored in the database (DQE) with other vehicle’s information (if available). INPUT IMAGE VLPD System Pre - Processing Eliminate noise enhance the features
and
Plate Extraction
Sub-image containing License plate only Character Segmentation N images containing segmented characters
Character Recognition License Plate Number in ASCII format
Database Query Engine
Fig. 1
GUI Front End
VLPD model with DQE.
Gray = 0.2299*Red+0.587*Green+0.114*Blue
B. Plate Extraction The plate extraction process contains five different phases as shown in Fig 2; where each phase performs a segmentation process on the gray image to eliminate the redundant pixels that don’t belong to a plate region. For example, the horizontal localization phase is responsible for finding horizontal segments that may contain the license plate. The license plate region is characterized by a row of transitions from black to white and vice versa, such transitions are called edges. The total change in intensity from plate’s characters to its background is called the strength of the edge. The strongest edge value that can occur can be found in the case of a transition from a black pixel to a white pixel or from a white pixel to a black pixel. In this paper, we are going to use Sobel operators to find the edge image. The Sobel operator performs a 2-D spatial gradient measurement on an image. Typically it is used to find the approximate absolute gradient magnitude at each point in an input grayscale image. The Sobel edge detector uses a pair of 3x3 convolution masks, one estimating the gradient in the x-direction (columns) and the other estimating the gradient in the y-direction (rows). The actual Sobel masks are shown below:
GRAY IMAGE Edge Detection Using Sobel Operators
Horizontal Localization
Cleaning and Verification
Vertical Localization
License Plate Fig. 2
Plate extraction Process
After creating the edge image, the system will search for regions like high edge values which are most likely to contain a license plate. To do so, the system will construct a horizontal projection profile for the edge image. A projection profile of an image is a compact representation of the spatial pixel content distribution; a horizontal projection profile is defined as the vector of the sum of the pixel intensities over each row. The next step in the plate extraction process is to find the vertical position of the license plate. For this part, the methods generally use a statistical study of histograms of the picture of the plate [7]. We propose a simple method to locate the vertical coordinates of the plate using the edge image. The algorithm starts by sliding a window through each horizontal segment and sums the value of edges inside the window’s area to get the average gradient magnitude per pixel inside the window at that specified location. For each horizontal segment, the algorithm will store the resulted value of each window step in a new vector for later processing; we shall call this vector an HDV which can be calculated as:
where, i = 0,1….(ImW – M+1), G(x,y) is the edge image, W is the sliding window, Aw is the area of the image and M is the window, ImW is the width of the image and M is the width of the sliding window. The segmentation of plates that contain two rows of characters will be done through a horizontal projection of binary image for the extracted
segment and searching for a valley that goes beyond a specific threshold. If found, the segment will be considered as a two row plate and will be cut into two spate segments. The isolation of the license plate from any superfluous background is performed using a projection profile that reflects the number of white pixels in each row and in each column. In most cases, projecting the amount of white pixels both vertically and horizontally reveals the actual position of the plate. The last step in the localization process is the verification phase. The input to this phase is all the extracted regions from the previous stages which may contain license plate. A rule-based classification approach is followed, where an extracted region will be deleted if one of the following conditions is satisfied: 1. Aspect ratio (width/height) of the region is smaller/greater than a threshold. 2. The area of the region is less than a threshold value. 3. Average gradient magnitude per pixel inside the region is smaller/greater than a threshold value. 4. Density of the region is less/greater than a threshold (density = objects area/total area). C. Character Segmentation To ease the process of identifying the characters, it is preferable to divide the extracted plate into different images, each containing one isolated character. These are some widely used methods for character isolation which are used in almost all available LPR systems. Those methods are: static bounds [8], vertical projection [9] and Connected-Components. In this paper the following steps are used to segment the characters of the license plate: 1. Stretch the contrast of the image to extend over the entire of levels available (0-255). 2. Threshold the plate image Otsu method [10]. 3. Search for connected components in the image, each connected components will be assigned a special label in order to distinguish between different connected components in the image. 4. Resize each character from the previous step to the standard height and width (20x10) in order to be used in the next step. D. Character Recognition ANN is used for character recognition. We used a Multi-Layer Perception Neural Network (MLP NN) trained with the back-propagation algorithm. During learning phase, characters of the constituted database are successively presented at the input
layer of MLP network and their corresponding outputs are compared to the desired outputs. Weights are iteratively modified in order to obtain, at the network outputs, responses which are as close as possible to the desired outputs. The input layer consists of 135 input neurons and there is one hidden layer which has 40 neurons with log-sigmoid transfer functions. The output stage has 36 neurons, each outputting the probability that the given input is the corresponding character. E. Database Query Engine The resultant License plate number after getting fetched from the VLPD System in ASCII format will be then directly stored in the Database Query Engine. This will be considered as one entry. The database will comprise of following schema: S. no
Vehicle Type
Vehicle Model
License plate no.
Date and Time
The database will be capable to handle various ambiguities and errors like multiple/vague entries for a single vehicle etc. The purpose of integrating the processed results in database to keep the track of the vehicle’s information. This can be used as an application for vehicle monitoring in traffic departments or parking lot management system. In Database Query Engine, the authenticated user can log-in to the system and can query or search for any data required based on the date/time or any attribute listed in the schema. The database proposed to be used is MySql. The aim of this paper is to combine whole system under a Graphical User Interface for better handling and access, with data security. The GUI can be built using QT creator (Version 4.2.3) which supports MySQL database server (Version 5.0.27). IV. CONCLUSIONS This paper consists of a conceptual model for Vehicle License Plate Detection and Recognition using advance methods like ANN, Multilayer Perception etc. to make a system for automatic recognition of license plates and providing a Graphical User Interface with database access. Given an input image, it should be able to extract the plate, isolate the characters contained in the plate and finally identify the characters in the license plate. The proposed system will search the image for high density edge regions which may contain a license plate. After that a cleaning and verification process will be performed on the extracted regions to filter out those regions that do not contain license plate. Then the plate will be passed to the segmentation phase where it will be divided into a number if sub-images equal to the
number if characters contained in the plate. Finally, each of the sub images will be passed to a Multi-Layer Perception Neural Network for exact character identification. This concludes the Identification Mechanism. Now the Identified group of characters is stores in the Database Query Engine (DQE) with other vehicle information for quick and secure access.
V. REFERENCES [1] Jun-Wei Hsieh, Shih-Hao Yu, Yung-Sheng Chen, “Morphology- based License Plate Detection from Complex Scenes”, 16th International Conference on Pattern Recognition (ICPR'02) Vol 3, 2002. [2] Kwang In Kim, Keechul Jung and Jin Hyung Kim, “Color Texture- based Object Detection: an Application to License Plate Localization”, Lecture Notes in Computer Science, pp. 293-309, Springer, 2002. [3] Kamat, Varsha, and Ganesan, “An Efficient Implementation of the Hough Transform for Detecting Vehicle License Plates Using DSP’S”, Proceedings of Real-Time Technology and Applications, pp58-59, 1995. [4] Yasuharu Yanamura, Masahiro Goto, Daisuke Nishiyama, “Extraction nd Voted Block Matching”, IEEE IV2003 Intelligent Vehicles Symposium Conference, 2003. [5] A. Rahman, Ahmad Radmanesh, “A Real Time Vehicle’s License Plate Recognition”, Proceedings of the IEEE on Advanced Video and Signal Based Surveillance, AVSS’03, 2003. [6] K. Kanayama, Y. Fujikawa, K.Fujimoto, and M.Horino, “Development of Vehicle-License Number Recognition System Using Real-Time Image Processing and Its Application to Tracal-time measurement”, In Proceedings of IEEE Vehicular Technology Conference, pp798-804, 1991. [7] Gonzalez R. C. and Woods R. E., Digital Image Processing, Addison-Wesley, 1993. [8]http://en.wikipedia.org/wiki/Automatic_number_pl ate_recognition [9] Lu Y., “Machine Printed Character Segmentation,” Pattern Recognition, vol. 28, no.1 pp. 67-80, Elsevier Science Ltd, UK, 1995. [10] Otsu N., “A Threshhold Selection Method for Gray Level Histograms,” IEEE Transactions on System, Man and Cybernetics, vol. 9, no. 1, pp.62-66, January 1979.
