Vehicle Stop Detection Algorithm Based on Motion Analysis in Access Control System Application Ali Majidi#1, Hossein Pourghassem#2, Mansour Nejati#3 1,3 #
Department of Electrical and Computer Engineering, Isfahan University of Technology, Isfahan, 84156-83111, Iran. 2# Department of Electrical Engineering, Najafabad Branch, Islamic Azad University, Isfahan, Iran. 2
[email protected]
Abstract— In This paper, a vehicle stop detection algorithm based on motion analysis in access control system application is proposed. In this algorithm, motion of vehicle in consecutive frames is analyzed and used to determination of frame in which vehicle has been stopped. Motion analysis is carried out based on thresholded difference image calculated for each two sequential frames of a video stream and a verification mask which shows the variations of important edges in these two frames. Then, a decision value is extracted from the refined difference image that determines the vehicle stop frame. In this algorithm, an adaptive thresholding approach based on decision value of previous frames is proposed that compensates the various illumination conditions in day and night. The proposed vehicle stop detection algorithm was evaluated on several videos captured in day and night. The obtained results show efficiency of the proposed algorithm in real and operational conditions. Keywords— Access Control, motion analysis, stop detection, adaptive thresholding.
I. INTRODUCTION In the current information technology era, the use of automations and intelligent systems is becoming more and more widespread. The Intelligent Transport System (ITS) technology has gotten so much attention that many systems are being developed and applied all over the world [1]. These systems are having a wide impact in people’s life as their scope is to improve transportation safety and mobility and to enhance productivity through the use of advanced technologies [2]. The Intelligent Transport Systems have been used in many applications such as parking lots [3], automatic toll collection [4], traffic law enforcement [5, 6] and access control of restricted areas [7]. In intelligent access control systems, whenever a vehicle approaches the gate of the restricted area, a few frames are captured at entry point using an analogue or digital camera. Then, the access control application requests the license plate recognition (LPR) module to analyse these frames and read the license plate number of the vehicle which in database module checked against different lists of permissions. LPR algorithms are generally composed of the following six processing steps: 1) plate region extraction; 2) derotating of plate image; 3) plate image binarization; 4) character segmentation; 5) character recognition using a classifier such as neural network; and 6) searching in data
Vehicle Detection
Convert gray plate to binary
Plate region extraction
Character segmentation
Derotate plate image
OCR
Search in data base Fig. 1 Block diagram of access control.
base for recognized licence plate number. Performance of each of these steps affects the total performance of the system, thus, many efforts have been made to increase the performance of individual step, especially the plate region extraction and character recognition steps [8-10]. As mentioned, the first step in intelligent access control is the detection of vehicle's arrival at entry point. For this we have two solutions: use of cross detection sensor, or use of image processing techniques. Sensor-based vehicle stop detection algorithms have some problems in implementation conditions. (a) Life time of sensor and duty cycle of service them, (b) direct dependency of sensors location to the camera field of view or camera location, (c) cost of this type of sensors (d) transmission of many false positive signals for non-vehicle objects. Therefore, vehicle stop detection algorithms based on computer vision techniques are preferred and can obtain better performance than sensor-based algorithms. In this paper, a sensor less vehicle stop detection algorithm based on motion analysis is proposed. In this algorithm, by calculation of difference image and verification mask for each two sequential frames and suggestion of an adaptive thresholding approach, vehicle stop frame is extracted from video stream. The proposed algorithm obtains reliable results that are non-sensitive to various illumination conditions.
This paper is organized as follows: Section 2 presents details of the proposed algorithm for vehicle stop detection. Experimental results are shown in Section 3, and a conclusion to the work is presented in Section 4. II. STRUCTURE OF PROPOSED ALGORITHM Block diagram of the proposed algorithm is shown in Fig 2. In this algorithm, for each two sequential frames, a set of processes are carried out until a frame selected as vehicle stop frame. This decision is done based on difference and verification images and an adaptive thresholding algorithm. In the following, more details of these processes are presented. A. Difference Image Computation In this paper, we estimate time of the vehicle entrance in to the camera field of view in front of gate. When the vehicle enters to the field of view, pixel values of input image are changed considerably. We can extract these changes in an image that it is called difference image. The difference image is defined as Pd ijn = Pijn −1 − Pijn
(1)
where Pd ij n is the value of difference of pixel (i, j ) and Pijn−1 is the value of pixel in frame n-1 and Pijn is the value of pixel in frame n, at location (i, j ) . To measure and quantify the changes in the difference image, a thresholding method is applied as, 1 if Pd ijn > Td Indexij = n 0 if Pd ij < Td M
D=
(2)
N
Indexij
(3)
i =1 j =1
where Td is a constant threshold that is determined as heuristically and D is an index for measuring of difference between two sequential frames. When the vehicle enters to the field of view, the rising edge of change of difference is occurred. Then, when the car velocity decreases, the change of D with a less slope falling is created. Therefore, in this method we search for such falling edges. If the falling edge occurred and crossing a specified threshold in falling pass, then we decide the vehicle is stopped in front of the gate. During this procedure, minimum changes in the valley are selected. This frame is saved for sending to the LPR part. However, should be noted that because we cannot find minimum in online stream (the system cannot predict the future), after crossing the threshold we capture one frame in every ten consecutive frames. Entrance and exit of vehicle in the field of view of camera creates two peaks and two valleys in graph of difference index D. Two peaks have not equal size; the first peak is smaller because the vehicle is farther to the camera at entrance and therefore takes a lesser part of field of view.
Frame i
Frame i-1
Compute verification mask
Compute difference image
Apply mask on difference image
Compute decision value (D)
Select candidate Plate image
Yes
Stop frame detection
No
Next frame
Fig. 2 block diagram of vehicle stop detection algorithm.
Difference index, D, is very sensitive to illumination of frames, also having a fixed threshold is not proper because in various operating area, there are various light conditions. So for solving these problems we use adaptive threshold and combine difference by edge-based criteria. Sensitivity of D due to the intensity of pixels can be decreased if we use another criteria based on an illumination independent factor. The calculation time of second criteria is very important because we have 50 milliseconds between each two sequential frames for all processes.
B. Compute Verification Mask To compute verification mask, we compare the edge images of two consecutive frames. The dependency of edge strength to illumination conditions is mush less than pixel intensity of image. However, for taking the advantages of difference index, we employed a combination of these two criteria. The edge criterion is used as a mask for difference image Pd ijn . The edge image can be obtained by a first-order difference, ∂Pi, j (4) Px (i, j ) = = Pi, j − Pi, j +1 ∂x ∂Pi, j Py (i, j ) = = Pi, j − Pi +1, j (5) ∂y Gi, j = Px (i, j ) + Py (i, j )
(6)
where Gi, j is the gradient magnitude at location (i, j ) . Then, the verification mask image M is calculated from gradient magnitude of each two consecutive frames: M i , j = Gin, −j 1 − Gin, j
(7)
If the value of each element M i, j is larger than 1 the value of Pd ijn is set to valid. In this procedure, the changes of Pd ijn is very suitable for selecting the peaks and valleys. C. Vehicle Stop Frame Extraction In the vehicle entrance and exit times, equation (1) has more values in the vehicle entrance and on vehicle stop frame has minimum value. In this algorithm, vehicle stop frame is selected if D is more than a adaptive threshold Ta and falling edge has been occurred. We extracted this frame as vehicle stop frame.
The difference index graph of 12 vehicles is shown in Fig 4. This measure for one vehicle is shown in Fig 5. As can be seen in Fig. 6, if we use the combination of two criteria for vehicle stop detection the variations of difference index graph would be smoother with respect to the graph showed in Fig 4. For computation of the adaptive threshold we need obtained local MIN and MAX in difference index graph. Local MAXs and MINs for 1000 frames are shown in Fig 7. The obtained adaptive threshold and difference index are shown in Fig 8. This algorithm works well for both day and night modes. Graphical user interface of an access control application in which the proposed algorithm is employed for vehicle stop detection, is shown in Fig 9.
D. Compute Adaptive Threshold In our algorithm, the selection of Ta has major effect on vehicle stop frame detection. The selection of constant threshold is not suitable for various illumination conditions. So, we propose an adaptive thresholding algorithm based on intensity conditions of the last 1000 frames. In the first 1000 frames, a fixed value for Ta is set and after that adaptive thresholding algorithm selects new threshold. In this algorithm, all local minimums (MIN) and maximums (MAX) are calculated from validated D graph. However, the number of local MAX and MIN are not real due to small ripple. If distance between to consecutive MIN and MAX is less than predefine threshold, then MIN and MAX are removed. Block diagram of computing adaptive thresholding algorithm is shown in Fig. 3. Now we have some real local minimum and maximum outcome. Adaptive threshold is defined based on averaging of minimums and maximums as: AVGMAX − AVGMIN Ta = + AVGMIN 3
Fig. 3 Block diagram adaptive thresholding algorithm.
(8) 6000
where AVGMAX and AVGMIN are the average of maximum and minimum values, respectively. This threshold is used for the next coming 20 frames.
5000
A. Vehicle Stop Frame Detection
D ifference
4000
III. EXPERIMENTAL RESULTS In our experiments, the proposed system was implemented in VC++9 language and OpenCV library. We tested our algorithm on a standard Intel Pentium 4, 3-GHz and 1 MB Ram personal computer. The frame size was 576×768 pixels. Our dataset includes 320 vehicles which captured by a color analogue camera in diverse illumination conditions through day and night. The distance between vehicle and camera is in the range of 10 to 15 m.
3000
2000
1000
0
0
500
1000
1500
2000
2500
3000
Frames
3500
4000
Fig. 4 Difference index graph for 12 vehicles.
4500
5000
IV. CONCLUSION In this paper, we presented a novel methodology to vehicle stop detection applied in intelligent access control systems, based on image processing techniques instead of traditional sensor-based solutions. In this proposed algorithm, motion of vehicle in consecutive frames is analysed and used to determination of frame in which vehicle has been stopped. We tested the system on several video streams captured at gate of the restricted area. The results show that our algorithm is efficient and fast and the system performs well even in worst light condition.
4500 4000 3500
Difference
3000 2500 2000 1500 1000 500
3500
0 0
200
400
600
800
Frames
1000
Difference Adaptive threshold
1200 3000
Fig. 5 Difference index for one vehicle.
2500
Difference
3500
3000
2000
1500
Different for edge
2500 1000
2000 500
1500 0 0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Frames
Fig. 8 Decision measure based on adaptive thresholding algorithm.
1000
500
0 0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Frames
Fig. 6 Difference index after using verification mask. 3500 Difference Local minimum Local maximum
3000
Difference
2500 2000 1500 1000 500 0 0
(a) (b) Fig. 9 GUI of the proposed algorithm in, (a) night and (b) day. 100
200
300
400
500 Frames
600
700
800
900
1000
Fig. 7 Local minima and maxima for difference index graph.
Table 1 shows vehicle stop frame detection performance. In this Table, the results of the proposed algorithm are evaluated on four datasets in various illumination conditions. The DB1 and DB3 datasets have been captured in day mode and two DB2 and DB4 datasets have been captured in night mode. False accept and reject rates of the proposed algorithm obtained for these datasets are presented in Table 1. In access control application, false reject rate is important so that no vehicle miss and false accept rate is less respectable because it causes some frames have not vehicle.
TABLE I: PERFORMANCE OF VEHICLE STOP FRAME DETECTION ALGORITHM
Test Datasets Number of frames Number of vehicle Number of vehicle frame detection Number of frame without vehicle Number of missed vehicle False accept rate (%) False reject rate (%)
DB 1 6838 70
DB 2 7392 60
DB 3 5120 30
DB 4 10360 160
143
151
40
263
10
8
4
22
0
0
0
1
6.99 0
3.31 0
10 0
8.36 0.625
REFERENCES [1]
[2]
[3] [4] [5]
H. K. Sulehria, Y. Zhang, D. Irfan, A. K. Sulehria, “Vehicle Number Plate Recognition Using Mathematical Morphology and Neural Networks,” WSEAS Transaction on Computers, Vol.7, No. 6, pp. 781-790, June 2008. E. Anagnostopoulos, I. E. Anagnostopoulos, V. Loumos and E. Kayafas, “A License Plate-Recognition Algorithm for Intelligent Transportation System Applications,” IEEE Transaction On Intelligent Transportation System, Vol. 7, No. 3, pp. 377-392, Sept. 2006. T. Sirithinaphong and K. Chamnongthai, “The recognition of car license plate for automatic parking system,” in Proc. 5th Int. Symp. Signal Processing and its Applications, pp. 455–457, 1998. R. A. Lotufo, A. D. Morgan, and A. S. Johnson, “Automatic number plate recognition,” in Inst. Elect. Eng. Colloquium on Image Analysis for Transport applications, pp. 611-616, 1990. P. Davies, N. Emmott, and N. Ayland, “License plate recognition
[6] [7] [8] [9] [10]
technology for toll violation enforcement,” in Inst. Elect. Eng. Colloquium Image Analysis for Transport Applications, pp. 711715, 1990. F. Martín, M. García, L. Alba, “New methods for automatic reading of VLP's (Vehicle License Plates),” in Proceedings of IASTED International Conference on SPPRA, 2002. S. Draghici, “A neural network based artificial vision system for license plate recognition,” Int. J. Neural systems, Vol. 8, pp. 113126, 1997. S. L. Chang, L. S. Chen, Y. C. Chung, and S. W Chen, “Automatic License Plate Recognition,” IEEE Transaction on Intelligent Transportation System, Vol. 5, No. 1, pp. 42–53, March 2004. X. Fan, G. Fan, and D. Liang, “Joint segmentation and recognition of license plate characters,” in Proceedings of International Conference on Image Processing, Vol. 4, pp. 353-356, 2007. S. Z. Wang and H. J. Lee, “A Cascade Framework for a Real-Time Statistical Plate Recognition System,” IEEE Transaction on Information Forensics and Security, Vol. 2, No. 2, pp. 267-282, June 2007.
