FPGA-based Road Traffic Videodetector - CiteSeerX

FPGA-based Road Traffic Videodetector Marek Gorgoń, Piotr Pawlik, Mirosław Jabłoński, Jaromir Przybyło Biocybernetic Laboratory, Department of Automatics AGH University of Science and Technology Al. Mickiewicza 30, 30059 Kraków, Poland {mago,piotrus,mjk,przybylo}@ agh.edu.pl Abstract In the present paper the background generation and motion detection algorithms, which are of key importance for the implementation of videodetection, have been presented. A modification of the background generation algorithm, essential for proper algorithm functioning at medium and high road-traffic conditions, has been proposed. Algorithm adaptation for the implementation in reprogrammable device has been also presented. A modification of the SAD algorithm, used in motion detection, has been introduced. The modification allows for Regions of Interest of irregular shape and structure in the analyzed image. It creates a capability to conduct the calculations in independent and parallel manner for specific, user-defined, active videodetection regions. PixelStream-based implementation has been successfully performed. Real-time verification on reconfigurable platform has been done.

stream, the videodetection algorithms have to include the input data reduction and software optimization. Great new possibilities are opened by the application of specialized real-time image processing devices, based on the FPGA (Field Programmable Gate Array) technology. In the present stage of videodetector development it seems plausible to simulate the functioning of induction loops [1] (see Fig. 1). The reason is to preserve the compatibility with the existing software used for traffic lights control. According to the operating principle of the induction loop the vehicle detection is realized in two ways: the device detects motion and/or presence of the vehicle [6]. The detection of vehicles in motion is based in the image thresholding, realized by calculating a

3 2

1. Introduction With increasing computing power and accessibility of industrial computers and the lowering prices of moni-toring devices interest is growing in videodetection, or automated calculation of road traffic parameters, based on the analysis of videocamera image stream fed into a computer. Videodetectors, or systems located on a road crossing, consisting of several video cameras and a computer analyzing the digital images, have a considerable chance to become an alternative for induction loops commonly applied in road traffic analysis (mounted under surface of a road) [5]. With moderate additional expenses the videostreams from the cameras can be transmitted to a monitoring center, becoming a valuable source of information about possible hazards to the traffic flow. Because of vast amount of information, which has to be processed by the computer analyzing the image

1

Fig. 1. Objects simulating 3 induction loops differential image of two consecutive frames [6]. In order to detect the immobile vehicles (e.g. waiting for the green) the differential image is analyzed, obtained by subtracting from the present frame the reference image, so called background image. Both tasks (the detection of moving and resting vehicles) are realized by application of the SAD (Sum of the Absolute value of Differences) algorithm, described in detail in the following sections.

10th Euromicro Conference on Digital System Design Architectures, Methods and Tools (DSD 2007) 0-7695-2978-X/07 $25.00 © 2007

Figure 2. Background generated on the basis of: a) time histograms - streaks clearly visibible within the traffic lanes; b) weighted average (motion condition switched on) Proper generation of the background image for videodetection is a necessary condition for correct realization of the vehicle detection processes. In city agglomerations, in particular for multilane arteries with high and very high traffic intensity, the correct generation of background images encounters multiple obstacles. Whereas in the low traffic intensity conditions the averaging algorithms are enough, they are not sufficient in the high traffic intensity conditions. In the present paper an elaborated modification has been shown for videodetector background generation method (presented in [7]) and the results are presented for studies oriented at implementation of videodetection algorithm on FPGA platform. In particular the SAD algorithm realization is discussed for the ROI (Region of Interest) image areas arbitrarily defined by the user.

2. The background generation algorithm 2.1. Time-histogram-based method It is difficult to develop motion detection algorithms that would work correctly for an image of a busy urban thoroughfare. This is due to the incessantly changing grey levels within the vehicle traffic detection field. A reference image – the background image is the precondition for the correct detection in differential algorithms. For roads with dense traffic, background pixels in the field in which detection is carried out are highly variable. A properly generated background should contain only the image of the roadway free of vehicles. Generating such a background is easy for a road with little traffic, where statistically, within a specific time bracket, the road-related pixels (road surface, road lines) represent the majority of pixels recorded in the sample. However, when the traffic is denser and the reference values necessary for the definition are much less frequent in the sample, the

image of the generated background may be highly distorted by unwanted values due to the incorrect differentiation between the pixels of the background and of moving objects. Figure 2a shows a background image generated by continuously analysing the statistics of pixel variance over time for every point of the image (a time histogram). Still, this image contains streaks resulting from distortions caused by the recorded pixels of vehicles. The time-histogram-based method required storing the number of occurrences for individual values of each pixel during the whole execution of the algorithm (in other words permanent actualization of time histograms attributed to all individual pixels). In paper [7] the background elements have been determined as local maxima of the time histograms located near the determined weighted average value. The above mentioned method sets high memory requirements (e.g. for a 512x512 pixels resolution 134217728 bytes are required, assuming that each histogram occupies 256 2-byte numbers) and requires the actualization of values in each histogram for the average value calculation.

2.2. A method based on calculating the average pixel value in selected recording periods During the analysis oriented towards transferring the described method to the intended hardware platform it has been noticed, that instead of calculation of the weighted average for each histogram, the accumulation of consecutive pixel values can be used.

10th Euromicro Conference on Digital System Design Architectures, Methods and Tools (DSD 2007) 0-7695-2978-X/07 $25.00 © 2007

T

Mk =

∑X t= 0

T

t

(1)

where : Xt

2.3. The algorithm adaptation for FPGA-based implementation – pixel value for a given time (frame

number) t. The above mentioned calculation method results in determination of identical backgrounds as for the calculation with time histograms. However this method does not allow the above-mentioned modification of the histogram method, comprising the determination of the local maximum in the histogram. On the other hand a considerable memory saving effects are obtained (the memory required is reduced 256 times !), as well as some reduction of the calculation time. During the completed tests it has turned out, that the time effect resulting from modification related to the local maximum calculation (see Fig. 2a) is incomparably small in relation to the additional efforts required for the storage and actualization of histograms. Another, much more essential modification of the weighted average method contained in thessis [7], was the proposition of holding the calculations of background values during the time periods, when vehicles wait for the change of traffic lights. In that period the background determination has been mutually connected with the detection of presence and motion of vehicles. Detailed observation of the movie, collected during the present study, has lead to formulation of a working hypothesis that in the high traffic intensity periods the distribution of grayscale levels for points belonging to the road surface, collected with presence and motion detection, is less favorable for the proper background determination than the distribution obtained from the analysis of areas, where motion is detected and presence is not verified. Data accumulation combined with motion detection and presence verification conditions leads to the detection of points belonging to vehicles and not to the background. During accumulation carried out in motion detection periods the videodetection areas contain both the gray levels attributed to the background (road surface or the road surface signs) and the gray levels related to vehicle pixels. Whereas statistically the vehicle pixels are distributed uniformly the dominant value (for the average calculation) is a value related to the background. The analysis of time histograms has confirmed that such property is observed after time period equivalent to several hundreds of frames, what can be considered as short time in the background generation process.

For the needs of hardware implementation a variant of the above mentioned modification has been elaborated, in which the background generation has been carried out only during the vehicle motion intervals. If the total traffic, detected by subtraction of two consecutive frames, was less then a predefined threshold the background generation process was stopped. The results exhibit a considerable improvement in relation to the method proposed in paper [7] (comp. Fig. 2b). It should be pointed out that the algorithm using the accumulation in motion detection periods turns out to be effective in the high and medium traffic intensity, but in the periods when the traffic is incidental, on the whole carriage way or on some individual traffic lanes, the background will not be regenerated. Therefore in the further stages an additional condition will be determined for switching on the accumulation in the traffic absence periods for a predetermined time. It is also necessary to vary the type of algorithm used for individual traffic lanes, because the "no traffic" condition may be observed on a single traffic lane (e.g. the turn right/left lane). Therefore it has become necessary to specify separately verified detection areas for individual traffic lanes. The hardware implementation has also imposed some limitations on the weighted average determination algorithm. Direct application of formula (1) would result in the necessity of using floating point division, the realization of which would be too memory demanding and ineffective (as calculation time is concerned) for the applied hardware platform. The solution leading to practical absence of division has been connected with another problem encountered in the described methods. Both in the "histogram" solution as well as in formula (1) the summation of pixel values from consecutive frames (time accumulation) is carried out. In the continuous operation mode of the application based on the above mentioned algorithms it has often lead to overflows. Therefore a periodic re-scaling of the sum of pixel values is included, realized as a periodic division of accumulating values (accompanied by division of the frame counter). For the purpose of hardware implementation. the following algorithm modification has been introduced. Up to a certain value of the frame counter (being a integer power of two - experimentally 512 has been verified as sufficient) the pixel values are merely added, without calculation of the weighted average. Due to the fact that the frame counter value is a power

10th Euromicro Conference on Digital System Design Architectures, Methods and Tools (DSD 2007) 0-7695-2978-X/07 $25.00 © 2007

a)

b)

c)

d)

Figure. 3. Exemplary application of the generated background for detection of vehicle presence : a) individual video frame; b) generated background; c) differential image between actual frame and background; d) the result of segmentation by percentage thresholding of two the actual division in formula (1) can be replaced by a right shift operation. At the same moments the periodic reductions of the pixel sum value and respectively frame counter value are executed, whereas the actual division is replaced by easily executable and faster fixed-point multiplication by inverse number. The multiplier is properly selected in order to decrease the frame counter by the required value (the value determines the frame interval between the consecutive background determination operations in presented example it was taken as 32). When the frame counter reaches the predetermined value (512) again the operation is repeated. Due to this procedure the sum of pixel values periodically oscillates instead of growing towards overflow, and as a "spin-off" the cumbersome division operation can be replaced by much faster shift operation. Fig. 3 contains the results of consecutive videdetection stages . After the background generation the further stage of videodetection is the calculation of differential image (Fig. 3c), between the current frame (Fig.3a) and the background (Fig. 3b). For the purpose of testing of the background generation method quality

the last stage consisted of segmentation method using a percentage thresholding. The results of segmentation are, at the present stage, an approximate but sufficient measure of functioning quality of the background generation method. For example the absence of light spots in Fig. 3d in the areas where the road surface signs are found and the absence of streaks on the traffic lanes with high traffic intensity (the straight-through traffic lanes) is a direct proof of correct functioning of the background generation algorithm.

3. SAD algorithm 3.1. Reference software implementation The SAD algorithm has been employed in many applications, in which traffic estimation is necessary, mainly as a measure of template matching. Exemplary applications - e.g. MPEG video signal compression have been presented by the authors of the following papers: [4], [8], [9], and [12].

10th Euromicro Conference on Digital System Design Architectures, Methods and Tools (DSD 2007) 0-7695-2978-X/07 $25.00 © 2007

In the above mentioned cases the SAD operation can be very time-consuming, because it is carried out for vast image areas and for many various templates. There were many operation speed-up attempts using hardware implementations [11]. For example in paper [13] the authors present an analysis of the SAD algorithm hardware implementation in FPGA devices. The SAD operation has been split into two stages: determination of absolute values and sum calculation. On the basis of estimates of the expected velocity values and respective image areas the SAD operation has been implemented using the carry generator (the absolute value determination stage) and the summation tree (the sum calculation stage). The equation describing a 2D discrete SAD algorithm looks as follows:

C( j, k) =

( Mt−1) ( Nt−1)

∑ ∑abs(I (m + j, n + k) − T (m, n))

m=0

n=0

(3)

where: 0≤ j