pedestrians and more recognizable human shapes for middle and foreground pedestrians. ... pedestrian candidates in the middle-ground. We search test.
7.2-2 An Efficient Region of Interest Generation Technique for Far-Infrared Pedestrian Detection Ronan O’Malley, Student Member IEEE, Martin Glavin, Member IEEE, and Edward Jones, Member, IEEE Abstract--In recent years, vision systems, including thermal night vision systems, have become a standard part of modern luxury automobiles. The next generation of automotive thermal imaging systems will be expected to automate the detection of pedestrians, to enhance safety. In this paper, we outline the development of a system that will extract Regions of Interest (ROI) from automotive thermal imagery as part of such an automated system, while placing emphasis on low processing requirements, a typical restriction of automotive embedded systems.
I. INTRODUCTION Driven by consumer demand and legislative requirements, the issue of safety in the automotive environment remains a priority. However, recently the issue of safety has extended beyond driver and passengers and the focus has turned towards other Vulnerable Road Users (VRU). It has been shown that the vast majority of accidents are caused by human error or misjudgment, and that more than two thirds of fatal pedestrian accidents happen during the hours of darkness. Night vision systems have already become important safety features and popular consumer product in luxury cars. There is interest in the development of machine vision systems to automate the protection of pedestrians and other VRUs so as not to distract the driver’s attention from the road. Machine vision will perform the task of pedestrian detection and classification as thermal vision and active safety technology becomes more common in road vehicles. The first step in this task is to generate Regions of Interest (ROI) from a scene. These regions are then further processed to classify the region as a pedestrian or non-pedestrian. This paper will outline the current state of the art in automotive infrared ROI generation methods and suggest an adapted method, with emphasis placed on using simple image processing techniques. In the first section we will outline some existing thermal image preprocessing methods and describe our preprocessing step which improves image contrast, producing better end results. The following section reviews the current state of the art in the area of ROI generation. The remainder of the paper outlines a thresholding technique which enhances ROI generation performance, and presents our experimental results. II. THERMAL IMAGE PREPROCESSING When translating the matrix of temperature measurements This research is supported by the Irish Research Council for Science, Engineering and Technology (IRCSET) through the Embark Initiative.
from a thermal sensor into a grayscale image, pixel temperature measurements are assigned intensity values. Several preprocessing procedures have been proposed to improve image contrast and aid the detection and classification processes. It has been suggested to equalize the histogram, to attempt to form a flat histogram [1]. However experimental results have shown that while this does highlight the brightest objects in the scene, object shapes can become severely distorted. Another approach that has been suggested is that the temperature range be distributed evenly over the available intensity range [2]; this is commonly referred to as intensity adjustment. This method is well suited to the most common usage of thermal imagery, industrial machinery maintenance, where the images usually consist of a narrower temperature range than an outdoor environment. However in an outdoor environment, such as the automotive environment, a temperature measurement of the sky tends to return the minimum temperature possible for the device to measure. This phenomenon means the range of temperatures measured is greatly increased, resulting in a compression of the interesting thermal data. This can be observed in Fig. 1(a), with the large spike at zero intensity, followed by a section of low or zero intensity values before returning to positive data values. To compensate for this phenomenon we perform intensity adjustment, and redistribute the thermal data to fill the entire image range. However when redistributing the data we take an
Fig. 1. (a) Original thermal image and corresponding histogram. (b) Image after two-step intensity adjustment and corresponding.
extra step and also fill the intensity values previously assigned to the sky. The resulting image and histogram from this intensity adjustment can be seen in Fig. 1(b). The adjusted image shows improved contrast, and shows improved results in ROI generation testing.
III. ROI GENERATION REVIEW Region of Interest (ROI) generation or segmentation is the process of traversing the image or video frame, and highlighting areas that appear interesting, areas that may potentially be a pedestrian. Fang et al [3] approach the ROI segmentation challenge by devising a two step approach. Their method utilizes projection-based horizontal segmentation followed by brightness and body-line based vertical segmentation. This method uses no machine learning or pattern recognition techniques. Other potential methods for investigation include hotspot thresholding [1] based on the assumption that the pedestrian is one of the hottest object in the image. Grey-level symmetry, edge symmetry and edge density of pedestrians can also be used to segment regions of interest [4]. IV. EFFICIENT ROI GENERATION The majority of the techniques outlined in section III all begin with a common image processing tool, thresholding. While [1] and [3] both define formulae for choosing the threshold value, it has been found that the ability of these formulae to choose an appropriate threshold value varies significantly across different scenes. A suitable threshold value is one which is high enough to exclude background heat around a pedestrian but also low enough to allow the majority of the heat from the pedestrian through. It will also produce a recognizable pedestrian shape in the resulting thresholded image, such as a uniform vertical blob for background pedestrians and more recognizable human shapes for middle and foreground pedestrians. We have determined a formula which consistently defines a suitable threshold value, as follows
T Topt = arg min ∑ H i ≥ kA T i =1
(1)
where Topt is the optimum threshold value, H is the smoothed, intensity adjusted image histogram, A is the total area of the histogram and k is a constant between zero and one. We have determined by experimentation that the ideal value for k is 0.97 for our sample thermal images preprocessed by the method outlined in Section II. Future work may include setting the value of k adaptively depending on the properties of the scene. Noise is then removed from the resulting binary image using morphological operations. The image is labeled and resulting shapes that are outside the height/width aspect ratio bounds of 2.4 - 4.0, proposed in [5] are removed. Xu et al. [1] define body-ground candidates as pedestrian candidates that are split into upper and lower body sections, which is common for pedestrian candidates in the middle-ground. We search test images for such candidates by considering shape pairs whose centroids align in the horizontal to within one pixel. These combined candidates must also meet the preset aspect ratio
constraints applied to the single shape candidates. An example of such a candidate is displayed in Fig. 2.
Fig. 2. The intensity adjusted image after applying thresholding, and highlighting of body-ground regions of interest. V.
RESULTS
From a sample of 14 test images containing 19 pedestrians, 13 are highlighted as ROI by the algorithm, while 6 are not highlighted. The main cause of unlighted pedestrians is background thermal energy, such as a car exhaust, which can distort the candidates shape and put it outside the stringent aspect ratio constraints. Groups of pedestrians are also sometimes not highlighted. There is an average of 0.36 nonpedestrian candidates per image. From a sample of 14 test images with no visible pedestrians there is an average of 0.5 ROI generated per image. VI. CONCLUSION This summary paper outlines the need for an efficient automotive thermal image ROI generation system, so that this technology can progress and meet the rigorous requirements of a consumer product. The paper introduces a preprocessing step to enhance image contrast, which is beneficial from an image processing point of view, but could also be implemented in an automotive consumer night-vision design. We have outlined existing efficient ROI techniques, presented an enhanced thresholding technique and have examined the results produced by the method. Future plans for this system include creating an embedded prototype to demonstrate the achieved processing efficiency, and implementing an adaptive parameter to our thresholding algorithm, dependent on the properties of the scene. REFERENCES [1]
[2]
[3]
[4]
[5]
F. Xu, X. Liu, and K. Fujimura, “Pedestrian detection and tracking with night vision,” IEEE Transactions on Intelligent Transportation Systems, vol. 6, no. 1, p. 63, 2005. C. Bellotti, F. Bellotti, A. D. Gloria, L. Andreone, and M. Mariani, “Developing a near infrared based night vision system,” Intelligent Vehicles Symposium, 2004 IEEE, pp. 686–691, 2004. B. Horn, I. Masaki, Y. Ninomiya, Y. Fang, and K. Yamada, “A shape independent method for pedestrian detection with far-infrared images,” Vehicular Technology, IEEE Transactions on, vol. 53. M. Bertozzi, A. Broggi, P. Grisleri, T. Graf, and M. Meinecke, “Pedestrian detection in infrared images,” Intelligent Vehicles Symposium, 2003. Proceedings. IEEE, pp. 662–667, 2003. M. Bertozzi, A. Broggi, A. Fascioli, T. Graf, and M.-M. Meinecke, “Pedestrian detection for driver assistance using multiresolution infrared vision,” Vehicular Technology, IEEE Transactions on, vol. 53, pp.1666–1678, 2004.
