Moving Shadow Detection with Support Vector Domain Description in the Color Ratios Space Kais Siala
Moez Chakchouk Faten Chaieb
Olfa Besbes
Telecommunication Studies and Research Centre (CERT) - RACINES Cit´e Technologique des Communications, BP. 111, El Ghazela 2088, Ariana, Tunisia.
[email protected]
Abstract Moving shadow detection is a fundamental step in videosurveillance applications since it is generally confused with foreground. In this paper, we propose a novel statistical non-parametric method to detect moving shadow in a road traffic image sequences. We consider a diagonal model to describe the shadow distortion in the RGB color space. A Support Vector Domain Description (SVDD) algorithm is applied in the color ratios space in order to discriminate shaded pixels from foreground.
1. Introduction The main purpose of computer vision based traffic monitoring applications is to detect and track moving objects (vehicles, pedestrians, etc.). Nowadays, Intelligent Transportation Systems (ITS) including efficient video processing and analysis modules are becoming a reliable solution for traffic flow management. They provide efficient analysis of traffic scenes and road activities. Detecting moving shadows is still a challenge since they cause object misclassification, object merging, etc. A survey of moving shadow detection methods is proposed in [8]. The authors present a four-classes taxonomy of shadow detection algorithms according to the decision process. They distinguish Statistical Non-Parametric (SNP), Statistical Parametric (SP), Deterministic Model-based (DM) and Deterministic Non-Modelbased (DNM) approaches. Evaluation metrics are also introduced in order to compare these different approaches. The algorithm described in [3] is an example of the DNM approach. The moving shadow detection is carried out in the HSV color space that generally provides more accuracy in distinguishing shadows [8]. In fact, shadow has a constant hue and a lower saturation. Furthermore, Horprasert,
et al. proposed a SNP method that models the shadow appearance in the RGB color space [6]. Both color and brightness distortions are computed and thresholded in order to identify shadow pixels. The appropriate thresholds are computed by a statistical learning procedure. In addition, a SP method is introduced in [7]. It maximizes the a posteriori probabilities of belonging to background, foreground, and shadow classes. The a priori probabilities of a pixel belonging to a shadow are computed by assuming the diagonal model distortion [4]. An Expectation Maximization (EM) is applied to select automatically the appropriate parameters [5]. The shadow detection problem has also been addressed for gray level video-surveillance images [1, 10]. The luminance ratio between current and background image is considered constant for shadow. A Gaussian fitting is used to resolve the classification problem. In this contribution, we propose a statistical nonparametric shadow detection method. The diagonal model is used to describe shadow appearance. This model takes into account the difference between sensor’s channels sensitivities. A Support Vector Domain Description (SVDD) algorithm [11] is performed in order to classify shadow pixels in the 3D color ratios space. This paper is organized as follows. We first recall the diagonal model used for shadow appearance description. In the next section, we present our shadow detection algorithm. Then, some experimental results are presented and evaluated. Finally, a conclusion and some future work are given.
2. Shadow modeling For an image acquired by a CCD camera, the color intensity of a pixel Ü is given by:
Ü
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
Ü Ü
(1)
where is the visible spectrum, is the wavelength parameter, is the illuminant function, is the reflectance factor and is the channel camera sensitivity. The illuminant function is expressed as a function of the direction Ä of the light source, the object surface normal Æ, and the intensities and of light source and ambient light respectively [10]:
Ü
Æ Ü
Ü Ä if illuminated Æ Ü Ä if penumbra if umbra
(2) where Ü is a parameter describing the penumbra transition between shaded and illuminated regions. Therefore, shadow (umbra and penumbra) is considered as a case of an illumination change. Under the assumptions of lambertian surfaces and narrow band sensors sensitivities, shaded pixels values are given by an independent scaling of the sensor’s channels responses [4]. The distortion between a background image and a current image (at time ) of a video-surveillance sequence expressed in the RGB color space, can be approximated for shaded regions by the well known diagonal model [4]:
(3)
where and are respectively the RGB color values of a shaded pixel in and . The color ratios , and are less than unity because shaded pixels are darker than the background. As shadow regions are composed of an umbra and a penumbra involving soft luminance transitions from shaded and non-shaded background. The mean color ratios vector characterizes the umbra distribution. For the penumbra pixels, the elements of color ratios vector vary between and one.
Figure 1. A scatter plot in the color ratios space of a shaded pixels set. The line corresponds to .
diagonal model seems then to be more efficient for shadow description than the scalar one used in [1, 6]. The shadow detection is obtained by performing a oneclass classification in the color ratios space. We are interested in a non-parametric data modeling in order to describe the shadow domain in that space. We propose to apply the Support Vector Domain Description (SVDD) algorithm [11]. The SVDD algorithm consists in finding the minimal radius hypersphere containing the above described training samples . This yields to maximize a criterion with respect to the Lagrange multipliers :
(4)
with constraints et . The parameter regulates the trade-off between the volume of the hypershpere and some supposed outliers. Analogous to [12], inner products in the equation 4 are replaced by a Kernel function in order to deal with non-spherical shapes. We use a classical Gaussian Kernel given by
3 Shadow detection
In the learning step, a representative image containing the three classes : foreground, moving shadow and background is arbitrary selected. The moving shadow regions are manually segmented. Color ratios are computed for pixels issued from a bootstrap sample. Their scatter plot shown in figure 1 includes a central part that characterizes the umbra. The penumbra points vary from this central part towards the unit vector corresponding to the non-distorted pixels . In addition, it is important to notice that the shadow features (color ratios) are not centered on the line . The
(5)
The equation 4 then becomes
(6)
In the online phase, the shadow detection is only performed for moving pixels determined by a change detection algorithm [2, 9]. For each moving pixel, a color ratio vector is determined. An observation is considered within the shadow description domain when
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
(7)
where is the radius of the optimal hypersphere determined as in [11]. In figure 2 we show the boundary of the shadow domain in the color ratios space. It is to notice that only support vectors corresponding to non-zero are involved in the acceptation rule (7). This is suitable for fast detection since we generally obtain a reduced number of support vectors.
(a)
(b)
(c) CR-1
(d) HW1-200
(e) CR-20
(f) HW1-243
Figure 2. The boundary of shadow domain obtained with and . black discs and black dots correspond respectively to support vectors and training points.
4 Experimental results We applied our method on two video-surveillance image sequences: cross-road (CR) and highway-I (HW1). Figure 3 shows some frames of these sequences. The background images presented in figures 3(a) and 3(b) are estimated by a spatio-temporal median filtering. In the CR images, shadow appearance is not constant. It is darker in pixels lying close to moving objects and lighter in the frontier due to the penumbra effect. The HW1 sequence presents dark shadow regions and strong self-shadows. In the learning step, the SVDD algorithm is applied on a bootstrap training shadow samples extracted from the images shown in figures 3(c) and 3(d).The moving shadow detection results for the CR and HW1 images are shown in the figure 4. Black, gray and white colors correspond to background, foreground and shadow pixels respectively. It is important to notice that the erroneous small regions can be removed by using some morphological operations. Furthermore, in the HW sequence, the foreground detection seems to be clearly affected by the self shadow effect. In order to evaluate our method, we have used the shadow detection rate and the shadow discrimination rate defined in [8] as
Figure 3. The estimated background image of the (a) CR and (b) HW1 sequences. (c-f) Some images from CR and HW1 sequences.
follows:
(8)
(9)
where (resp. ) is the number of shadow (resp. foreground) points correctly identified, (resp. ) is the number of shadow (resp. foreground) points badly identified, and is the number of ground truth of the foreground objects minus the number of points detected as shadows but belonging to foreground objects. These evaluation metrics are applied for the detection results presented in figure 4. Results are presented in table 1.
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
[3]
[4]
(a) CR-1
(b) HW1-200
[5]
[6]
[7] (c) CR-20
(d) HW1-243
Figure 4. The shadow and foreground detection masks.
Table 1. Evaluation of the proposed method results. Image
CR-1 77.21% 94.85%
CR-20 68.80% 96.52%
HW1-200 89.08% 70.3%
HW1-243 77.51% 67.53%
[8]
[9]
[10]
[11]
5 Conclusion and future work In this paper, a novel shadow detection method was proposed. The SVDD algorithm is used to identified shadow in the RGB color ratios space. Promising results on real video-surveillance traffic sequences are obtained. In our future works, we aim to develop an incremental learning approach in order to deal with temporal variation in shadow appearance. The self shadow and color invariants could be considered to improve accuracy.
[12]
Proceedings of SPIE Electronic Imaging - Visual Communications and Image Processing, San Joe, CA USA, pages 465–475, 2001. R. Cucchiara, C. Grana, M. Piccardi, and A. Prati. Detecting moving objects, ghosts, and shadows in video streams. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10):1337–1342, october 2003. D. A. Forsyth. A novel algorithm for color constancy. International Journal of Computer Vision, 5(1):5–36, august 1990. N. Friedman and S. Russell. Image segmentation in video sequences : A probabilistic approach. In Proceedings of 13th conference on Uncertainty in Artificial Intelligence (UAI’97), August 1–3, 1997, Rhode Island, USA, pages 175– 181, 1997. T. Horprasert, D. Harwood, and L. S. Davis. A statistical approach for real-time robust background subtraction and shadow detection. In Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV’99), September 20–27, 1999, Kerkyra, Greece, 1999. I. Mikic, P. C. Cosman, G. T. Kogut, and M. M. Trivedi. Moving shadow and object detection in traffic scenes. In Proceedings of the International Conference on Pattern Recognition (ICPR’00), September 3-8, 2000, Barcelona, Spain, volume 1, pages 1321–1324, 2000. A. Prati, I. Mikic, M. M. Trivedi, and R. Cucchiara. Detecting moving shadows algorithms and evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(7):918–923, july 2003. K. Siala, O. Besbes, and M. Chakchouk. Vehicle tracking by label following. In Proceedings of the International Symposium on Computational Intelligence and Intelligent Informatics (ISCIII’03), 19-23 May 2003, Nabeul, Tunisia, pages 21–23, 2003. J. Stauder, R. Mech, and J. Ostermann. Detection of moving cast shadows for object segmentation. IEEE Transactions on Multimedia, 1(1):65–76, march 1999. D. M. J. Tax and R. P. W. Duin. Support vector domain description. Pattern Recognition Letters, 20(11–13):1191– 1199, november 1999. V. Vapnik. The nature of statistical learning theory. Springer, New York, 1995.
References [1] A. Bevilacqua and M. Roffilli. Robust denoising and moving shadows in traffic scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’01), Hawaii, USA, December 8–14, 2001. [2] A. Cavallaro and T. Ebrahimi. Video object extraction based on adaptive background and statistical change detection. In
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
