Moving Cast Shadow Detection Using Texture Information Mani Ranjbar1, Shohreh Kasaei2 1,2
Department of Computer Engineering, Sharif University of Technology, Tehran, Iran
[email protected] ,
[email protected]
Abstract In this paper, we have proposed an efficient moving cast shadow detection using texture features. Texture features are estimated using fractal dimension and is employed to discriminate between shadow region and moving foreground region. Complexity and noise reduction is achieved using wavelet transform which has not been performed before in the literature. Block-wise calculation has accelerated the method. The proposed texture modeling method unlike most other approaches is able to cope with very dark shadows as well as light shadows.
Keywords Cast Shadow Detection, Dark Shadow, Texture.
1. Introduction Detection and tracking of moving objects is the core of many applications dealing with image sequences. One of the most important challenges of these applications is identifying the moving object and its cast shadow. Shadows cause serious problems when segmenting and extracting moving objects, due to misclassification of shadow regions as foreground. Shadows can cause object merging, object shape distortion and even object missing. The difficulties associated with shadow detection arise since shadows and objects share two important visual features. First, shadow regions are detectable as foreground areas since they typically differ significantly from the background. Second, shadows have the same motion as the objects casting them. As such, shadow identification is critical both for still images and for image sequences (video) and has become an active research area. Shadow analysis, considered in the context of video data, is typically performed for enhancing the quality of segmentation results instead of deducing some imaging or object parameters. In the literature, shadow detection algorithms are normally associated with techniques for moving object segmentation, such as the ones based on inter-frame differencing [3], background subtraction [4], optical flow [5], statistical point classification [6], feature matching, and tracking [7]. In general, there is a difference between dealing with dark shadows and light shadows. Because, light shadows only affect the intensity of the background and the moving object does not change the background color and texture, while dark shadows change the background intensity, color and texture. As a result, distinguishing between object and its shadow is easier in the scene with
light shadow. Most of the available methods give their results for light shadows, but our results are given for quite dark shadows. Moreover, our method uses background texture information which is reliable in quite dark shadow situations.
2. Related Works For moving object segmentation, various methods such as inter-frame differencing [3], background subtraction [4], optical flow [5], statistical point classification [6], feature matching [7], and the forth have been proposed. Background subtraction due to its low complexity and proper accuracy is used more than other methods for movement detection. Success of this method is based on the accurate background modeling. A popular method for background modeling is using single Gaussian distribution as used in [8]. Improved versions have used more than one Gaussian distribution for background modeling [9]. Not completely static background can be modeled better with mixture of Gaussian distribution. Many methods have proposed model parameter estimation approaches. In [10], one of the commonly used approaches for updating Gaussian mixture model is presented. In [11], the number of mixture components is constantly adopted for each pixel. Non-parametric approaches for dealing with limitation of parametric models such as Gaussian assumption for pixels intensity is proposed in [12]. Background modeling is performed using edge features in some methods. In [13], edge histograms are used for background modeling. In some later methods like [14], fusion of edge and intensity are used. To the authors’ best knowledge, [1] is the first method which has used texture information for shadow identification. That method uses local binary pattern
(LBP) operator to estimate the texture of a region. This operator works well in light shadow conditions but as mentioned above, darker shadows decrease the difference between pixels’ intensity and cause LBP operator to fail. In [2], a morphology-based multi-fractal method for texture segmentation is proposed which in the first step calculates the fractal dimension of a texture and uses it for texture segmentation in the second step. This method reaches very good accuracy/complexity compared to other methods. According to above reasons, we adopted this texture estimation method in our algorithm.
3. Proposed Method
where f t (i, j ) is the image intensity value in (i, j) location, Pε is the number of structure elements (SE), β is a constant which determines the shape of SE, and the scale. Let
f ∝' (i, j ) = max { f (i + iek , j + j∝k )} . k =1, 2 ,.., P∝
As
βε
The main novelty of this paper is the segmentation of an object and its shadow where the shadow is quite dark; and therefore we have assumed that the background can be modeled by a Gaussian distribution as used in [8]. In non-static background conditions, the methods proposed in [9] or [11] can be used to obtain the background model, test image and a mask image which shows the location of moving objects (objects and their shadows). In our background training phase, the mean and the variance of N consecutive frames are calculated to obtain the Gaussian model of the background. Then, the difference between the background model and the input image sequence is used to discriminate between the background and the moving objects.
is
(2)
is independent to k, (1) can be rewritten as
f ε (i, j ) = f ε' (i, j ) + βε .
3.1. Background Modeling
ε
(3)
The local nature measure in a window of size W × W is defined as:
f ε (i, j ) − f (i, j )
μ ε (i, j ) =
∑
i , j∈W
f ε (i, j ) − f (i, j )
.
(4)
The measure of order q can be calculated as:
I ( q, ε ) ≡ α
∑ μ ε (i, j )
q
(5)
i , j∈W
where
4. Texture-Based Moving Shadow Detection
α=
∑
i , j∈W
4.1. Assumptions
f ε (i, j ) − f (i, j )
ε
.
(6)
In our point of view, the difference between moving object and its shadow is that the moving object changes the background texture but the shadow does not. Therefore, the shadow intensity should not be that high that totally changes the background texture.
Now, fractal dimension that is assigned to the central pixel of the window, is calculated by finding the slope of the line that is constructed using { ln(1 / ε ) , ln( I ( q, ε )) } points.
4.2. Texture Estimation
4.3. Complexity and Noise Reduction
Fractal estimation is, as described by Mandelbrot, based on the idea of “measure the fractal set at different scales.” The fractal dimension can be obtained from the relation between the measures acquired and the scales used. He gives, as an example, the calculation of the length of the British coastline [15]. Different types of fractal dimension estimation are proposed. Here, we have used the method presented in [2] with some modifications and adjustments. This method leads to accurate estimation and lower execution complexity compare to other methods. The algorithm for estimating fractal dimension is as follow:
To decrease the complexity of the overall texture feature extraction and reducing the image noise concurrently, we have used the wavelet transform. At the first level of the wavelet domain, the size of the averaged sub-image is a quarter of the original image size which reduces the complexity cost by a factor of 4. As the edge information is very important for texture estimation, before extracting the texture, the horizontal and the vertical sub-images are added to averaged sub-image. Note that most of the noise data are located in the diagonal sub-image and thus not using its data results in noise reduction and improves the texture extraction quality.
f ε (i, j ) = max { f (i + iek , j + jεk ) + βε } k =1, 2 ,.., Pε
(1)
4.4. Discriminating between Object and Shadow Regions Shadow regions do not change the background texture seriously as moving object regions do. This means that the fractal dimension of a background point in a window does not change seriously when shadow occurs in it. Another obvious fact is that when shadow occurs in a scene, the intensity of the related background region becomes darker. Above assumptions helps us to discriminate between a moving object and its shadow. We can formulate these assumptions to find the class of each pixel in the image. In the following, class 1 indicates the moving foreground region and class 0 indicates the moving shadow region. f t (i, j ) > f b (i, j ) ⎧ ⎪1 or ⎪ PT (i, j ) = ⎨ ( FDmax (i, j ) / FDmin (i, j )) > τ f ⎪ ⎪ otherwise ⎩0
⎫ ⎪ ⎪ ⎬ ⎪ ⎪ ⎭
(7)
where
and
T_Texture=Find_FD(WT_Test) Max_Texture=Max(B_texture,T_Texture) Min_Texture=Min(B_texture,T_Texture) if ( Max_Texture/Min_Texture > Texture_Thr) WT_PT(i,j)=1 end end end PT=Scale(PT,2)
5. Experimental Results
FDmax (i, j ) = max( FDt (i, j ), FDb (i, j )) FDmin (i, j ) = min( FDt (i, j ), FDb (i, j )) where f t (i, j ) and f b (i, j ) are
WT_Test=Wavelet Transform(Test_image) WT_Moving = Scale(Moving,0.5) WT_PT=Scale(PT,0.5) For i=1 to Height(WT_Test) For j=1 to Width(WT_Test) if ( WT_Moving(i,j) WT_PT(i,j)==0) B_Texture=Find_FD(WT_Background)
the
foreground
background intensity values in (i, j) and τ
f
and
is the fractal
dimension threshold. Experiments have shown that a value between 1.02 and 1.2 is the best choice.
4.5. Overall Algorithm In this section, the pseudo code of the overall algorithm is presented. The algorithm consists of two separate parts. In the first part, the moving foreground region and its shadow are detected. In the second part, the moving foreground area is discriminated from its shadow.
In our experiments we have set the Diff_Thr value to 0.1 and Texture_Thr value to 1.1. As in other works reported in the literature, the results are given only for very light shadows, in Figure 1, we have compared the performance of our proposed algorithm with other methods in a scene having a light shadow. The result shows that our method obtains significantly better segmentation result than the two other methods. In Figure 2, a scene with darker shadow is examined. The result shows that our algorithm can cope with dark shadows where two other methods lead to low performance results. As mentioned above, it can be seen that methods which assume that hue and saturation do not change significantly due to shadow (as in [16]) cannot cope with quite dark shadows. It can be seen from these results that texture-based methods obtain higher performance than intensity-based methods.
6. Conclusion Part1: Mean=0 For i=1 to N Mean=Mean+Background(i) end Mean=Mean/N Difference=abs(Mean-Test_image) Moving=Difference>Diff_Thr Part2: For i=1 to Height(Mean) For j=1 to Width(Mean) if ( Test_image(i,j)>Mean(i,j)) PT(i,j)=1 end end end WT_Background = Wavelet Transform(Mean)
In this paper, we have proposed an efficient texture-based moving shadow detection method using the difference between fractal dimension of the background and the foreground. For accelerating the method and denoising it, the wavelet transform is used. As the experimental results show for light shadows the proposed method performs superior to other available algorithms and for dark shadows the performance of the proposed algorithm is promising when compared to other available algorithms.
References
(a)
(b)
(c)
(d)
(e)
(f)
Fig. 1: (a) Input image, (b) background mean, (c) ideal segmentation, (d) using LBP operator [1], (e) using HSV information, (f) proposed method.
(a)
(b)
(c)
(d)
(e)
(f)
Fig. 2: (a) Input image, (b) background mean, (c) ideal segmentation, (d) using LBP method [1], (e) using HSV information, (f) proposed method.
Acknowledgement This work was supported in part by a grant from ITRC.
[1] Marko Heikkilä, and Matti Pietikäinen, “A Texture-Based Method for Modeling the Background and Detecting Moving Objects, ” IEEE Trans. On Pattern Analysis and Machine Intelligence, vol. 28, No. 4, pp. 657-662, April 2006. [2] Yong Xia, Dagan Fang, and Rongchun Zhao “Morphology-Based Multifractal Estimation for Texture Segmentation,” IEEE Trans. On Image Processing, vol. 15, No. 3, March 2006. [3] Y. Kameda and M. Minoh, “A Human Motion Estimation Method Using 3-Successive Video Frames, ” in Proceedings of International Conf. on Virtual Systems and Multimedia, 1996, pp. 135-140 [4] A. Nerri, S.Colonnese, G. Russol, and P. Talone, “Automatic Moving Object and Background Sepration,” Signal Processing, vol. 66, No. 2, pp. 219-232, 1998 [5] A. Bainbridge-Smith and R.G. Lane, “Determining Optical Flow Using a Differential Method,” Image and Vision Computing, vol. 17, pp. 11-22, 1997. [6] I. Mikie, P. Cosman, G. Kogut, and M.M. Trivedi, “Moving Shadow and Object Detection in Traffic Scene,” in Proceedings of Int'l Conf. on Pattern Recognition, Sept. 2000. [7] D. Koller, K. Daniilidis, and H.H. Nagel, “Model-Based Object Tracking in Monocular Image Sequences of Road Traffic Scenes,” International Journal of Computer Vision, vol. 10, pp. 257-281, 1993. [8] C. R. Wren, A. Azarbayejani, T. Darrell, and A. P. Pentland, “Pfinder: Real-Time Tracking of the Human Body,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 780–785, 1997. [9] N. Friedman and S. Russell, “Image Segmentation in Video Sequences: A Probabilistic Approach,” Proc. Conf. Uncertainty in Artificial Intelligence, pp. 175–181, 1997. [10] C. Stauffer and W. E. L. Grimson, “Adaptive Background Mixture Models for Real-Time Tracking,” Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 246–252, 1999. [11] Z. Zivkovic, “Improved Adaptive Gaussian Mixture Model for Background Subtraction,” Proc. Int’l Conf. Pattern Recognition, vol. 2, pp. 28–31, 2004. [12] A. Elgammal, R. Duraiswami, D. Harwood, and L. S. Davis, “Background and Foreground Modeling Using Nonparametric Kernel Density Estimation for Visual Surveillance,” Proc. IEEE, vol. 90, no. 7, pp. 1151–1163, 2002. [13] M. Mason and Z. Duric, “Using Histograms to Detect and Track Objects in Color Video,” Proc. Applied Imagery Pattern Recognition Workshop, pp. 154–159, 2001. [14] S. Jabri, Z. Duric, H. Wechsler, and A. Rosenfeld, “Detection and Location of People in Video Images Using Adaptive Fusion of Color and Edge Information,” Proc. Int’l Conf. Pattern Recognition, vol. 4, pp. 627–630, 2000. [15] B. B. Mandelbrot, “How long is the coast of Britain? Statistical self similarity and fractal dimension,” Science, vol. 156, pp. 636–638, 1967. [16] R. Cucchiara, C. Grana, M. Piccardi, A. Parati, S. Sirotti, “Improving Shadow Suppression in Moving Object Detection With HSV Color Information,” IEEE Intelligent Transportation Systems, pp. 334-339, 2001.
