Moving Object Contour Detection Based on S-T Characteristics in Surveillance Yuan-yuan Cao1, Guang-you Xu1, and Thomas Riegel2 1
Tsinghua National Lab. On Information Science and Technology, Tsinghua University, Beijing, 100084, P.R. China
[email protected],xgy-dcs@ tsinghua.edu.cn 2 Siemens AG, Corporate Technology, Munich, 81730, Germany
[email protected]
Abstract. We present a method for moving object contours detection based on spatial-temporal characteristics. Using S-T features, the contour of moving object can be well distinguished from background; therefore the moving objects are detected without the need of establishing and updating background models. The detection method can handle situations where the background of the scene suffers from the noises due to the various facts, including the weather condition such as snow or fog and flicker of leafs on trees, and bushes. The algorithm estimates the probability of observing pixel as a contour pixel based on a sample of intensity values for each pixel during a period of time and its local gradient in current frame. The experiments show that this method is sensitive to changes caused by moving objects and is able to avoid the affection of complex background. The paper also shows how to separate multi-person based on the contour detection results using template matching. The approach runs in realtime and achieves sensitive detection. Keywords: motion detection, contour detection, spatial-temporal characteristics, object classification, visual surveillance.
1 Introduction Visual surveillance technology has been attracting more and more efforts due to its importance in security systems. An effective surveillance system relies on moving object detection and classification heavily. A lot of algorithms on motion detection have been proposed, which can be classified mainly into two categories: the region-based and the contour-based. The most popular region-based approach is background modeling with mixture of Gaussians [1], which can handle tough cases like illumination changes and the noise caused by small movement in the background. However, a common problem of background modeling is that it takes somewhat long time to estimate the background models due to the slow cadence of illumination changes and small movement. In the case of contour-based approaches, active contours, such as snake [2], geometric active contour [3], and level sets [4] are widely used. In [5], accurate contour of moving object was extracted by means of integrating color segmentation, motion segmentation, and active contour. Geodesic active M.J. Smith, G. Salvendy (Eds.): Human Interface, Part I, HCII 2007, LNCS 4557, pp. 575–583, 2007. © Springer-Verlag Berlin Heidelberg 2007
576
Y.-y. Cao, G.-y. Xu, and T. Riegel
contours and level sets were used in [6]. However, these techniques are computationally too expensive for real time applications. Some work combined the information from motion and edge detection to extract the contour of a moving object [7], but this method will run into difficulties when scenes are cluttered. In this paper, a novel method for detecting moving objects is proposed. It is observed that the intensity of the pixel where the contour of moving object is passing by, will undergo a sharp change either from foreground to background or background to foreground. In contrast, the intensity of pixels on the background will keep stable except the noise. Figure 1 shows how the gray level of a pixel from outdoor scene changes over a short period time (x-axis means frame number and y-axis means intensity value). The locations of four points (A, B, C, D) in original frame images are shown in figure 2. Because the background usually disturbed by some kind of flicker motion, the intensities in background usually vibrate in high frequency with small amplitude. In contrast, the intensity at the pixel where objects are passing by will undergo significant change. The intensity varies of those pixels depend on object moving speed and the intensity gradient along the motion direction. Therefore the contour of object can be detected by modeling the changes of intensity value in a short period of time. Further, in the case that the moving objects are human being, in order to separate human bodies from a group and the shadow, the detected contours are matched with 2D human contour templates hierarchically based on Distance Transform, which can separate multi-person. The templates need not to be established explicitly. The approach can eliminate the shadow of humans and handle partial occlusions.
(A)
(B)
(C)
(D)
Fig. 1. Intensity value over a period of time (60-140 frames)
Moving Object Contour Detection Based on S-T Characteristics in Surveillance
577
Fig. 2. Position of four points in original frame image
The outline of the paper is as follows. Section 2 presents the proposed algorithm of moving object contour detection. Separation of multi-person based on 2D templates matching is described in Section 3. Section 4 lists experimental results. We conclude in Section 5.
2 Moving Object Contour Detection Let x1 , x2 ," x N be a recent sample of intensity values for a pixel, that this pixel will have intensity value x N at time N. Using this sample, the probability density function can be estimated using the kernel estimator K:
Pr( x N ) =
1 N ∑ K ( x − xi ) N i =1
where
x=
1 N ∑ xi n i =1
(1)
If we choose our kernel estimator function, K, to be a Normal function N (0, ∑) , where ∑ represents the kernel function bandwidth, then the density can be estimated as:
Pr( x N ) =
1 N
N
∑ i =1
1 d
1
e
1 − ( x − xi )T ∑ −1 ( x − xi ) 2
(2π ) 2 ∑ 2
(2)
The larger Pr( xN ) is, the more likely this pixel is a contour pixel. However the detected results will be weak when the contour parallels moving direction. An extreme example see figure 3. To solve this problem, spatial characteristic is combined to measure the probability density by computing the gray gradient of the pixel x N in position (i, j ) . The formula is as follows: ⎧∇x I (i, j) = [I (i − 1, j + 1) + 2I (i, j + 1) + I (i + 1, j + 1)] − [I (i − 1, j −1) + 2I (i, j − 1) + I (i + 1, j − 1)] ⎨ ⎩∇ y I (i, j) = [I (i − 1, j − 1) + 2I (i − 1, j) + I (i − 1, j + 1)] − [I (i + 1, j −1) + 2I (i + 1, j) + I (i + 1, j + 1)]
(3)
578
Y.-y. Cao, G.-y. Xu, and T. Riegel
G( x N ) = ∇ x I 2 + ∇ y I 2
(4)
G ( x N ) is gray gradient of the pixel located in (i, j ) . Normalized G ( x N ) is denoted as Gn ( x N ) . The sum of the weighted Pr( xN ) and Gn ( xN ) is calculated as probability of a pixel being a contour pixel, which is estimated by the formulation:
P( x N ) = α Pr( x N ) + β Gn ( x N )
where
0 < α, β th
(6)
The threshold th is a global threshold over all the images that can be adjusted. The moving object contour detection results are shown in figure 4. For finer results, morphological transformation is employed to eliminate background noises and fill in tiny holes in the detected contour, which are shown in Figure 5.
(a)
(b)
Fig. 3. (a) shows a gray rectangle in a black background is moving in direction which parallels edges a and c. Contour detection result is shown in (b), where a and c are missing for there’s no intensity change on the two edges.
Fig. 4. Moving object contour detection results
Moving Object Contour Detection Based on S-T Characteristics in Surveillance
579
Fig. 5. Detection results after morphological transformation
This detection method can handle situations when the background of the scene is blurred due to the weather condition, such as snow and fog, and the noises caused by the flicker of tree branches and bushes. We test the approach of background modeling with mixture of Gaussians and the method proposed above in surveillant videos captured by a single static camera with a resolution of 320 × 240 . They show a crowd crossroad in snow weather. The results are compared in figure 6.
Fig. 6. Images in the first column are original frame samples (frame 80, frame 117 and frame 178); the second column shows detection results of background modeling and the results of approach proposed in this paper are shown in the last column
580
Y.-y. Cao, G.-y. Xu, and T. Riegel
The experimental results show that background model method is likely to miss the objects whose color is similar to the background. The reason is that in background modeling method, the threshold used to differentiate foreground and background pixels is estimated based on the global statistics, thus can not adapt to different color objects. In our method, the characteristics of relative color change in a local area, instead of absolute difference of color value, are taken into consideration to distinguish moving object contour pixels from others. In the above video viewing from far away, moving objects are relatively small in the image, therefore the detected contour of an object tend to connect as its foreground, However this does not affect the objects detection results. The region bounding algorithm is applied to locate the blob of the detected moving object, results of which are shown in figure 7.
Fig. 7. Detected blob of moving objects
3 Human Detection by Means of Templates Matching For close-view surveillance where the moving objects are relatively large, visual surveillance usually endures difficult situations like a group of people with partial occlusion and heavy shadow. Based on the contour detection results, 2D template matching is employed to separate a group of human and meanwhile eliminate shadows. 3.1 Contour Blob Classification It is supposed that there are only two kinds of objects: human and vehicles. Moving objects are first classified based on the knowledge and formulated as follows:
⎧a single hum an Dispersedn ess > θ 2 area < θ1 ⎪ ClassID = ⎨a group of humans Dispersedn ess > θ 2 area > θ 3 ⎪vehicles Dispersedn ess < θ θ < area < θ 2 1 3 ⎩ where
Dispersedn ess =
height Perimeter 2 Aspect Ratio = width area ,
(7)
(8)
Moving Object Contour Detection Based on S-T Characteristics in Surveillance
581
3.2 Human Contour Template Hierarchy Initialization A human contour template hierarchy is constructed and object contours are matched in a coarse-to-fine way with templates based on Distance Transform hierarchically. 2D human contour templates are constructed by detecting contours of human with different gait (walking with both legs together, one leg forward, and standing, etc.) from training videos. A template hierarchy is initialized in the way that similar templates are grouped together and represented by a kernel template which holds common properties of this group. Matching is first done with kernel templates rather than with each individual template, a speed-up will then be achieved. See Figure 7 for a general view of template hierarchy which is generally enough to represent human contour in surveillance. Human head and shoulder contour, the common part of all human contour templates, is chosen as the kernel template. 3.3 Human Separation and Shadow Elimination Distance Transform (DT) [8] based matching is employed to match contour images with the templates. The average distance to the nearest feature, namely chamfer distance, is chosen here as the match measure, which is represented as follows: Dist chamfer (M, I) =
1 M
∑ d ( I , p)
(9)
p∈M
Where |M| denotes the number of contour pixels in template M and d ( I , p) denotes the distance between contour pixel p in M and the closed contour pixel in I. A template is considered to be matched at a certain location when the distance measure Dist chamfer (M, I) at this position is below a predefined threshold θ .
Distchamfer (M, I) < θ
(10)
Fig. 8. Human contour template hierarchy
4 Experiment Results The proposed method for detection and recognition of moving object have been tested with the videos captured by a single static camera in a resolution of 320 × 240 . There are a wide road with shadows of trees and humans casting heavy shadows in the video.
582
Y.-y. Cao, G.-y. Xu, and T. Riegel
Figure 8 shows some segmentation results,for individuals. The human contour detection results are shown in the second column. The right column shows multiperson separation and shadow elimination results, where humans without being occluded are outlined with blue lines; occluded ones are outlined in the head and shoulder with red lines. The method, however, encounters some difficulties in certain circumstances. When human or vehicle is far away from the camera, the algorithm fails to detect a clear contour; as mentioned in section 2, as a result the template-based algorithm can not be applied. Another difficulty arises when the head and shoulder of a human is occluded.
Fig. 9. The first column is original frame images; the second column are results of moving object contour detection; the right column are results of multi-person separation and shadow elimination
5 Conclusion In this paper, a method has been proposed for moving objects detection, multi-person separation and shadow elimination. We employed spatio-temporal characteristics to model the intensity value change of each pixel through a short period time and picked out those lying in contour area. A coarse-to-fine strategy based 2D template matching method is employed to separate multi-person and eliminate shadows. Acknowledgement. This work was funded under Project 60673189 supported by National Science Foundation of China: Event Detection and Understanding in Dynamic Context for Implicit Interaction
．
Moving Object Contour Detection Based on S-T Characteristics in Surveillance
583
References 1. Harville, M.: A framework for high-level feedback to adaptive, per-pixel, mixture-ofgaussian background models. In: European Conference on Computer Vision (2002) 2. Seo, K.H., Lee, J.Y., Lee, J.J.: Adaptive color snake tracker using condensation algorithm. In: 5th Asian Control Conference (2004) 3. Huang, F.Z., Su, J.B.: Face contour detection and tracking with complex backgrounds. Proceedings of 2004 International Conference on Machine Learning and Cybernetics (2004) 4. Sethian, J.: Level set methods and fast marching methods. Cambridge Univ. Press, Cambridge (1999) 5. Qiu, L., Li, L.: Contour extraction of moving objects. In: Proc. IEEE Int’l Conf: Pattern Recognition (1998) 6. Paragios, N., Deriche, R.: Geodesic active contours and level sets for the detection and tracking of moving objects. IEEE Trans. Pattern Analysis and Machine Intelligence (2000) 7. Nagao, K.: Detecting contours in image sequences. IEICE Trans. Information and Systems, vol. E76-D(10) 8. Borgefors, G.: Distance Transformations in Digital Images. In: CVGIP (1986)
