Feature Selection for Reliable Tracking using Template Matching Toshimitsu Kaneko Multimedia Laboratory Corporate R&D Center, Toshiba Corporation Kawasaki, 212-8582 Japan
[email protected]
Abstract A new feature selection method for reliable tracking is presented. In this paper, it is assumed that features are tracked by template matching where small regions around the features are defined as templates. The proposed method selects features based on the upper bound of the average template matching error. This selection criterion is directly related to the reliability of tracking and hence, the performance is better than that of other feature detectors. Experimental results are presented to confirm the efficiency of the proposed method.
1. Introduction Feature tracking through image sequences is one of the basic procedures in computer vision and widely used in many applications. Features are usually selected prior to the tracking procedure. This selection method is important because the selection result has significant influence on the performance of the tracking. In many cases, features to be tracked are selected by corner detectors[4][2][7]. This is because humans think that corners are usually good features to be tracked. However, since the cornerness measure does not always correspond to goodness for tracking, good tracking performance of the selected features is not guaranteed. On the other hand, the feature selection methods based on how reliable tracking can be done by Lucas-Kanade tracking procedure[3] have been developed [8][5]. This approach is preferable because the goodness of a feature depends on tracking methods. However, no feature selection method specialized for template matching has been proposed until now though template matching is widely used. In this paper, a feature selection method suitable for tracking using template matching is proposed. The selection criterion is the upper bound of the template matching error. This criterion is directly related to the performance
1063-6919/03 $17.00 © 2003 IEEE
Osamu Hori Multimedia Laboratory Corporate R&D Center, Toshiba Corporation Kawasaki, 212-8582 Japan
[email protected]
of tracking. In fact, the exprimental results show that the proposed feature selection method gives better performance than well-known feature detectors. The relation between the proposed method and the local autocorrelation based feature detectors, and the evaluation of feature combinations for affine transformation parameter estimation, are also discussed.
2 Average Template Matching Error 2.1 Template Matching Model In this section, an upper bound related to template matching errors is derived after the definition of the template matching model. Let f0 be the image including a feature to be tracked, and f1 be the image in which the feature position is searched (see Figure 1). Here, we assume that f1 is the image corrupted by pixel-wise noise on f0 . The origin of the coordinate system in each image is defined as the true feature position. In this coordinate system, the estimated feature position in f1 is equal to the error vector. Denote an image block in fi (i ∈ {0, 1}) whose center is p = (px , py )T by Wi (p). The template matching tries to find the feature position in f1 by searching a block W1 (p) in S which has minimum dissimilarity to W0 (0). W0 (0) is called a template1 and S is a search region. Let fi (p) be the intensity of fi at p. Assuming that the difference between f0 (p) and f1 (p) is a Gaussian noise N (0, σ 2 ), the sum of squared distance (SSD) is the optimum dissimilarity in terms of maximizing likelihood. SSD between W1 (p) and W0 (0) is calculated as d(p) =
2
(f1 (q) − f0 (q − p)) .
q∈W1 (p) 1 Here, a template is assumed to be a block for simplicity. The discussion in this paper is applicable to templates of arbitrary shape.
W1 (e) W0 (0) (Template) W1 (p) r X X X XX XXX p X z Feature W1 (0) S (Search region) Image f0 Image f1
In the following, an estimation method of the probability that a certain error occurs by template matching is considered.
2.2 Upper Bound of the Average Error of Template Matching Let Pr(f1 |v) be the probability that the image f1 is observed when the displacement of the feature is v. Then, the probability Pr(e) that the error vector e occurs by template matching is (1) Pr(e) = Pr(f1 |0)δ(f1 , e)df where the indicator δ(f1 , e) is defined as 0, Pr(f1 |v) > Pr(f1 |e)for some v ∈ S 1, Pr(f1 |v) ≤ Pr(f1 |e) . δ(f1 , e) = for all v(= e) ∈ S
(2)
holds for arbitrary v. Therefore, by substituting (2) into (1) and selecting v = 0, Pr(e) is upper bounded by ρ Pr(f1 |e) df1 (3) Pr(e) ≤ Pr(f1 |0) Pr(f1 |0) for 0 ≤ ρ ≤ 1. In template matching, Pr(f1 |0) and Pr(f1 |e) are evaluated using only intensities in blocks W1 (0) and W1 (e). Therefore, under the assumption that the noise occurs independently, Pr(f1 |e) is calculated as Pr(f1 |e) = Pr(f1 (p)|e). (4) p∈W1 (e)
0
WE (e) 3 (((( WC (e) ((( e
WT (e) f1
Figure 2. Definition of WT (e), WC (e) and WE (e).
Figure 1. Feature tracking using template matching.
By using Chernoff bound, ρ Pr(f1 |e) δ(f1 , e) ≤ (0 ≤ ρ ≤ 1) Pr(f1 |v)
W1 (0)
Similary, Pr(f1 |0) is given by simply replacing e by 0. For further calculation, define 3 regions related to Pr(f1 |0) and Pr(f1 |e) as WT (e) = W1 (0)∩W1 (e), WC (e) = W1 (0)∩ W1 (e) and WE (e) = W1 (0)∩W1 (e) as shown in Figure 2. By substituting (4) into (3) and reordering the components, the upper bound becomes Pr(e) ≤ PT PC PE where PT
=
p∈WT (e)
PC
=
p∈WC (e)
PE
=
Pr(f1 (p)|0)1−ρ df1 (p), Pr(f1 (p)|0)1−ρ Pr(f1 (p)|e)ρ df1 (p), Pr(f1 (p)|0) Pr(f1 (p)|e)ρ df1 (p).
p∈WE (e)
Pr(f1 (p)|e) is equal to the probability that the noise at p is f1 (p) − f0 (p − e). Therefore, (f1 (p) − f0 (p − e))2 1 exp − Pr(f1 (p)|e) = √ . σ2 2πσ (5) Let the number of pixels in a block be N , and let R(e) be R(e) = |WE (e)|/|W1 (0)|. Then, calculating PT , PC and PE by using (5), the following result is obtained (see Appendix). Upper Bound of Pr(e): Let the average SNR’s be γE (e) γC (e)
= =
1 |WE (e)| 1 |WC (e)|
p∈WE (e)
p∈WC (e)
2
(f0 (p) − f0 (p − e)) , σ2 (f0 (p) − f0 (p − e))2 . σ2
Then, the probability Pr(e) that the error e occurs by template matching is upper bounded by 1 Pr(e) ≤ exp − N E(e) , 2
where max E(e, ρ), ρ 2 + log(1 − ρ ) E(e, ρ) = R(e) γE (e) 1+ρ +(1 − R(e))γC (e)ρ(1 − ρ). E(e)
=
0
