New preceding vehicle tracking algorithm based on

Measurement 73 (2015) 262–274

Contents lists available at ScienceDirect

Measurement journal homepage: www.elsevier.com/locate/measurement

New preceding vehicle tracking algorithm based on optimal unbiased finite memory filter In Hwan Choi a, Jung Min Pak a, Choon Ki Ahn a, Young Hak Mo a, Myo Taeg Lim a,⇑, Moon Kyou Song b a b

School of Electrical Engineering, Korea University, 145, Anam-ro, Seongbuk-gu, Seoul 136-701, Republic of Korea Department of Electronics Convergence Engineering, Wonkwang University, 344-2, Shinyong-dong, Iksan 570-749, Republic of Korea

a r t i c l e

i n f o

Article history: Received 8 December 2014 Received in revised form 18 March 2015 Accepted 20 April 2015 Available online 27 April 2015 Keywords: Preceding vehicle tracking Optimal unbiased finite memory filter (OUFMF) Finite memory tracker (FMT) Finite measurements

a b s t r a c t In recent years, visual object tracking technologies have been used to track preceding vehicles in advanced driver assistance systems (ADASs). The accurate positioning of preceding vehicles in camera images allows drivers to avoid collisions with the preceding vehicle. Tracking systems typically take advantage of state estimators, such as the Kalman filter (KF) and the particle filter (PF), in order to suppress noises in measurements. In particular, the KF is popular in visual object tracking, because of its computational efficiency. However, the visual tracker based on the KF, referred to as the Kalman tracker (KT), has the drawback that its performance can decrease due to modeling and computational errors. To overcome this drawback, we propose a novel visual tracker based on the optimal unbiased finite memory filter (OUFMF) in the formulation of a linear matrix inequality (LMI) and a linear matrix equality (LME). We call the proposed visual tracker the finite memory tracker (FMT), and it is applied to the preceding vehicle tracking. Through extensive experiments, we demonstrate the FMT’s performance that is superior to that of the KT and other filter-based tracker. Ó 2015 Published by Elsevier Ltd.

1. Introduction Preceding vehicle tracking refers to detecting and tracking positions of a preceding vehicle in camera images. Vehicle tracking technologies have been studied for use in unmanned ground vehicles (UGVs) and advanced driver assistance systems (ADASs) [1,2]. Tracking the positions of preceding vehicles allow drivers to avoid collisions with preceding vehicles. Vision-based vehicle tracking algorithms typically consist of a detection algorithm and a filtering algorithm. The detection algorithm plays the role of a measurement system that provides information on ⇑ Corresponding author. E-mail addresses: [email protected] (I.H. Choi), [email protected] (J.M. Pak), [email protected] (C.K. Ahn), [email protected] (Y.H. Mo), [email protected] (M.T. Lim), [email protected] (M.K. Song). http://dx.doi.org/10.1016/j.measurement.2015.04.015 0263-2241/Ó 2015 Published by Elsevier Ltd.

the position of the preceding vehicle. AdaBoost [3,4] is a popularly used detection algorithm. However, the measurements given by the detection algorithm are not accurate and include significant noise. Thus, the filtering algorithm is used to suppress the measurement noises and provides more accurate position information [5–8]. The most renowned filtering algorithm is the Kalman filter (KF), which is an optimal filter that has been widely in the field of visual object tracking [9–14]. The visual object tracking algorithm based on the KF is called the Kalman tracker (KT). The KT has many advantages, such as computational efficiency and ease of use [15,16]. Due to these advantages, the KT is a popular algorithm for designers of visual tracking systems and has been developed into many advanced algorithms, such as the pixel-based algorithm [17], unscented KT [18], and adaptive KT [19]. However, the KT has disadvantages as well.

263

I.H. Choi et al. / Measurement 73 (2015) 262–274

Since the KT uses all past measurements, errors included in the measurements are accumulated over time. This error accumulation can result in performance degradation or failures. In addition, the KT often exhibits poor performance when the tracked object abruptly changes its motion. This low tracking speed issue also originates from the fact that the KT uses all of past measurements. Finite impulse response (FIR) filters [20–29] have been developed to overcome the KF’s disadvantages. By using only recent finite measurements, the FIR filters overcome the problems of error accumulation and low tracking speed. The FIR filter has several advantages over the KF. First, the FIR filter has built-in bounded-input bounded-output (BIBO) stability. Second, the FIR filter has inherent robustness against modeling and computational errors. Third, the tracking speed of the FIR filter is usually faster than that of the KF. Finally, the FIR filter does not require the process of initialization. However, in spite of these advantages, there have been no studies on employing the FIR filter for the vehicle tracking. This situation has motivated us to study vehicle tracking using the FIR filter. In this paper, we propose a novel vehicle tracking algorithm based on an FIR filter in order to overcome the drawbacks of the KT. We derive a new FIR filter named the optimal unbiased finite memory filter (OUFMF). The optimal unbiased gain of the OUFMF is computed by solving an optimization problem based on the linear matrix inequality (LMI) and linear matrix equality (LME). The combination of the OUFMF and the AdaBoost algorithm results in a new tracking algorithm, named the finite memory tracker (FMT). In the FMT, the AdaBoost algorithm provides the measurements (i.e., the measured position of the preceding vehicle in the images). The OUFMF suppresses the noise included in the measurements and improves the tracking accuracy. The improved performance of the proposed FMT is demonstrated in a comparison with the KT. In addition, we consider well-known filter-based trackers, such as the particle filter-based tracker (PT) and H1 filter-based tracker (HT). We implement the FMT, the KT, the HT, and the PT using C++/MATLAB and conduct experiments on preceding vehicle tracking. In the winding road environment where the KT’s performance degrades, the proposed FMT ensures superior tracking performance compared to the KT, the HT, and the PT. The rest of this paper is organized as follows. In Section 2, the FMT made by combining the AdaBoost and the OUFMF is proposed. In Section 3, the experimental results of preceding vehicle tracking in outdoor road environments are presented to demonstrate the performance of the FMT. Finally, conclusions drawn from this paper and future work are presented in Section 4.

2. Finite memory tracker In this section, we propose the FMT for use in preceding vehicle tracking. The FMT is made by combining the AdaBoost algorithm, modified for adapting preceding vehicle detection, with the OUFMF. Fig. 1 shows the block

diagram of the preceding vehicle tracking system based on the FMT. The image sequences are processed with the modified AdaBoost algorithm [3,4], which detects the preceding vehicle in the images. The positions of the detected preceding vehicle in the 2D image plane are used as the measurements in the FMT. The ‘‘measured’’ positions of the preceding vehicle are typically inaccurate, because the visual detection algorithms, including preceding vehicle detector, are influenced by illumination conditions. In other words, the measurements (i.e., the outputs of the detection algorithms) include noises. To suppress these noises, the FMT adapts the OUFMF. Through the OUFMF, the measured positions are converted into the estimated positions, which allows us to obtain more accurate positions for the preceding vehicles. Further details of the proposed FMT will be explained in the following subsections. 2.1. Detecting a preceding vehicle using the AdaBoost

Algorithm 1. AdaBoost with modified features 1: Given : ðx1 ; y1 Þ; . . . ; ðxm ; ym Þ where xi 2 H; yi 2 f1; þ1g. 2: Using HOG features and three newly proposed features symbolizing preceding vehicle 3: Initialize: D1 ðiÞ ¼ 1=m for i ¼ 1; . . . ; m. 4: for t ¼ 1; . . . ; T do 5: Train weak learner using distribution Dt . 6: Obtain weak hypothesis ht : H ! f1; þ1g. 7: Aim: select ht : with low weighted error.

et ¼

N X f ðwti ; yi – ht ðxi ÞÞ: i¼1

8: Choose at 1 2

at ¼ ln

1 et

et

:

9: Update, for i ¼ 1;...;m: . Z is a normalization factor.

Dtþ1 ðiÞ ¼

Dt ðiÞ expðat yi ht ðxi ÞÞ Z

10: end for 11: return Output the final hypothesis:

HðxÞ ¼ sgn

T X

!

at ht ðxÞ :

t¼1

In the proposed FMT, the AdaBoost [3,4] is modified slightly for use in detecting a vehicle in consecutive image sequences. The original version of AdaBoost constructs a composite classifier by sequentially training each weak classifier while putting more and more emphasis on certain

264


associated with sample xi . On each round, t ¼ 1; . . . ; T, a distribution, Dt, is computed over the m training examples, and a given weak learner or weak learning algorithm is applied to find a weak hypothesis, ht : H ! f1; þ1g, where the aim of the weak learner is to find a weak hypothesis with low weighted error, et , relative to Dt . The final or combined hypothesis, H, computes the sign of a weighted combination of weak hypotheses.

FðxÞ ¼

T X

at ht ðxÞ:

ð1Þ

t¼1

This is equivalent to saying that H is computed as a weighted majority vote of the weak hypotheses, ht , where each is assigned the weight at . For the front vehicle detection, xi is an image sub-window of a fixed size (50 50 in our system) containing an instance of the vehicle ðyi ¼ 1Þ or non-face ðyi ¼ þ1Þ pattern. Lastly, the M weak classifiers constitute one stronger classifier in the form of a linear combination. In the AdaBoost, each example of xi is associated with a weight of wi , and the weights are updated iteratively using a multiplicative rule according to the errors in the previous learning, so that greater emphasis is placed on those examples that were erroneously classified by the weak classifiers learned previously. Greater weights are given to the weak learners with lower errors. The whole process of AdaBoost is given in Algorithm 1.

Fig. 1. Block diagram of proposed preceding vehicle tracking system.

patterns. It employs Haar features to analyze sample images, which have two types of features: (1) Edge features and (2) line features. To detect a vehicle in a complex scene accurately, Haar features are insufficient owing to the ambiguity of the detected target object. Thus, we consider combining the histogram of gradient (HOG) features and three more features we proposed, which is made by simplifying appearances of preceding vehicles according to a process as shown in Fig. 2(a). To acquire new features suitable for preceding vehicle detection, we collect lots of backside appearance partial images –700 or more images- and then, make an average image using collected images. To emphasis the characteristics of the averaged image, morphology methods and contrasting methods is used and then, to make the final version of simplified feature, we apply magnification and simplification processes to the emphasized image. Two more rotated versions of the proposed feature are shown in Fig. 2(b). The degree of rotation is ±10°, which is determined via averaging and approximating the lots of collected rotation degree values. For the two-class classification problem, assume that a set of m-labeled training image samples are ðx1 ; y1 Þ; . . . ; ðxm ; ym Þ, where yi 2 f1; þ1g is the class label

Remark 2.1. The AdaBoost detector used in the this paper is trained using 700 or more various vehicle backside images for the positive images and 3000 images for the negative images. Thus, the FMT can track the trained object (i.e., the backside of the vehicles). However, the range of cases in which the FMT can track the new vehicle is limited because of the vehicle’s aspect. In a general road environment, the image of the new appearance of vehicles passing ahead of the observing vehicle shows the backside of the new vehicle. In that case, the FMT can track new vehicles that newly appear from diverse directions as shown in Fig. 3(a) and (b). The results in Fig. 3 employ videos from Youtube. However, a vehicle driving from left to right or from right to left cannot be detected, as shown in Fig. 3(c). Using two or more AdaBoost trained to detect different target can be a solution to this problem. This is an important issue in the visual tracking research field and will be our next research topic.

Remark 2.2. General object detection mechanisms are categorized into (1) point detectors, (2) segmentation, (3) background subtraction, and (4) supervised classifiers. In the outdoor environment used in this paper as experimental situations, lots of objects, patterns, and features belong to the scene. In this situation, firstly, point detectors cannot be proper options as an object detection part of the FMT because outdoor scenes contain lots of useless point and edge features (i.e., features unrelated to preceding vehicle region). Secondly, The segmentation mechanism is not a good method for preceding vehicle detection owing to its low robustness against image transformation such as large illumination change and partial occlusion by shadows.


265

Fig. 2. New features for preceding vehicle detection of the AdaBoost: (a) the whole process of generating new features representing preceding vehicle appearance, (b) three new features employed in the AdaBoost training.

Fig. 3. Detection result of new vehicle appearance: (a) from the right direction, (b) from the left direction, and (c) detection failure.

266


Thirdly, background subtraction cannot work with consecutive images with moving background. In contrast, supervised classifiers learn characteristics of target object in advance. AdaBoost [3,4] is one of the best known and most popular supervised classifiers. AdaBoost is a powerful method for detecting the target object in the kind of complex image used in the experiments of this paper because AdaBoost is adaptive in the sense that subsequent weak learners are tweaked in favor of those instances misclassified by previous classifiers. Due to this property, AdaBoost has advantages as follow: (1) AdaBoost has a low error rate and low computational complexity, (2) AdaBoost is an optimal classifier, (3) AdaBoost is easy to implement. These are the reason why we have chosen the AdaBoost as a object detector of proposed FMF. Additionally, various researchers have been using AdaBoost, according to a recent survey paper [32].

2.2. Optimal unbiased finite memory filter (OUFMF) The positions of the detected vehicles in a 2D image plane can be denoted by x and y. The velocities along the _ respectively. x-axis and the y-axis are denoted by x_ and y, The state vectors at the discrete time, k, are then defined T as sk ¼ ½xk x_ k yk y_ k . The measurement is the detected position such that zk ¼ ½xk yk T . Using the definitions of the state and measurement vectors, the system and measurement models are described as follows:

skþ1 ¼ Ask þ wk ; zk ¼ Csk þ v k ;

ð2Þ

where A and C are the system and the measurement matrices, respectively, which are defined as:

2

1 T

0

0

3

60 1 0 07 7 6 A¼6 7; 40 0 1 T 5 0

0

0

C¼

1 0 0 0 0 0 1 0

;

ð3Þ

1 0 60 0 6 6 61 T 6 60 0 6 O¼6 6 1 2T 6 60 0 6 6 4 1 3T 0 0

3

0

0

1

0 7 7 7 0 7 7 T 7 7 7: 0 7 7 2T 7 7 7 0 5

0 1 0 1

e N skN þ G e N W k1 þ V k1 ; Z k1 ¼ C

ð5Þ

where T

Z k1

, ½zTkN zTkNþ1 . . . zTk1 ;

W k1

, ½wTkN wTkNþ1 . . . wTk1 ;

T

ð6Þ

T

, ½v TkN v TkNþ1 . . . v Tk1 ;

V k1

e N and G e N are obtained as follows: and C

3 CAN 6 Nþ1 7 " e # 7 6 CA eN ¼ 6 7 ¼ C i1 A1 ; C .. 7 6 C 5 4 . 2

ð7Þ

CA1 3 2 1 CA G CA2 G CAN G 7 6 6 0 CA1 G CANþ1 G 7 7 6 Nþ2 6 e N ¼ 6 0 0 CA G7 G 7 7 6 . .. .. .. 7 6 . . . . 5 4 . " ¼

e i1 G 0

CA1 G

0 0 # e i1 A1 G C : CA1 G

ð8Þ

e N W k1 þ V k1 is a multi-variable Gaussian In (5), the term G noise with the covariance matrix PN , which is given by:

e N ½diagðQ ; . . . ; Q Þ G e T þ ½diagðR; . . . ; RÞ: PN , G N

ð9Þ

In Eq. (9), diagðÞ refers to a block-diagonal matrix with the indicated matrices on its main diagonal. The OUFMF can be expressed as a product of the filter gain, H, and the finite measurements, Z k1 , as:

^sk ¼ HZ k1 :

ð10Þ

Substituting (5) into (10) gives:

1

where T is the sampling time. The system matrix A has a nonzero determinant value for any sampling time T > 0. Thus, A is invertible, and then, the system matrix A is nonsingular. To check observability, the observability matrix O is calculated as:

2

The recent finite measurements on the horizon ½k N; k 1, where N is the horizon size, can be expressed as a function of skN as follows:

n o e N sk þ G e N W k1 þ V k1 : ^sk ¼ H C

ð11Þ

By taking the expectations of both sides of (11), we obtain:

e N E½sk ; E½^sk ¼ H C

ð12Þ

where E½ represents the expectation operator. To satisfy the unbiased condition (i.e., E½^sk ¼ E½sk ), the following LME constraint should be satisfied:

ð4Þ

0 1 3T

For any T, the observability matrix O has full rank. Thus, the observability is ensured. We assume that the process noise, wk , and the measurement noise, v k , are zero-mean white Gaussian and have the covariance matrices Q and R, respectively.

e N ¼ I: HC

ð13Þ

Substituting (13) into (11) yields

e N W k1 þ HV k1 : ^sk ¼ sk þ H G

ð14Þ

Thus, the estimation error can be represented by

e N W k1 þ HV k1 : ek ,^sk sk ¼ H G

ð15Þ

The optimal unbiased gain matrix H is obtained by minimizing the E½eTk ek subject to the unbiasedness constraint eNQ G e T HT þ HRN HT , (13). Since E½eT ek ¼ trE½ek eT ¼ tr½H G k

k

N

N

267


Introduce the following inequality: 160

T e NQ G eT Hð G N N þ RN ÞH < cI:

ð17Þ

140

RMSE (pixel)

120

Algorithm 2. Optimal unbiased finite memory filter (OUFMF)

100 80

1: Data: A; C; Q ; R; N 2: Result: ^sk 3: Compute the gain matrix H by solving the following optimization problem:

60 40 20

0

5

10

15

20

25

30

min c H;c

Horizon Size

subject to

Fig. 4. RMSE of FMT over various horizon sizes from 0 to 30.

"

where Q N ¼ ½diagðQ ; . . . ; Q Þ and RN ¼ ½diagðR; . . . ; RÞ and tr½ is a matrix trace operator, the sum of the main diagonal of a square matrix, the optimal unbiased H can be obtained eNQ G e T HT þ HRN HT , by minimizing the cost function, tr½H G N

N

consider the following inequality: T eNQ G eT tr½Hð G N N þ RN ÞH < c:

HT

#

H eNQ G eT ðG N N þ RN Þ

1

> 0;

e N ¼ I: HC

N

e N ¼ I. We employ the LMI and LME approaches subject to H C to solve this optimization problem. For a positive scalar eNQ G e T HT þ HRN HT , c > 0, based on the cost function tr½H G N

cI

4: for k ¼ 1; 2; . . . do T

Z k1 ¼ ½zTkN zTkNþ1 . . . zTk1 . 6: ^sk ¼ HZ k1 . 7: end for 5:

ð16Þ

Fig. 5. Resulting images obtained from preceding vehicle tracking under clear weather condition.

268


By using the Schur complement [30], inequality (17) is found to be equal to:

"

cI HT

#

H eNQ G eT ðG N N þ RN Þ

1

> 0:

ð18Þ

C++/MATLAB, and the speculation of the system is configured to Intel dual-core i5 2.3 GHz CPU, 8 GB memory, and Window 7 operating system. To demonstrate performance of the FMT, we consider the KT, the HT, and the

e N ; Q , and R are fixed when the system matrices are given G N eNQ G e T þ RN is a and the horizon size is fixed. Note that G positive definite matrix; and thus, its inverse matrix always exist. In (18), c and H are the LMI variables. Based on this argument, the optimal unbiased gain matrix, H, can be determined by solving the following constrained minimization problem:

min c; H;c

400

ð19Þ

subject to the LME (13) and the LMI (18). The constrained minimization problem can be easily solved by using existing numerical algorithms, such as the Scilab LMI Toolbox [31]. The algorithm of the OUFMF found via the LMI and LME approach is summarized in Algorithm 2.

Remark 2.4. In this paper, gain matrix H satisfying LMI and LME constraints using existing numerical algorithms, such as Scilab LMI Toolbox [31], is determined by using offline computation before the tracking system starts because the calculation process leads to heavy computational cost. First of all, to start the optimization process, horizon size N should have been decided. The optimal horizon size N is determined in advance and is fixed in the experiments. As shown in Fig. 4, horizon size 5 brings the best performance among the 27 different horizon size cases.

3. Experiments

FMT KT HT PT

300 250 200 150 100 50 0

260

280

300

320

340

360

380

400

420

440

Frame Number (k) 400

(b)

Tracking Error (pixel)

350

FMT KT HT PT

300 250 200 150 100 50 0

480

500

520

540

560

580

600

620

640

660

680



350

(c)

FMT KT HT PT

300 250 200 150 100 50 0

700

720

740

760

780

800

820

840

860

880

Frame Number (k) 400 350


Remark 2.3. The FIR structure is the most significant feature of OUFMF compared with existing IIR filters-based tracker such as the KT, HT, and PT. The FMT has the FIR structure, which means that it uses recent finite measurements for generating state estimates. Owing to the FIR structure of OUFMF, the FMT can prevent the accumulation of modeling and computational errors. Thus, the FMT is robust against modeling uncertainties, incorrect noise information, linearization errors, round-off errors, and so on. However, all existing IIR filters-based trackers (KT, HT, and PT) use all past measurements to generate state estimates. Thus, computational and modeling errors are accumulated over time, and this may lead to filter divergence. The KT and HT require initial values of the state estimate and the estimation error covariance. However, the FMT does not require initial information. Thus, the FMT is more robust against incorrect initial information than the KT, HT, and PT.

(a)

350

N


N

(d)

FMT KT HT PT

300 250 200 150 100 50 0

3.1. Experimental settings To test the proposed FMT, we conducted experiments on preceding vehicle tracking. For the experiments, the FMT, the KT, the HT, and the PT were implemented using

900

920

940

960

980 1000 1020 1040 1060 1080 1100 1120

Frame Number (k) Fig. 6. Tracking errors of experiment under clear weather conditions: (a) Frames 253–450, (b) Frames 470–680, (c) Frames 681–895, and (d) Frames 900–1120.


PT as the subjects of comparison. The noise covariance matrices for the FMT, the KT, the HT, and the PT were taken as

Q ¼ 0:1I4 ;

R ¼ 0:9I2 ;

ð20Þ

where I4 and I2 are 4 4 and 2 2 identity matrices, respectively. In representing the visual tracking system model as a state space model, we should assume and approximate the system and measurement model and parameters into Gaussian probability density function (PDF) as in [32] so that the process noise and the measurement noise are assumed to be independent of each other in advance. In our system, the proposed filtering method employs vision measurement information that contains noises and uncertainties, which is generated by capturing

Table 1 Average tracking errors (in units of pixels) of FMT, KT, HIT, and PT under clear weather conditions. Frames

FMT

KT

HIT

PT

253–450 470–680 681–895 900–1120

21.56 19.24 15.73 17.95

49.09 53.21 37.78 45.20

24.71 24.45 20.87 24.76

483.7 412.5 488.8 354.8

Total

18.62

46.32

23.70

434.9

269

process of video cameras. Thus, the process and measurement noise covariances have been experimentally determined for the FMT to achieve optimal performance in the experiments. The minimum horizon size we can employ in this system is N ¼ 4. However, we set the horizon size used for the FMT as N ¼ 5 in this paper. One of the most significant issues in the OUFMF method is to manage the horizon size (also known as the memory size) because it is an important parameter for the OUFMF’s estimation performance. Shmaliy et al. [35–38] developed various methods to find the optimal constant horizon size. These methods are very effective in linear time-invariant systems. To acquire optimal horizon size N, we have tested various horizon sizes under MATLAB environment according to Shmaliy’s method [35–38]. As shown in Fig. 4, when we varied horizon size from 4 to 30, the root mean square error (RMSE) was lowest at horizon size 5. Thus, we have employed horizon size 5 to achieve optimal performance of the tracking method. Remark 3.1. In discrete-time system we used horizon size N ¼ 4. The bigger the horizon size we use, the more measurements are accumulated to the system, which is similar to the infinite impulse response (IIR) filter structure. Thus, as shown in Fig. 4, we can select the proper horizon size N ¼ 5, which is optimal for the system.

Fig. 7. Resulting images obtained from preceding vehicle tracking under cloudy weather condition.


The sampling time, T, was 0.033 s. In this preceding vehicle tracking system, we recorded videos using a black box camera installed in the car. The frame rate of the camera we used is 30 frames per seconds (FPS). Thus, we employ 0.033 s as the sampling time in the experiments. With these design parameters, we obtained the large e 4 and G e 4 , as follows: matrices of the OUFMF, C

2

K4

3

K2 ¼ K3 ¼

3

K1

K2

K3

K4

6 0 e4 ¼ 6 G 6 4 0

K1

K2

0

K1

K3 7 7 7; K2 5

K1 0 1 0:0330 0

0 0

0

K1

6 K3 7 7 e4 ¼ 6 C 7; 6 4 K2 5 K1 ¼

2

0 0 1 0:0330 1 0:0660 0 0

0 0 1 0:0660 1 0:0990 0 0

ð21Þ

Ek ¼

;

ð22Þ

400 350

With the help of the Scilab LMI Toolbox [31], the optimal unbiased gain matrix, H, of the OUFMF was then computed as: 2

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 ^k Þ2 ; ðxk ^xk Þ þ ðyk y

ð24Þ

where ½xk ; yk are the ground true positions (GTP) of the ^k are the xk ; y preceding vehicle in the k-th image, and ½^ tracked positions estimated by the trackers. The GTP of the preceding vehicle used in the experiments has been acquired manually frame-by-frame. We have chosen this method to improve confidence in the experimental results,

;

; 0 0 1 0:0990 1 0:1320 0 0 : K4 ¼ 0 0 1 0:1320

the preceding vehicle. This low illumination condition was effective as an extremely bad case for tracking a preceding vehicle. In other words, the tracking system became less efficient at estimating the preceding vehicle position, because both low-contrast levels and the complex shadows of trees acted as noise in the scene, which was similar to grains in the film. To evaluate the performance of the preceding vehicle trackers, such as the FMT, the KT, the HT, and the PT, we defined the tracking error as follows:

3

0:4870 0 0:0075 0 0:4760 0 1:0185 0 6 7 6 7 6 7 6 9:1226 0 2:9333 0 2:9314 0 9:1245 0 7 6 7 6 7: H¼6 7 6 0 0:4870 0 0:0075 0 0:4760 0 1:0185 7 6 7 6 7 4 5 0 9:1226 0 2:9333 0 2:9314 0 9:1245


270

(a)

FMT KT HT PT

300 250 200 150 100 50 0

60

80

100

400


350

140

160

180

200

220

(b)

FMT KT HT PT

300 250 200 150 100 50 0

240

260

280

300

320

340

360

Frame Number (k) 400 350


Videos filmed in the three different road environments (in different places and different illumination conditions) were used for the experiments. In general, the preceding vehicle tracking is more difficult on a winding road than on a straight road. Thus, to demonstrate the superior performance of the proposed FMT over the KT, the HT, and the PT, we filmed the videos on a winding roads. The MEAN-shift [33] or the CAM-shift [34] can be used for the tracker performance comparison as fully vision-based object trackers. However, the MEAN-shift and the CAM-shift algorithms exhibited poor performance in tracking using videos containing heavily wooded winding roads and they were excluded from the comparison owing to their inherent weaknesses of low robustness against rapid image transformation in illumination and color. Moreover, the three videos contained rear-view scenes of the preceding vehicle under three different illumination conditions: (1) in clear weather, (2) in cloudy weather, and (3) at nighttime. On the tree-lined road, the clear weather made the shadows of the trees more complex and dark. In these conditions, the shadows can act as disturbances or measurement noises in the tracking system. When the weather was cloudy, the average brightness in the images was lower than in clear weather. Thus, the scene in the cloudy weather reflected degraded contrast level over the video sequence. At last, in the nighttime, there was only moonlight and the headlights of the following vehicle, which were not sufficient to recognize objects including

120

Frame Number (k)

ð23Þ

(c)

FMT KT HT PT

300 250 200 150 100 50 0

400

450

500

550

600

Frame Number (k) Fig. 8. Tracking errors of experiment under cloudy weather condition: (a) Frames 45 to 220, (b) Frames 221 to 370, and (c) Frames 371 to 620.


because human recognition ability should be the first criteria for judging the reference position, referred to as the GTP. Thus, the GTP information acquired frame-by-frame is independent of the detected position information of the preceding vehicle using the AdaBoost. 3.2. Experimental results 1: preceding vehicle tracking in clear weather We firstly tested the proposed FMT under the clear weather conditions. Fig. 5 shows the estimated positions and the true positions of the preceding vehicle. The yellow square indicates the true position, which is manually acquired. The blue, the red, the green, and the black square indicate the estimated positions obtained using the FMT, Table 2 Average tracking errors (in units of pixels) of FMT, KT, HIT, and PT in the nighttime. Frames

FMT

KT

HIT

PT

4–170 171–370 371–500

12.95 12.19 13.71

37.52 29.30 34.60

20.52 17.50 20.26

497.6 279.7 421.0

Total

12.95

33.81

19.43

421.0

271

the KT, the HT, and the PT, respectively. In Fig. 5, there are strong shadows near the preceding vehicle, which makes the detection with AdaBoost inaccurate. The comparison of the tracking performances of the FMT with others is shown in Fig. 6. We divided the whole footage into four sections: (1) the frames from 253 to 450, (2) The frames from 470 to 680, (3) the frames from 681 to 895, and (4) the frames from 900 to 1120. In each section, we compared the tracking errors of the FMT and the other filter-based trackers. As shown in Fig. 6, the tracking errors are smaller in the FMT than in the others. In this experiment, we set the initial position of the preceding vehicle to [0, 0], so that we can see that the PT shows the sample impoverishment problem. After resampling, most particles will have very similar, or even identical states, leading to a problem called sample impoverishment. Particle degeneracy and sample impoverishment are similar problems that manifest as unbalance between the need for diversity and for concentration of particles. In several frames, the tracking errors of the other trackers are close to zero; however, this does not mean the tracking was accurate. In the winding road scene, the estimated position intersects with the true position, because the vehicle changes its direction from left to right or vice versa. Thus, the points that are close to zero are not actually zero errors.

Fig. 9. Resulting images obtained from preceding vehicle tracking in the nighttime.

272


Table 1 compares the average tracking errors of the FMT with those of the others. The average tracking errors are much smaller in the FMT than in the others. Therefore, the superior performance of the FMT over that of the others under clear weather conditions is verified. 200

FMT KT HT PT


(a) 150

100

50

0

30

40

50

60

70

80

90



120

FMT KT HT PT

(b)

100 80 60 40 20 0

90

100

110

120

130

140

150

160


FMT KT HT PT


(c) 150

100

50

0 160

170

180

190

200

210

220

230

240


FMT KT HT PT


(d) 150

3.3. Experimental results 2: preceding vehicle tracking in cloudy weather For the second experiment, we conducted preceding vehicle tracking in cloudy weather conditions. Figs. 7 and 8 show the resulting images and the tracking errors obtained by performing the preceding vehicle tracking. In Fig. 8, the other trackers exhibit several zero crossing points; however, these points do not represent actual zero error. The zero crossing points in the others’ tracking errors were caused by change of the preceding vehicles’ courses. The FMT exhibits smaller tracking errors than the others. The PF exhibits the sample impoverishment problem in this case as well. Table 2 shows the average tracking errors of the FMT compared with those of the others. The FMT exhibits smaller average errors than the others not only in each section of the frames, but also in the entirely of frames. Thus, the superior performance of the FMT over that of the others under the cloudy weather conditions is verified.

3.4. Experimental results 3: preceding vehicle tracking in the nighttime For the third experiment, we conducted the preceding vehicle tracking in the nighttime. Figs. 9 and 10 show the resulting images and the tracking errors obtained by performing the preceding vehicle tracking. As shown in Fig. 9, each frame under the night conditions contains extensive dark regions, which means that these frames are short of information; thus, this condition can make the tracker’s performance degraded. In Fig. 10, error changes more diversely over frames than in the other weather cases, which means that the result of the detection process are degraded due to insufficient vision information. Nonetheless, the performance error of the FMT is relatively lower than that of others. This constrained driving condition also shows that the FMT’s tracking performance is superior to those of the others. In extremely bad conditions such dark nights without sufficient illumination, in spite of difficulty of detection, the proposed FMT estimates the position of the preceding vehicle better than trackers using HF, PF, or KF. Table 3 shows that the FMT exhibits the smallest average errors among the four trackers. In the night tracking case, the superior performance of the FMT over those of the others is also verified. Table 4 compares the average computation time of the FMT and those of the others. The PT also shows the sample impoverishment problem

100

Table 3 Average tracking errors (in units of pixels) of FMT, KT, HIT, and PT under cloudy weather conditions.

50

0

Frames 240

260

280

300

320

340

Frame Number (k) Fig. 10. Tracking errors of experiment in the nighttime: (a) Frames from 29 to 96, (b) Frames from 90 to 150, (c) Frames from 160 to 240, and (d) Frames from 230 to 354.

Frames Frames Frames Frames Total

29–96 90–150 160–240 230–354

FMT

KT

HIT

PT

21.62 13.78 37.00 23.75

65.04 32.12 79.26 54.32

35.38 61.56 78.41 58.94

36.77 6.768 66.44 17.84

24.04

57.68

58.57

31.81

273

I.H. Choi et al. / Measurement 73 (2015) 262–274 Table 4 Average computation time comparison of FMT, KT, HT, and PT under each weather condition, unit(s). Weather condition

FMT

KT

Clear

3:084 10

Cloudy

3:101 105 5

At night

3:098 10

5

but not in every section; the PT does not exhibit the sample impoverishment problem in a couple of sections. Fig. 10 exhibits those few sections. However, the average performance of the PT is inferior to that of the FMT, as shown in Fig. 10. Table 4 shows the calculation time for each filtering method. The FMT shows the lowest calculation time among the four trackers as the computation complexity of the FMT can be reduced via off-line LMI and LME optimization. The computation time of the AdaBoost part is excepted because the AdaBoost part is a shared part of each tracker.

4. Conclusion This paper proposed a novel visual tracker, named the FMT, for use in preceding vehicle tracking. The FMT exhibited a superior performance over that of the existing filter-based vehicle trackers in the winding road environment under three different illumination conditions. In particular, the FMT showed a superior tracking speed over that of the others when the vehicle abruptly changed its course. Additionally, the FMT exhibited fairly low computation complexity compared to the tracker using particle filter. In the near future, we plan to apply the FMT to the visual tracking of other objects in the road, such as traffic signs and pedestrians. Acknowledgement This work was supported in part by the Energy Efficiency & Resources Core Technology Program of the Korea Institute of Energy Technology Evaluation and Planning (KETEP), granted financial resource from the Ministry of Trade, Industry & Energy, Republic of Korea (No. 20142010102390), in part by the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (NRF-2014R1A1A1006101), in part by General Research Program through National Research Foundation of Korea funded by the Ministry of Education (Grant No. NRF-2013R1A1A2008698), and in part by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2013R1A1A2060663). References [1] G. Chunzhao, S. Mita, D. McAllester, Robust road detection and tracking in challenging scenarios based on Markov random fields with unsupervised learning, IEEE Trans. Intell. Transp. Syst. 13 (3) (2012) 1338–1354.

HIT 4

PT

9:464 10

4

1:500 10

1:830 102

9:474 104

1:520 104

1:850 102

4

4

1:840 102

9:444 10

1:490 10

[2] A. Mammeri, A. Boukerche, M. Almulla, Design of traffic sign detection, recognition, and transmission systems for smart vehicles, IEEE Trans. Wireless Commun. 20 (6) (2014) 36–43. [3] P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, December 2001, pp. 1511–1518. [4] P. Viola, M. Jones, Robust real-time face detection, Int. J. Comput. Vision 57 (2) (2004) 137–154. [5] M. Kamil, T. Chobtrong, E. Gunes, M. Haid, Low-cost object tracking with MEMS sensors, Kalman filtering and simplified two-filtersmoothing, Appl. Math. Comput. 235 (5) (2014) 323–331. [6] B. Tian, Y. Li, B. Li, D. Wen, Rear-view vehicle detection and tracking by combining multiple parts for complex urban surveillance, IEEE Trans. Intell. Transp. Syst. 15 (2) (2014) 597–606. [7] R. O’Malley, E. Jones, M. Glavin, Rear-lamp vehicle detection and tracking in low-exposure color video for night conditions, IEEE Trans. Intell. Transp. Syst. 11 (2) (2010) 453–462. [8] L.H. Juang, M.N. Wu, Image noise reduction using Wiener filtering with pseudo-inverse, Measurement 43 (10) (2010) 1649–1655. [9] T. Yamaguchia, H. Yamasakib, Active vision system integrated with angular velocity sensors, Measurement 15 (1) (1995) 59–68. [10] D.D. Doyle, A.L. Jennings, J.T. Black, Optical flow background estimation for real-time pan/tilt camera object tracking, Measurement 48 (2) (2014) 195–207. [11] Z. Fu, Y. Han, Centroid weighted Kalman filter for visual object tracking, Measurement 45 (5) (2012) 650–665. [12] V. Govinda, J.F. Ralph, J.W. Spencer, J.Y. Goulermas, Small object monitoring for sensitive indoor compounds, Measurement 42 (7) (2009) 815–829. [13] J. Sladek, K. Ostrowska, P. Kohut, K. Holak, A. Gaska, T. Uhl, Development of a vision based deflection measurement system and its accuracy assessment, Measurement 46 (3) (2013) 1237– 1249. [14] D.U. Guanglong, P. Zhang, Human–manipulator interface using hybrid sensors with Kalman filters and adaptive multi-space transformation, Measurement 55 (9) (2014) 413–422. [15] P. Li, T. Zhang, B. Ma, Unscented Kalman filter for visual curve tracking, Image Vision Comput. 22 (2) (2004) 157–164. [16] S.K. Wenga, C.M. Kuob, S.K. Tub, Video object tacking using adaptive Kalman filter, J. Visual Commun. Image Represent. 17 (6) (2006) 1196–1197. [17] L.H. Matthies, R. Szeliski, T. Kanade, Kalman filter-based algorithms for estimating depth from image sequences, Comput. Vision 3 (3) (1989) 209–236. [18] P. Li, T. Zhang, B. Ma, Unscented Kalman filter for visual curve tracking, Image Vision Comput. 22 (2) (2004) 157–164. [19] S.K. Wenga, C.M. Kuob, S.K. Tub, Video object tracking using adaptive Kalman filter, J. Visual Commun. Image Represent. 17 (6) (2006) 1190–1208. [20] C.K. Ahn, S.H. Han, W.H. Kwon, H1 FIR filters for linear continuoustime state-space systems, IEEE Signal Process. Lett. 13 (9) (2006) 557–560. [21] C.K. Ahn, S.H. Han, W.H. Kwon, H1 finite memory controls for linear discrete-time state-space models, IEEE Trans. Circuit Syst. II 54 (2) (2007) 97–101. [22] C.K. Ahn, Robustness bound for receding horizon finite memory control: Lyapunov–Krasovskii approach, Int. J. Control 85 (7) (2012) 942–949. [23] C.K. Ahn, Strictly passive FIR filtering for state-space models with external disturbance, Int. J. Electron. Commun. 66 (11) (2012) 944– 948. [24] C.K. Ahn, A New solution to induced l1 finite impulse response filtering problem based on two matrix inequalities, Int. J. Control 87 (2) (2014) 404–409. [25] H.D. Choi, C.K. Ahn, M.T. Lim, Time-domain filtering for estimation of linear systems with colored noises using recent finite measurements, Measurement 46 (8) (2013) 2792–2797.

274


[26] J.M. Pak, C.K. Ahn, M.T. Lim, M.K. Song, Horizon group shift FIR filter: alternative nonlinear filter using finite recent measurements, Measurement 57 (2014) 33–45. [27] Y.S. Shmaliy, Unbiased FIR filtering of discrete-time polynomial state-space models, IEEE Trans. Signal Process. 57 (4) (2009) 1241– 1249. [28] Y.S. Shmaliy, Linear optimal FIR estimation of discrete timeinvariant state-space models, IEEE Trans. Signal Process. 58 (6) (2010) 3086–3096. [29] P.S. Kim, An alternative FIR filter for state estimation in discretetime systems, Digital Signal Process. 20 (3) (2010) 935–943. [30] S. Boyd, L.E. Ghaoui, E. Feron, V. Balakrishinan, Linear Matrix Inequalities in System and Control Theory, vol. 68, SIAM, Philadelphia, PA, 1994. pp. 1–8. [31] F. Delebecque, R. Nikoukhah, L. El Ghaoui, LMITOOL: A Package for LMI Optimization in Scilab – Users Guid Domaine de Voluceau, Rocquencourt, BP 105, 78153 LE CHESNAY Cedex (France), INRIA, February 1995.

[32] S.Y. Chen, Kalman filter for robot vision: a survey, IEEE Trans. Ind. Electron. 59 (11) (2012) 1797–1809. [33] C. Yizong, Mean shift, mode seeking, and clustering, IEEE Trans. Pattern Anal. Mach. Intell. 17 (8) (1995) 790–799. [34] G.R. Bradski, Computer Vision Face Tracking For Use in a Perceptual User Interface Intel Technology Journal Q2, 1998. [35] Y.S. Shmaliy, J. Munoz-Diaz, L. Arceo-Miquel, Optimal horizons for a one-parameter family of unbiased FIR filters, Digital Signal Process. 18 (5) (2008) 739–750. [36] Y.S. Shmaliy, Linear optimal FIR estimation of discrete timeinvariant state-space models, IEEE Trans. Signal Process. 58 (6) (2010) 3086–3096. [37] Y.S. Shmaliy, Suboptimal FIR filtering of nonlinear models in additive white Gaussian noise, IEEE Trans. Signal Process. 60 (10) (2012) 5519–5527. [38] F. Ramirez-Echeverria, A. Sarr, Y.S. Shmaliy, Optimal memory of discrete-time FIR filters in state-space, IEEE Trans. Signal Process. 62 (3) (2014) 557–561.