Research Article
Improved kernelized correlation filter algorithm and application in the optoelectronic tracking system
International Journal of Advanced Robotic Systems May-June 2018: 1–10 ª The Author(s) 2018 DOI: 10.1177/1729881418776582 journals.sagepub.com/home/arx
Fengfa Yue and Xingfei Li
Abstract In order to improve the tracking accuracy and real-time performance of the optoelectronic tracking system, an improved kernelized correlation filter approach is developed to obtain precise tracking of a maneuvering object. The proposed strategy contains merits of adaptive threshold approach, kernelized correlation filter method, and Kalman filter algorithm. The adaptive threshold approach can choose the suitable threshold in accordance with the size of the target in the image to improve the tracking performance of the kernelized correlation filter method. When the change between previous position and current position is larger than the distance threshold, Kalman filter algorithm is used to predict the target position for tracking. The tracking accuracy of the proposed algorithm is improved by updating the prediction of the target position with a trusted algorithm. The experimental results on comparison with some state-of-the-art trackers, such as kernelized correlation filter; Tracking-Learning-Detection; scale adaptive with multiple features; minimum output sum of squared error; and dual correlation filter, demonstrate that the proposed approach has the effectiveness of tracking accuracy and real-time performance in tracking the maneuvering object. Keywords Visual tracking, optoelectronic tracking system, adaptive threshold, kernelized correlation filters, Kalman filter Date received: 31 May 2017; accepted: 22 January 2018 Topic: Vision Systems Topic Editor: Antonio Fernandez-Caballero Associate Editor: Grazia Cicirelli
Introduction Optoelectronic tracking system has been widely used in military and civil domain in recent years.1,2 It is a comprehensive technology including optics, mechanics, electrics, automation, and sensors. As a representative of real-time monitoring fields, the optoelectronic tracking system can track the moving target and record video in order to achieve the goal of target tracking and video forensics. Image tracking algorithm of the optoelectronic tracking system has to meet the requirements of real-time performance and tracking accuracy for tracking maneuvering object. Although a variety of object tracking algorithms3–5 have been developed in computer vision,6–10 object tracking is still a critical issue, due to the presence of scale variation, illumination, occlusion, background clutter, and viewpoint.
In general, the effective appearance models can be categorized into generative or discriminative algorithms. Generative approaches consider the tracking problem as searching for image region with the most similarities to represent the target. A novel object tracking method based on adaptive sparse representation11 was designed for tracking the target. A joint model of appearance and spatial
The State Key Laboratory of Precision Measuring Technology and Instruments, Tianjin University, Tianjin, China Corresponding author: Xingfei Li, The State Key Laboratory of Precision Measuring Technology and Instruments, Tianjin University, Tianjin 300072, China. Email:
[email protected]
Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/ open-access-at-sage).
2 configuration of pixels12 was developed to estimate the local distortion of the object. The orthogonal matching pursuit strategy13 was designed for handling the optimization problems efficiently. Despite much verified success of these generative tracking methods, the drift problem remains to be settled. The generative algorithms mentioned above cannot treat major appearance changes as the track evolves. An online boosting feature selection method14 was formulated for visual tracking. However, the drift problem occurred as only one positive sample was used to update the classifier. The performance of tracking was enhanced by researching the relationship between the target and the structured environment in Zhu et al. 15 A semi-supervised learning algorithm16 was presented to choose the positive and negative samples by an online classifier with structural constraints. Babenko et al.17 presented the idea of multiple instance learning (MIL) to alleviate drift and improved the stability of tracking with fewer parameter tweaks. A novel online weighted multiple instance learning tracker that had the performance of rapidity and robustness was presented in a study.18 It could naturally integrate the sample significant into the learning process by assuming the tracking position at current frame was the position of most correct positive sample. A discriminative appearance model19 based on superpixels was introduced to deal with occlusions and recovery from drift. Tracking by detection strategies was widely utilized in computer vision in recent years. A classifier of these strategies was trained using positive samples and negative ones. By utilizing a series of structural constraints with boosting classifier, the tracking task is considered as Tracking-Learning-Detection (TLD).20 A novel online tracking method which integrated the online MIL into the recent sparse representation was proposed by Yan et al.21 Object tracking22 was considered as a binary classification issue in which the correlation of object appearance and class labels from foreground and background was modeled by partial least squares analysis. A novel discriminative, generative, and collaborative appearance model23 was presented for robust object tracking. A robust visual tracking method24 based on online learning sparse representation was designed for handling appearance variations. Recently, the methods based on correlation filters had shown advantages both in tracking accuracy and robustness. A novel extension to correlation filter25 was developed to consider the employment of multichannel signals with the efficient utilization of memory and computations. Kernelized correlation filter (KCF)26 used the histogram of oriented gradients (HOG) characteristic instead of raw pixel value and Gaussian kernels to improve the tracking accuracy. Dual correlation filter (DCF)26 adopted a fast multichannel extension of linear correlation filters using a linear kernel to improve the tracking performance of tracker. Scale adaptive with multiple features (SAMF)27 increased scale to the KCF framework by sampling the
International Journal of Advanced Robotic Systems primitive object with different scales and learning the model at each scale. Although it integrates HOG descriptor with a color-naming28 technique to improve the tracking performance, it leads to a large amount of computational cost. Minimum output sum of squared error (MOSSE) algorithm29 could be seen as a kernelized version of a linear correlation filter, which formed the basis for the fastest trackers available. In this article, an improved kernelized correlation filter (IKCF) approach is proposed to gain good tracking performance for the maneuvering target. The main contribution is that the adaptive threshold approach and Kalman filter (KF)30 are incorporated into KCF algorithm to strengthen the tracking accuracy and real-time performance of the optoelectronic tracking system. The adaptive threshold is selected by experiments to improve the tracking performance of KCF algorithm. KF algorithm is used to estimate the target position when the change in position is larger than the distance threshold. The tracking performance of the proposed algorithm is verified by experimental results on comparison with some state-of-the-art trackers such as KCF, TLD, SAMF MOSSE, and DCF for tracking maneuvering target. The article is organized as follows: The second section describes the proposed visual tracking strategy. It contains adaptive threshold approach, KCF method and KF algorithm. Then, the flow chart of the IKCF method is described. In the third section, firstly, the composition of the system is introduced. Then, the experimental results are shown to verify the validity of the proposed algorithm. The fourth section gives the conclusions and future work.
Proposed visual tracking strategy In this section, the principle of the proposed visual tracking strategy is introduced. It consists of adaptive threshold approach, KCF algorithm, and KF-based tracking with novel algorithm of position updating. Then, the flow chart of the IKCF method is presented. The details are introduced as below.
Adaptive threshold approach The traditional fixed-size search window usually selects a fixed threshold value for the whole video frames and does not consider any current information gained from the feature of object and video frame. Consequently, this strategy is more likely to fail to track object under the condition of maneuvering object. The bounding box is defined as the size of the extra area surrounding the object. The selection of its threshold value4 is critical for KCF algorithm. If it is too large, it expands the search scope of the object. Then, the running efficiency of KCF algorithm will reduce and even lead to the tracking failure. If it is too small, the object may move out of the bounding box, resulting in failure of tracking. Therefore,
Yue and Li
3
adaptive bounding box value approach is designed by studying several video sequences. This method is based on the image pixels LH WH and the object pixels lh wh, where LH and WH represent the length and width of the image, and lh and wh represent the length and width of the object. The threshold value H is described as below. 8 > < H1 if M 1 A 1 and M A H ¼ H2 else if M 1 < A 1 ð1Þ > : H3 otherwise where M 1 ¼ ðlh whÞ=ðLH WHÞ represents the ratio of the object pixels to the image pixels and M ¼ lh=wh represents the ratio of the length to width of object. The appropriate threshold is selected by adaptive threshold approach which utilizes the ratio M1 and M matching score instead of utilizing global value.
Summary of KCF method The reason why KCF algorithm has the performance of running at hundreds of frames per second (fps) is that the training samples are constructed by cyclic shifts, which makes the data matrix into cyclic matrix. According to the characteristic of the cyclic matrix, the solution of the problem is transformed to the discrete Fourier transform (DFT) domain. Therefore, it avoids the process of matrix inversion and reduces the complexity of several orders of magnitude. The summary of the KCF method is briefly introduced as follows. The KCF classifier is trained by utilizing image patch x ¼ ½x 1 ; x 2 ; . . . ; xm . The cyclic shifts are defined as fQt xjt ¼ 0 . . . m 1g, where Q is the permutation matrix. All of the positive and negative samples xt ¼ Qt x for the KCF classifier training are gained by a Gaussian function v. Then, vt is the label for xt. Classifier. The classifier of KCF tracker is trained by minimizing the following function m 2 X min ð2Þ gðxt Þ vt þ gjjdjj 2 d
t
where g represents a regularization parameter. The solution of equation (2) can be extended as a linear combination of the samples xt X d¼ bt ðxt Þ ð3Þ t
where b ¼ ½b1 ; b 2 ; . . . ; bm are the parameters under optimal conditions. ðxÞ represents the mapping to the Hilbert space introduced by the Gaussian kernel k, and bt represents the coefficients. T
ðxÞ ðx0 Þ ¼ kðx; x0 Þ
ð4Þ
Kernel matrix K with elements Kt,s is described as below
Kt;s ¼ kðxt ; xs Þ xt ; xs 2 fQt xjt ¼ 0 . . . m 1g
ð5Þ
FðbÞ is defined as follows FðbÞ ¼
FðvÞ Fðk xx Þ þ g
ð6Þ
where k xx represents the first row of the kernel matrix K. F represents the DFT. Fast detection. In the KCF algorithm, the kernel matrix is defined as K p ¼ kðQt1 p; Qs1 xÞ t; s 2 f1; 2; . . . ; mg. p and x represent the new image patch and training sample, respectively. Cyclic shift p can be gained by equation (7) gðpÞ ¼ F 1 Fðk xp Þ FðbÞ ð7Þ where g(p) represents a 1 n vector and F1 represents the inverse DFT. The coordinates corresponding to the maximum value of g(p) is defined as the new position. The details of the KCF method can be obtained from the study by Henriques et al.26
KF algorithm In order to improve the robustness of the tracking algorithm, the KF algorithm that estimates the position of the target in the next frame is used when the difference of object position between previous frame and current frame is larger than a certain threshold value. The predicted position is developed by the following conditions. For one thing, a certain threshold is selected by the characteristic of the target which includes the speed and the position of the target. For another, the predicted position is obtained according to the predicted position of the previous frame. The model of the object is usually considered as “constant velocity with random walk.” The state equation and the observation equation of the object are listed as follows Xi þ 1 ¼ ⍀Xi þ Ri ;
Yi þ1 ¼ HXi þ Si
ð8Þ
where Xi represents the state vector which includes four states (speed and position in x and y direction). Xi is given by Xi ¼ ½xi x_i yi y_i T ⍀ is the state transition matrix. It is defined as 2 3 1 T1 0 0 6 7 ⍀ ¼ 4 0 1 T1 0 5 0 0 0 1
ð9Þ
ð10Þ
where T1 represents the sampling time. Ri represents the system noise vector, it is described as follows Ri ¼ ½0 rxi 0 ryi T
ð11Þ
4
International Journal of Advanced Robotic Systems its value is larger than the distance threshold, the information of the target coordinates provided by the KCF algorithm is no longer trust worthy. Therefore, the coordinates predicted by KF is considered as current position of the target, and KF is updated using its prediction algorithm. When the difference value is smaller than the distance threshold, the information of the target coordinates provided by the KCF algorithm is trust worthy. Then, the proposed algorithm will implement the KCF algorithm to track target. The tracking performance is evaluated by two strategies. The first strategy is the distance error, which shows the error between the center of the target and target’s ground truth value. The second strategy is precision curve, 31 which describes the percentage of correctly tracked frames for the bounds of threshold. The distance error Ds is described as below qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð15Þ D s ¼ ðx k x o Þ 2 þðy k y o Þ 2
Start
Initialize parameters
Adaptive threshold selection approach KCF algorithm
Ds>threshold
N
Y Kalman filter algorithm
n=n+1
N
Stop
where (xk, yk) and (yo, xo) are the center of the target and ground truth value of the target, respectively.
Y End
Figure 1. The flow chart of the IKCF method. IKCF: improved kernelized correlation filter.
System setup and experiments
where rxi and ryi represent the uncorrelated zero-mean Gaussian noise elements, respectively. Yi represents the measurement vector which is defined as below
In this section, first of all, the composition of the optoelectronic tracking system is introduced. Then, the performance of IKCF algorithm is compared with some state-of-the-art trackers such as KCF, TLD, SAMF, MOSSE, and DCF algorithms.
Yi ¼ ½xi yi T
ð12Þ
where xi and yi represent the object coordinates at the time step i. H represents the observation matrix which is defined as follows 1 0 0 0 H¼ ð13Þ 0 0 1 0 Si represents the observation noise vector which is described as Si ¼ ½sxi syi T
ð14Þ
where vxi and vyi represent the uncorrelated zero-mean Gaussian noise elements, respectively.
The flow chart of the IKCF method Figure 1 describes the flow chart of the IKCF method. When the target is moving fast in a long distance, scale changes and occlusion on the target will occur. When the condition occurs, the position of the target will change dramatically. The proposed IKCF algorithm will play an important role under this condition. In the process of target tracking, the different value of object position between previous and current frame is considered as threshold. As
System setup The system includes control cabinet and pointing unit. It can track moving target by vision camera or infrared thermal imager in pointing unit and record the video for searching, tracking, and monitoring the target. Control cabinet contains industrial personal computer, image capture card, and liquid crystal display. The pointing unit includes infrared thermal imager, vision camera, bearing base, laser, torque motor, and transformer. The optoelectronic tracking system is shown in Figure 2. The proposed IKCF algorithm has been implemented in MATLAB environment. The designed code can carry out dozens to hundreds of fps, according to the size and the complexity of background of the template adopted. Using automatic focusing module, the target can be clearly imaged in the range of the lens.
Experiments The effectiveness of the proposed tracking method is verified by experiments on challenging videos and selected video. The challenging videos are Couple, Deer, Dancer, Football, CarScale, and Car4. They have been used for benchmarking in several papers.24,32,33 The selected video
Yue and Li
5 is the video sequences of Civil aviation aircraft recorded by ourselves. For the sequences of Couple, Deer, Dancer, Football, CarScale, Car4, and Civil aviation aircraft, the ground truth location of the target for each frame is marked by the authors. The description of these videos is shown in Table 1. Tracking results of the proposed algorithm are compared with the five state-of-the-art tracking algorithms, that is, KCF,26 TLD,20 SAMF,27 MOSSE,29 and DCF.26 Figures 3, 5, 7, 9, 11, 13, and 15 describe precision, distance error, and time for the Couple, Deer, Dancer, Football, CarScale, Car4, and Civil aviation aircraft videos, respectively. The curves show the results of KCF, IKCF, TLD, SAMF, MOSSE, and DCF in red, black, blue, cyan, magenta, and green colors, respectively. Figures 4, 6, 8, 10, 12, 14, and 16 describe screenshots for the Couple, Deer, Dancer, Football, CarScale, Car4, and Civil aviation aircraft videos, respectively. The tracking box shows the results of KCF, IKCF, TLD, SAMF, MOSSE, and DCF in black, green, blue, red, yellow, and cyan colors, respectively. Figure 4 shows some sample tracked frames for Couple video sequence. The IKCF algorithm successfully tracks the target during fast motion and background clutter (e.g. Frames 91, 95, 102, and 122). Figure 6 describes the performance of the IKCF tracking method for Deer video during motion blur (e.g. Frame 25), fast motion, and background clutter (as shown in Frames 27, 29, and 35). Figure 8 explains that the IKCF tracking method successfully deals with the out-of-plane rotation of the target
Figure 2. The optoelectronic tracking system. Table 1. Description of sequences. Number of frames Characteristics involved
Couple Deer
140 71
Dancer Football CarScale
225 362 252
Car4
659
Civil aviation aircraft
109
1 0.8
P re c is io n
(b) 250
KCF IKCF TLD SAMF MOSSE DCF
0.6
Dis ta n c e e rro r (p ixe l)
(a)
Fast motion and background clutter Motion blur, fast motion, and background clutter Scale variation and deformation Occlusion and background clutter Fast motion, occlusion, and scale variation Illumination variation and scale variation Fast motion, occlusion, and scale variation
0.4 0.2 0 0
10
20
30
40
50
200 150
Figure 3. Target tracking performance of Couple.
0.1
0.05
50 0 0
KCF IKCF TLD SAMF MOSSE DCF
0.15
100
Threshold
Figure 4. Screenshots of Couple.
(c) 0.2
KCF IKCF TLD SAMF MOSSE DCF
Tim e (s )
Sequence
20
40
60
80
Frame #
100
120
140
0 0
20
40
60
80
Frame #
100
120
140
6
International Journal of Advanced Robotic Systems
0.8
P re c is io n
(b) 140
KCF IKCF TLD SAMF MOSSE DCF
0.6 0.4 0.2 0 0
10
20
30
120 100 80
KCF IKCF TLD SAMF MOSSE DCF
0.4
60
0.3 0.2
40 0.1
20 0 0
50
40
(c) 0.5
KCF IKCF TLD SAMF MOSSE DCF
Tim e (s )
1
Dis ta n c e e rro r (p ixe l)
(a)
0
80
60
40
20
20
40
60
80
Frame #
Frame #
Threshold
Figure 5. Target tracking performance of Deer.
Figure 6. Screenshots of Deer.
0.6
Dis ta n c e e rro r (p ixe l)
0.8
P re c is io n
(b) 70
KCF IKCF TLD SAMF MOSSE DCF
0.4 0.2 0 0
10
20
30
40
(c) 0.5
KCF IKCF TLD SAMF MOSSE DCF
60 50 40
KCF IKCF TLD SAMF MOSSE DCF
0.4
Tim e (s )
(a) 1
30
0.3 0.2
20 0.1
10
50
0 0
50
100
Threshold
150
Frame #
200
250
0 0
50
100
150
200
250
Frame #
Figure 7. Target tracking performance of Dancer.
Figure 8. Screenshots of Dancer.
with scale variation (e.g. Frames 45 and 122) and deformation in Dancer video (e.g. Frames 135 and 142). Figure 10 describes the tracking results of the IKCF algorithm on Football video. The video includes background clutter (e.g. appearance in Frame 277 and Frame 335) and occlusion (e.g. more than 50% of the head is occluded as shown in Frame 286).
Figure 12 depicts some frames from CarScale data set. The video includes fast motion, occlusion, and scale variation, which makes it difficult to track target. The situation becomes worse due to the occlusion (e.g. Frame 162) and scale variation (e.g. Frames 213 and 230). Figure 14 describes some frames of Car4 video. The proposed algorithm treats scale variation (e.g. Frames 10
Yue and Li
KCF IKCF TLD SAMF MOSSE DCF
P re c is io n
0.8 0.6 0.4 0.2 0 0
10
20
30
40
(c)
70
KCF IKCF TLD SAMF MOSSE DCF
60 50 40 30
KCF IKCF TLD SAMF MOSSE DCF
0.3 0.2
20 0.1
10 0 0
50
0.5 0.4
Tim e (s )
(b)
1
Dis ta n c e e rro r (p ixe l)
(a)
7
100
200
Threshold
300
400
300
200
100
0
400
Frame #
Frame #
Figure 9. Target tracking performance of Football.
Figure 10. Screenshots of Football.
0.8
P re c is io n
(b) 350
KCF IKCF TLD SAMF MOSSE DCF
0.6 0.4 0.2 0 0
10
20
30
40
50
(c) 0.25
KCF IKCF TLD SAMF MOSSE DCF
300 250 200
KCF IKCF TLD SAMF MOSSE DCF
0.2
Tim e (s )
1
Dis ta n c e e rro r (p ixe l)
(a)
150
0.15 0.1
100 0.05
50 0 0
50
100
Threshold
150
Frame #
200
250
300
0 0
50
100
150
200
250
300
Frame #
Figure 11. Target tracking performance of CarScale.
Figure 12. Screenshots of CarScale.
and 400) and illumination variation (e.g. Frames 240 and 645). Figure 16 shows a few frames of Civil aviation aircraft video. The IKCF method successfully treats the scale variation effects on the target (e.g. Frames 27 and 98) and occlusion (e.g. Frame 38). Table 2 shows the results of mean precision, and Table 3 describes the mean fps. The best result for video is described in bold-underline, the second best is in bold, and
the third best result is in underline style. The last line in each table describes the average score of the methods for all seven videos. It can be observed from Table 2 that the proposed algorithm performs better than other algorithms in mean precision. The tracking accuracy of IKCF algorithm is improved due to usage of KF algorithm. The number of fps depends upon the size of template and search window. According to the size of the template and the fast processing of the search
8
International Journal of Advanced Robotic Systems
0.8
P re c is io n
(b) 250
KCF IKCF TLD SAMF MOSSE DCF
0.6 0.4 0.2 0 0
10
20
30
200 150
0.3 0.2 0.1
50 0 0
KCF IKCF TLD SAMF MOSSE DCF
0.4
100
50
40
(c) 0.5
KCF IKCF TLD SAMF MOSSE DCF
Tim e (s )
1
Dis ta n c e e rro r (p ixe l)
(a)
100
200
400
300
500
600
0 0
700
100
200
300
400
500
600
700
Frame #
Frame #
Threshold
Figure 13. Target tracking performance of Car4.
Figure 14. Screenshots of Car4.
0.6
Distance error (pixel)
0.8
Precision
(b) 1400
KCF IKCF TLD SAMF MOSSE DCF
0.4 0.2 0 0
10
20
30
Threshold
40
KCF IKCF TLD SAMF MOSSE DCF
1200 1000
50
800 600
KCF IKCF TLD SAMF MOSSE DCF
0.25 0.2 0.15 0.1
400
0.05
200 0 0
(c) 0.3
T im e ( s )
(a) 1
20
40
60
Frame #
80
100
120
0
20
40
Frame #
60
80
Figure 15. Target tracking performance of Civil aviation aircraft.
Figure 16. Screenshots of Civil aviation aircraft.
window, the normalized correlation is computed in Fourier domain or spatial domain. Although the fps of proposed algorithm is slower than that of MOSSE and DCF algorithm, the accuracy of the proposed algorithm has been improved, which lays the foundation for the accurate tracking target for optoelectronic tracking system.
Through the above seven experimental results and tracking performance comparison experiment on Table 2 and Table 3, it can be seen that the IKCF algorithm has better advantages in tracking accuracy and mean fps than the KCF, TLD, and SAMF algorithms. Although the mean fps of MOSSE algorithm is faster than the other five trackers,
Yue and Li
9
Table 2. Mean precision (20 pixels). Sequence Couple Deer Dancer Football CarScale Car4 Civil aviation aircraft Average
KCF
IKCF
TLD
0.793 0.817 1.0 0.796 0.806 0.950 0.716
0.986 0.859 1.0 0.801 0.806 0.973 0.725
1.0 0.732 0.396 0.796 0.631 0.918 0.697
SAMF MOSSE DCF 0.929 0.887 1.0 0.796 0.849 1.000 0.688
0.029 0.930 0.209 0.793 0.651 0.281 0.349
0.664 0.817 1.0 0.796 0.806 0.950 0.789
0.840 0.879 0.739 0.878 0.463
0.832
KCF: kernelized correlation filter; IKCF: improved kernelized correlation filter; TLD: Tracking-Learning-Detection; SAMF: scale adaptive with multiple features; MOSSE: minimum output sum of squared error; DCF: dual correlation filter.
Couple Deer Dancer Football CarScale Car4 Civil aviation aircraft Average
KCF
IKCF
TLD SAMF MOSSE
50.22 50.10 12.47 20.01 37.21 7.21 31.27 45.07 6.94 65.14 93.56 5.75 127.50 164.16 8.20 14.94 17.58 8.70 39.51 38.43 6.45 49.80
Declaration of conflicting interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is supported by the Foundation of the National Natural Science Foundation of China (No. 61427810 and 61733012) and by State Key Laboratory of Precision Measuring Technology and Instruments (No. PILT1713 and PILQ1703).
References
Table 3. Mean fps. Sequence
of the proposed algorithm in the optoelectronic tracking system.
63.73
7.96
DCF
5.26 98.25 69.49 2.24 56.41 31.24 5.36 73.31 46.60 7.87 126.40 95.38 6.50 226.22 166.18 2.88 34.02 22.11 4.33 89.66 54.99 4.92 100.61
69.43
fps: frames per second; KCF: kernelized correlation filter; IKCF: improved kernelized correlation filter; TLD: Tracking-Learning-Detection; SAMF: scale adaptive with multiple features; MOSSE: minimum output sum of squared error; DCF: dual correlation filter.
the tracking accuracy is poor compared to the IKCF trackers. Therefore, the IKCF algorithm can meet the tracking performance of optoelectronic tracking system for maneuvering target.
Conclusions and future work In this article, IKCF approach is proposed to achieve good tracking performance for maneuvering target. The proposed algorithm integrates the merits of adaptive threshold approach, KCF algorithm, and KF algorithm. The adaptive threshold approach can obtain the threshold of bounding box on the basis of the size of the target in the image. KF algorithm will predict the next target position when the different value of object position between previous and current frame is larger than the distance threshold. The tracking performance of the IKCF algorithm is demonstrated by comparing to some state-of-the-art trackers such as the KCF, TLD, SAMF, MOSSE, and DCF algorithms. The experimental results verify the stability and effectiveness of the proposed approach for tracking maneuvering target. Further work will be focused on the adaptive distance threshold selected technique and the real-time application
1. Ji W, Li Q, Xu B, et al. Maneuvering targets state prediction based on robust H1 filtering in opto-electronic tracking system. Signal Proc 2010; 90(6): 2002–2008. 2. Yue F, Li X, Chen C, et al. Adaptive integral backstepping sliding mode control for opto-electronic tracking system based on modified LuGre friction model. Int J Syst Sci 2017; 48(16): 3374–3381. 3. Yang C, Fan B, Ji L, et al. Speeded up low-rank online metric learning for object tracking. IEEE Trans Circ Syst Video Technol 2015; 25(6): 922–934. 4. Yue F and Li X. Adaptive kernelized correlation filter algorithm and application in target tracking. In: Proceedings of international conference on optical instruments and technology: optoelectronic measurement technology and systems (eds J Zhu, K Xu, HY Tam and H Xiao), Beijing, China, 28–30 October 2017, pp. 1062109(1–6). SPIE. 5. Biresaw TA, Cavallaro A and Regazzoni CS. Tracker-level fusion for robust Bayesian visual tracking. IEEE Trans Circ Syst Video Technol 2015; 25(5): 776–789. 6. Li X, He Z, You X, et al. A novel joint tracker based on occlusion detection. Knowledge Base Syst 2014; 71: 409–418. 7. Smeulders AWM, Chu DM, Cucchiara R, et al. Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Int 2014; 36(7): 1442–1468. 8. Wu Y, He X and Nguyen TQ. Moving objects detection with freely moving camera via background motion subtraction. IEEE Trans Circ Syst Video Technol 2017; 27(2): 236–248. 9. Zhang K, Zhang L and Yang MH. Fast compressive tracking. IEEE Trans Pattern Anal Mach Int 2014; 36(10): 2002–2015. 10. Chen S, Li S, Ji R, et al. Discriminative local collaborative representation for online object tracking. Knowledge Base Syst 2016; 100(C): 13–24. 11. Han Z, Jiao J, Zhang B, et al. Visual object tracking via sample-based adaptive sparse representation. Pattern Recognit 2011; 44(9): 2170–2183. 12. Oron S, Bar-Hillel A, Levi D, et al. Locally orderless tracking. In: Proceedings IEEE computer society conference on computer vision and pattern recognition, Providence, RI,
10
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
International Journal of Advanced Robotic Systems USA, 16–21 June 2012, pp. 1940–1947. IEEE Computer Society. Li H, Shen C and Shi Q. Real-time visual tracking using compressive sensing. In: Proceedings IEEE computer society conference on computer vision and pattern recognition, Colorado Springs, CO, USA, 20–25 June 2011, pp. 1305–1312. IEEE. Grabner H, Grabner M and Bischof H. Real-time tracking via on-line boosting. In: Proceedings of the 17th British machine vision conference, Edinburgh, United kingdom, 4–7 September 2006, pp. 47–56. British Machine Vision Association (BMVA). Zhu J, Lao Y and Zheng YF. Object tracking in structured environments for video surveillance applications. IEEE Trans Circ Syst Video Technol 2010; 20(2): 223–235. Kalal Z, Matas J and Mikolajczyk K. P-N learning: bootstrapping binary classifiers by structural constraints. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, San Francisco, CA, USA, 13–18 June 2010, pp. 49–56. IEEE Computer Society. Babenko B, Yang MH and Belongie S. Robust object tracking with online multiple instance learning. IEEE Trans Pattern Anal Mach Int 2011; 33(8): 1619–1632. Zhang K and Song H. Real-time visual tracking via online weighted multiple instance learning. Pattern Recognit 2013; 46(1): 397–411. Wang S, Lu H, Yang F, et al. Superpixel tracking. In: Proceedings of IEEE conference on computer vision, Barcelona, Spain, 6–13 November 2011, pp. 1323–1330. IEEE. Kalal Z, Mikolajczyk K and Matas J. Tracking-learningdetection. IEEE Trans Pattern Anal Mach Int 2012; 34(7): 1409–1422. Yan J, Chen X, Deng D, et al. Visual object tracking via online sparse instance learning. J Vis Commun Image Represent 2015; 26(C): 231–246. Wang Q, Chen F, Xu W, et al. Object tracking via partial least squares analysis. IEEE Trans Image Proc 2012; 21(10): 4454–4465.
23. Zhang S, Lan X, Qi Y, et al. Robust visual tracking via basis matching. IEEE Trans Circ Syst Video Technol 2017; 27(3): 421–430. 24. Zhang S, Yao H, Zhou H, et al. Robust visual tracking based on online learning sparse representation. Neurocomputing 2013; 100(1): 31–40. 25. Galoogahi HK, Sim T and Lucey S. Multi-channel correlation filters. In: IEEE international conference on computer vision, ICCV 2013, Sydney, Australia, 1–8 December 2013, pp. 3072–3079. Institute of Electrical and Electronics Engineers Inc. 26. Henriques J, Caseiro R, Martins P, et al. High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Int 2015; 37(3): 583–596. 27. Li Y and Zhu J. A scale adaptive kernel correlation filter tracker with feature integration. In: Proceedings of the13th European conference on computer vision (eds L Agapito, C Rother and MM Bronstein), Zurich, Switzerland, 6–12 September 2014, pp. 254–265. Springer International Publishing. 28. Weijer JVD, Schmid C, Verbeek J, et al. Learning color names for real-world applications. IEEE Trans Image Proc 2009; 18(7): 1512–1523. 29. Bolme DS, Beveridge JR, Draper BA, et al. Visual object tracking using adaptive correlation filters. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, San Francisco, CA, USA, 13–18 June 2010, pp. 2544–2550. IEEE Computer Society. 30. Mazinan AH and Amir-Latifi A. Applying mean shift, motion information and Kalman filtering approaches to object tracking. Isa Trans 2012; 51(3): 485–497. 31. Chen S, Li S, Su S, et al. Online MIL tracking with instancelevel semi-supervised learning. Neurocomputing 2014; 139(139): 272–288. 32. Wu Y, Lim J and Yang MH. Online object tracking: a benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Portland, OR, USA, 23–28 June 2013, pp. 2411–2418. IEEE. 33. Xu C, Tao W, Meng Z, et al. Robust visual tracking via online multiple instance learning with fisher information. Pattern Recognition 2015; 48(12): 3917–3926.
