On the Solution of Data Association Problem Using ...

On the Solution of Data Association Problem Using Rollout Algorithms Selim Özgen, Mübeccel Demirekler, Umut Orguner Electrical and Electronics Engineering Department, METU, Ankara, Turkey {seozgen,demirek,umut}@metu.edu.tr

Abstract—The quality and precision of tracking maneuvering targets under heavy clutter is highly dependent on both the data association and the state estimation algorithms. In this study, measurement-to-track association problem for a single target when PD = 1 is discussed. The problem considers the batch set of measurements in a time interval. An approximate stochastic optimization algorithm for data association is presented. To reduce the computational load, the rollout algorithm is utilized. The algorithm is applied to a tracking scenario and GNN, JPDA and MHT algorithms are compared with their rollout versions. In the comparison several different track quality measures are used to demonstrate the efficiency of the algorithm. Index Terms—measurement-to-track association, Markov decision process, dynamic programming, rollout algorithms.

I. I NTRODUCTION The focus of this study is the optimal solution of the trackmeasurement association problem encountered in target tracking in clutter. With the presence of noise and clutter and the potential for missed detections, the association of measurement data to the targets become a fundamental problem for tracking the targets accurately [1]. Target tracking algorithms can be classified under two categories: recursive and batch tracking methods [2]. Recursive trackers try to find the optimal measurementto-track association considering the instantaneous measurement set. The simplest of these algorithms is Nearest Neighbor Algorithm (NN) [3]. This algorithm accepts the nearest measurement to the predicted state estimate as the original measurement from the target. Such a tracker has a very limited success. More sophisticated target tracking algorithms use probabilistic data association approaches. These trackers update the track with each of the measurements and obtain a weighted average of these measurement updates. Most frequently used algorithm from this class is called Probabilistic Data Association (PDA) [4]. PDA algorithm results from the minimum mean square estimation theory. Many different versions of PDA are available in the literature [4], [5]. In batch estimation methods, the decision given at any point of time depends on the future measurement data. These methods use many scans of the data to decide on the measurement originating from the target. Therefore, the decision comes with a delay. The well known Multiple Hypothesis Tracking (MHT) Algorithm can be seen as a batch estimator because former association decisions can be modified according to the future data. MHT is equivalent to testing all possible association hypotheses to reach the optimal solution [6]. MHT

algorithm is based upon the maximum likelihood estimation theory and there are many algorithms originating from it such as probabilistic MHT (PMHT) [7]. The use of batch estimation methods also include track splitting filter [8] and trackers using Viterbi algorithm for the association problem [9], [10]. The application of the Viterbi algorithm in the single-target case is originally proposed by Quach and Farooq in [9], who considered an approximate maximum likelihood formulation of the data association problem. The algorithm summarizes the number of different possible association sequences, thus the number of possible state estimates, to the number of measurements at each time instant. Without this necessary simplification, the computational cost increases exponentially as in the MHT case. The Viterbi algorithm determines the lowest cost path through the trellis, where the cost is a measure of the likelihood of the data association sequence [11]. The significant computational savings result from forcing data association conflicts, where two or more paths compete for the same node, to be resolved as soon as they occur. Pulford and Scala give a fixed-lag implementation of the Viterbi Data Association Algorithm [12]. In this study, a similar appoach to [12] is used. The decision about the measurement-to-track association is given after a fixed lag, say N time instants. The optimal solution is approximated using the rollout algorithm which is an approximation method for combinatorial optimization problems. To the authors’ best knowledge, the rollout algorithm has not been applied to a data association problem before and hence the approach presented here is novel. In Section II, the measurementto-track association problem and the optimal solution to it will be introduced. In Section III, the proposed rollout algorithm will be described. In Section IV, the simulation results will be given. II. T RACK -M EASUREMENT A SSOCIATION P ROBLEM The measurement-to-track association problem described in this study is defined for point targets, therefore it is assumed that there is at most one measurement coming from each target at each time instant. The target dynamics and the measurement model is given as below: xk = Axk−1 + Gwk−1

(1)

yk = Cxk + Hvk

(2)

where wk and vk denote the process and measurement noises respectively and they are assumed to be white Gaussian noises with known statistics [13]. When the sequence of measurement-to-target associations for all times are known, the optimal estimator for the system given by (1) and (2) is the Kalman smoother. Yet if the association decisions are not given, the problem changes fundamentally especially under high clutter. The clutter is assumed to be uniformly distributed in the measurement space. The number of clutter detections, mc , within a region of volume V is assumed to have a Poisson distribution: e−βF A V (−βF A V )m (3) m! where βF A is the clutter density per unit volume. Under high clutter, it is too costly to take into account all possible measurement-to-track associations. Hence, a subset of the measurements are selected by discarding the ones that are far away from the predicted state of the target using a gate [14]. If there are more than one measurements inside the gate, an ambiguity about the measurement-to-track association arises. Define the instant measurement set as Yk = {yk1 , . . . , ykmk } and the sets of measurements up to and including time k as Y k = {Y0 , . . . , Yk }. For recursive methods, if the state and measurement are linear Gaussian, it is known that the past information about the state can be summarized as follows; p(mc = m) =

p(xk |Y

k−1

) = N (xk ; x ˆk|k−1 , Pk|k−1 )

(4)

where N (xk ; (.), (.)) denotes normal probability distribution and x ˆk|k−1 and Pk|k−1 refer to the predicted mean and covariance of the state, respectively. After the instant measurement set Yk is taken, the updated probability distribution p(xk |Y k ) should be found. However, for this distribution to be found, the original measurement coming from the target should be known. The predicted likelihood for the measurement can be written as follows; p(yk |Y k−1 ) = N (yk ; yˆk|k−1 , Sk|k−1 )

(5)

This likelihood can be used to form a gate around the predicted measurement estimate yˆk|k−1 . As the system equations are linear and the noises are assumed to be Gaussian, this gate can be represented as an ellipsoid as defined below: −1 V(k, γ) = {y : (y − yˆk|k−1 )0 Sk|k−1 (y − yˆk|k−1 ) ≤ γ} (6)

where γ is the threshold value used for gating. A possible measurement scenario is illustrated in Figure 1. After the gate is opened and possible measurements are selected, the association hypotheses are determined. In a single target scenario, the association decision at each time k denoted with θk are defined as follows;

Fig. 1: Two dimensional case. The measurement estimate for the target at time k, the gate belonging to the measurement estimate and the measurement set at time k. where yki , is the ith measurement inside the gate and i ∈ {1, . . . , mk }. We also define the association decision sequence as Θk = (θ0 , . . . , θk ). Note that, the number of possible values of Θk increases exponentially. Yet, only one of these association decision sequences is valid. A. The Optimal Solution to Measurement-to-Track Association Problem with Batch Measurements The aim of this section is to define an objective function for the data association problem that is suitable for dynamic programming approach and explain its solution strategy. The algorithm proposed in this study makes a decision at each time instant k by considering the information state at time k − 1 and the measurements available in the time interval [k, k + N ]. To simplify the notation we assume that k = 1, x0 ∼ N (ˆ x0 , P0 ) and the available measurements are Y N . The optimal solution will be discussed for a single target when the probability of detection equals to 1. Both this assumption and single target assumption will be relaxed in future work, yet they prove advantageous for explanation purposes in this initial study. When the association history for the given time horizon, ΘN , is also supplied, the state estimates can be written as follows; k−1

p(xk |Y k−1 , Θk−1 ) = N (xk ; x ˆΘ k|k−1 , Pk|k−1 )

where the mean of the state estimate is a function of the association history, Θk−1 . Due to linearity and Gaussianity assumptions, the covariance of the state, Pk|k−1 , can be calculated without knowing the association sequence since there are no missed detections. The posterior probability of the association decision sequence can be written as follows: p(ΘN |Y N ) ∝

N Y k=1

( θk =

θk = 0, θk = i,

All of Yk is clutter yki belongs to the target

(7)

(8)

θ

e

k−1

θ

k−1

0 −1 k − 21 (ykk −C x ˆΘ ˆΘ k|k−1 ) Sk|k−1 (yk −C x k|k−1 )

(9) Note that the covariance matrix dependent coefficients of the posterior probability are omitted since they will be the same

for different association decision sequences. Moreover, the multiplication in Equation (9) is possible due to the independence of the innovations. This posterior probability will be used as the objective function for the data association problem, which is maximized with respect to the association sequence ΘN . In this study, we restrict ourselves to the set of association decisions that select one of the measurements at each time instant. Later we will relax this condition by accepting the no measurement case, i.e., the predicted position. Taking the logarithm of this expression will transform the objective function into the additive form. Furthermore by multiplying the objective function by -1, we convert the problem to a minimization problem. The new objective function that must be minimized is given below: J(ˆ x 0 , ΘN ) =

N X 1 k=1

2

k−1

k−1

θk 0 −1 (ykθk − C x ˆΘ ˆΘ k|k−1 ) Sk|k−1 (yk − C x k|k−1 )

(10) At this stage we can define the incremental cost at time k as: 1 −1 (ykθk − Cx) c(x, θk ) = (ykθk − Cx)0 Sk|k−1 (11) 2 Then the overall cost can be written as; N X

k−1

Θ c(ˆ xk|k−1 , θk )

(12)

V (ˆ x0 ) := min(J(ˆ x0 , ΘN ))

(13)

J(ˆ x0 , ΘN ) =

k=1 ΘN

Note that, the incremental cost at time k has the variable k−1 xΘ k|k−1 . Therefore the association decision at any step would affect the incremental costs at future times. One way to tackle with this problem is to compute the cost for all of the possible values of function ΘN . Yet, the enumeration and evaluation of all associations would correspond to the MHT algorithm. Quach et al. has simplified the problem by limiting the affect of the association decision at any time to the one step incremental cost while calculating the objective function [9]. Since the exhaustive and exact solution of the problem defined above is too computationally costly, we will solve the problem using the rollout approach. Next section gives information about the rollout approach and its application to the data association problem. III. ROLLOUT A LGORITHM Rollout algorithms are used for finding approximate solutions to discrete optimization problems. By the aid of rollout algorithms, the effectiveness of a given policy can be increased [15]. The rollout strategy is used to estimate the optimal control action at the current state by calculating the near-future benefits and approximating the far-future ones. The near-future benefits are calculated for one or two planning stages called as ’look-ahead’ stages. After the look-ahead stages, a fixed base policy is applied for calculating the far-future benefits. This approach is illustrated in Figure 2.

Fig. 2: Rollout algorithm is used for calculating the total benefits gathered from the current action. All possible association decision sequences are enumerated and calculated for the lookahead stages and the far future benefits are computed using the selected base policy. Figure is adopted from [16].

In Markov decision process literature, the optimal value can be found by either value or policy iteration techniques [17]. While value iteration techniques are based on calculating the optimal action for the state space at every stage to define the policy, policy iteration methods start with a selected base policy and search for the optimal value by iterating this policy. The rollout approach is a policy-iteration step performed on some fixed base policy π. In particular, for a fixed base policy π, one step of the policy-iteration consists of solving the following problem: V (x) = max {r(x) + Jπ (F (x, u))} u∈U

(14)

where Jπ is the reward of base policy π; F (.) is the state transition function and x, u are the current state and the input, respectively. Instead of applying the original base policy π immediately, all possible actions for the current time are considered. Therefore the overall cost is a one step improvement of the base policy π. To approach a better solution, it is desirable that π is near-optimal. Furthermore, Jπ should be easy to compute [18]. In this study, the rollout algorithm is going to be used for maximizing the measurement-to-track association likelihood. The aim of the algorithm is to decide the instantenous association hypothesis at time k by considering all of the measurement history Y k+N . Assume that the association decisions are given until time k−1. The optimization problem can be written

as follows:  V (ˆ xk|k−1 ) = min c(ˆ xk|k−1 , θk ) + Jπ (f (ˆ xk|k−1 , θk )) θk

(15) Since the association decisions are given until time k − 1, we Θk−1 . The covariances are intentionally use x ˆk|k−1 instead of x ˆk|k−1 discarded from the equation as they are not dependent on the association history. f (., .) calculates the predicted state estimate for time k + 1 for the instantaneous association decision θk when the measurement set Y k+N is taken as given information. The cost to go function at any step k is computed approximately by the base policy Jπ . One step of the the algorithm can be described as follows. Assume that the association decision at time k − 1 is given. The algorithm first calculates all possible association hypotheses for time k and all these hypotheses are kept in memory. For the following N frames, one of the base policies is used to handle measurement-to-track association problem. N will be called as the rollout horizon. The base policies in our case will be GNN, JPDA, or K-best MHT algorithms. While K-best MHT algorithm would result in K hypotheses at the end of time k +N for each hypothesis stored at time k, other base policies (GNN, JPDA) only create one hypothesis for each branch (of time k). Each association hypothesis at time k + N would get a different score, and biggest score is likely to come from the correct association hypothesis at time k. The branch with the largest score is selected and its association decision at time k is selected as the correct association hypothesis. This problem is similar to the path merge problem in MHT algorithms, in which all the surviving association hypotheses originate from a single branch. In that approach, the association decision is taken after the path merge occurs; therefore the time lag becomes a random variable. Notice that the calculation of the cost, Jπ , is only done to determine θk . Since the main concern of the data association algorithm is to prevent the track losses, track initiation calculations are omitted during the rollout horizon. The decision about time k is made only considering the existing track scores calculated according to Equation (10). As aforementioned, in this study we assume that PD = 1. Hence one of the measurements at each time step has to belong to the target. On the other hand, during the simulations, we have seen that including a ’no measurement association’ hypothesis into the set of possible hypotheses at each time step improves the results. The explanation for this interesting behaviour can be explained as follows. During the tracking, if there is almost no maneuver in the target movement and if the tracker thinks it already has reached a good estimate of the state vector, the data association algorithm sometimes might select no measurement update to filter large measurement noises in the measurements. It must be noted here that the action taken when ’no measurement association’ hypothesis is selected is not a standard prediction update of the Kalman filter. In the case when ’no measurement association’ hypothesis is selected, we still update the covariance while a standard

prediction is applied on the mean of the state vector. This is a necessary strategy for the covariance calculations not to depend on the association hypotheses. In the case when ’no measurement association’ hypothesis is selected, the score is taken as; c(ˆ xk|k−1 , θ0 ) = log(βF A ) (16) IV. S IMULATIONS We consider the problem of tracking a maneuvering single target in two dimensions x, y ∈ [0, 3000 m]. The state vector is x = [px py vx vy ]T . In the first part of the scenario, the target moves with constant velocity, then makes a coordinated turn with a maximum acceleration of 1.2 m/s2 and finishes the scenario by going with constant velocity as can be seen in Figure 3. For the tracker, a constant velocity model has been used as defined below;   2   T /2 0 1 T 0 0  T 0 1 0 0  0  w   xk =  0 0 1 T  xk−1 +  0 T 2 /2 k−1 0 T 0 0 0 1   1 0 0 0 yk = x + vk 0 1 0 0 k    0 where T is the sampling time. wk ∼ N , I2∗2 and 0    0 2 vk ∼ N , σ I2∗2 . The simulations are performed for 0 different values of the measurement noise standard deviation, σ. The measurement noise standard deviation used in the tracker is the same as the true measurement noise standard deviation. The probability of gating is taken PG = 0.999 to make sure that the original measurement falls inside the measurement gate. The simulation environment and the tracker codes were written in MATLAB environment using a standard desktop computer (Intel Core i7-3770 3.4GHz 8GB RAM). The algorithm is tested for 100 Monte Carlo runs for different values of measurement noise standard deviation, σ, and the clutter density, βF A . The volume of the surveillance region, V , in Equation (3) is 9.106 m2 . The rollout horizon N is taken as 7 to account for the maneuver of the target. While the rollout horizon itself is a random variable that should be selected considering the scenario difficulty, Pulford and Scala show by experiment that the path merge time for MHT, which is the time lag for giving an association decision, is around 5 dwells [12]. However, we observed that in our scenario N = 5 may cause track loss under high clutter. The performance measures used in this study are taken from [19] with a few exceptions. The used measures can be given as follows; • Number of Valid Tracks (NVT): A track is validated if it is assigned to the target. • Number of False Tracks (NFT): A track is detected as a false one if it is not assigned to the target.

Root Mean Squared Error (RMSE): If for any time instant a target is associated to the track, the l2 distance between them are calculated. • Target Coverage (TCvr): Target coverage shows in what percentage of the scenario, a track is assigned to the true target. This is not defined in [19]. • Track Continuity (TCnt): Track continuity is defined as target coverage devided by the number of associated tracks to the target. TCnt=TCvr/NVT • Total Execution Time (TET): The runtime of different algorithms are measured for the same set of measurements. • Rate of False Alarm (RFA): The average number of false tracks at each time instant of the scenario. Track assignment is made as follows. Let xpk denote the p denote the position of the true target and xpk|k and Pk|k position components of the mean and covariance of the state estimate, respectively. Then, the track is covered if p −1 p (xpk|k − xpk )0 (Pk|k ) (xk|k − xpk ) ≤ γ. The coverage threshold, γ, is set to be the same as the gate threshold. Target coverage is not listed in the tables since the track continuity and number of valid tracks supply a similar statistics. Note that RMSE calculation is only possible as long as there is an assigned track to the true target. If the track is not covered for a long time or if there is more than one track assigned to the target, this has no effect on the RMSE calculation. Therefore, we define the RMSE per track continuity as NRMSE = RMSE/TCnt. Note that NRMSE approaches the RMSE value if the track is covered for all times and there are no track losses. The average performance measures obtained in the MC runs are shown in Tables I-IV. Tables I-III show the effect of increasing clutter density, while Table IV shows the impact of increasing the measurement noise under moderate clutter. The computational burden of the batch algorithms compared to the recursive algorithms are evident. While GNN and JPDA use almost the same amount of time in all cases, their rollout versions have approximately 12 times more computation time. For the MHT and MHTR algorithms, the situation is different while both should be considered as batch algorithms. In this study, the K-best version of the MHT algorithm is implemented, which limits the number of maximum hypothesis at each step to K [20]. For the MHT algorithm, K is selected as 10. For the rollout MHT algorithm, a K-best MHT algorithm is applied for every association hypotheses formed at time k. But for time considerations, the number of maximum hypothesis is taken as 3 for the inner K-best MHT algorithms. Note that, if MHT is applied instead of the K-best MHT algorithm in these two cases, the number of calculations would be the same for MHT and MHT rollout algorithms. Therefore both of this algorithms are sub-optimal. Even for this sub-optimal case, the time disadvantage of MHT algorithms are evident from Tables I-IV. The computation time of the MHT and MHTR algorithms are at least ten times those of the other rollout algorithms. When the track loss performances of different data association algorithms are compared, it can be seen that the rollout •

algorithms are able to prevent track losses. In Table I, there is no track loss as the NVT is almost equal to 1 for all different data association algorithms. With increasing clutter density, track loss increases slightly for the cases where σ = 20 m as can be seen in Tables II and III. Increasing the measurement noise causes track loss rate to increase, but its main impact is on the number of false tracks initated as can be seen from Table IV. Notice that, the number of false tracks is similar in all algorithms in Tables I to IV while the same track initiation logic is used in all cases (4/9 logic). The persistence of false tracks increases with increasing measurement noise. In Table IV it is observed that, there is a false track in at least half of the scenario as can be seen from the RFA values. To compare the RMSE results of the experiments, it would be beneficial to give a lower bound on the performance measures. Here we consider the RMSE, TCnt and NRMSE values that would be obtained under no false alarms as the lower bounds for these quantities. In order to obtain the lower bounds, trackers with different data association algorithms were run for 100 times with σ = 20 m and σ = 40 m when the other two parameters are kept constant as βF A = 0 and N = 7. The results are demonstrated at Tables V and VI. When we compare the NRMSE results for different algorithms in Tables I to III, we can see that the rollout versions of the algorithms give a better performance in each case. As expected, increasing clutter density causes a performance deterioration in recursive methods, but the rollout algorithms are able to attain to the same NRMSE results in all three tables. Therefore the performance difference between the GNN, JPDA, MHT algorithms and their rollout counterparts gets larger when clutter is increased. Moreover, MHTR algorithm is able to reach the lower bound for the NRMSE value as can be seen from Table V. When Tables II and IV are compared, it can be seen that the NRMSE values increase almost as twice while σ is multiplied by two. When σ = 40, the performance difference between the GNN, JPDA, MHT algorithms and their rollout counterparts becomes more visible. When the measurement noise increases, the track loss becomes more likely, causing a drastic fall in the track continuity as can be seen in Table IV. While the use of rollout algorithm prevents track loss, the NRMSE performances increase. Moreover, MHTR is again able to reach the lower bound with the cost of high computation time. Increasing the clutter density or measurement noise standard deviation would emphasize the impact of the rollout algorithms as the association ambiguity is increased. However, such experiments become infeasible due to the too heavy computational load. A single run with parameters βF A = 10−5 , σ = 20 m, N = 7 is shown in Figure 3. To give an idea about the difficulty of the scenario, the measurements that are close to the target at every time instant are also shown in the figure. The incremental score function values for different algorithms are also provided in the figure. Notice that at many of the time instants the rollout algorithms attain the threshold value log(βF A ) = −11.5129, which is the cost of no measurement

TABLE I: Performance measures for 100 trials with σ = 20 m, βF A = 3.10−6 and N = 7 NVT NFT RMSE TCnt NRMSE TET RFA

GNN 1.160 0 16.47 0.844 19.50 1.489 0.072

JPDA 1.040 0 16.70 0.924 18.08 1.448 0.049

MHT 1.040 0 16.55 0.920 17.99 142.9 0.053

GNNR 1.070 0 14.80 0.911 16.25 18.11 0.047

JPDAR 1.050 0 14.77 0.921 16.04 17.85 0.047

MHTR 1.050 0 14.78 0.921 16.05 117.8 0.047

TABLE II: Performance measures for 100 trials with σ = 20 m, βF A = 6.10−6 and N = 7 NVT NFT RMSE TCnt NRMSE TET RFA

GNN 1.360 0 16.05 0.748 21.46 2.695 0.084

JPDA 1.080 0 16.70 0.911 18.33 2.616 0.044

MHT 1.080 0 16.19 0.898 18.03 311.5 0.056

GNNR 1.140 0 14.53 0.877 16.58 37.85 0.051

JPDAR 1.120 0 14.46 0.887 16.30 36.59 0.048

MHTR 1.040 0 14.60 0.929 15.72 253.6 0.044

TABLE III: Performance measures for 100 trials with σ = 20 m, βF A = 10−5 and N = 7 NVT NFT RMSE TCnt NRMSE TET RFA

GNN 1.567 0.167 16.16 0.645 25.07 12.45 0.123

JPDA 1.167 0.100 17.01 0.864 19.70 12.33 0.064

MHT 1.233 0.133 16.15 0.817 19.77 1378 0.089

GNNR 1.133 0.100 14.77 0.889 16.61 161.1 0.050

JPDAR 1.167 0.100 14.69 0.875 16.78 159.387 0.049

MHTR 1.067 0.100 14.59 0.926 15.76 1218 0.043

TABLE IV: Performance measures for 100 trials with σ = 40 m, βF A = 6.10−6 and N = 7 NVT NFT RMSE TCnt NRMSE TET RFA

GNN 1.833 4.133 28.51 0.502 56.78 5.031 0.593

JPDA 1.167 4.000 32.69 0.839 38.97 4.970 0.728

MHT 1.267 4.067 28.63 0.726 39.45 509.21 0.561

GNNR 1.200 4.033 26.56 0.823 32.27 68.94 0.580

JPDAR 1.167 4.067 27.33 0.811 33.71 67.80 0.583

MHTR 1.100 4.033 26.73 0.867 30.84 633.7 0.573

TABLE V: Performance measures for 100 trials with σ = 20 m, βF A = 0 and N = 7 RMSE TCnt NRMSE

GNN 16.19 0.910 17.80

JPDA 16.26 0.929 17.49

MHT 16.21 0.917 17.69

GNNR 14.32 0.914 15.67

JPDAR 14.29 0.909 15.72

MHTR 14.32 0.914 15.67

TABLE VI: Performance measures for 100 trials with σ = 40 m, βF A = 0 and N = 7 RMSE TCnt NRMSE

GNN 29.12 0.886 32.88

JPDA 29.84 0.918 32.51

MHT 29.13 0.887 32.83

GNNR 26.20 0.894 29.31

JPDAR 26.18 0.893 29.31

MHTR 26.20 0.894 29.31

update. The rollout algorithms choose no measurement update especially through the constant velocity parts of the scenario. As the target starts to maneuver, the measurements become important and the algorithms start using the measurements. The suboptimal MHT algorithm implemented shows a bad performance during the target maneuver under high clutter as can be seen in the figure. It is also evident that the rollout algorithms have made similar association decisions in a majority of the time instants. V. C ONCLUSIONS In this paper, we have dealt with the single target tracking problem under clutter. A novel approach to the problem is proposed using approximate stochastic optimization. In target tracking under high clutter, a common approach is to make a decision about the data association problem with a time lag. The proposed algorithm uses this methodology with the approximate form of the cost function, which is easier to calculate and makes the resulting method very time efficient compared to MHT. One interesting result of the study is the no measurement association decisions made by the data association algorithms using rollout method. When there is no target maneuver and the tracker has reached a good estimate of the state vector, the rollout algorithm is able to discard the ’very noisy’ measurements by looking at the future measurements. The current study is limited to the single target tracking problem. However, the algorithm is eligible for multiple target tracking if the base policies used in the rollout algorithm are chosen as JPDA, GNN and K-best MHT. R EFERENCES [1] G. Pulford and B. La Scala, “Multihypothesis Viterbi data association: Algorithm development and assessment,” Aerospace and Electronic Systems, IEEE Transactions on, vol. 46, no. 2, pp. 583–609, 2010. [2] D. Mušicki and B. L. Scala, “Multi-target tracking in clutter without measurement assignment,” Aerospace and Electronic Systems, IEEE Transactions on, vol. 44, no. 3, pp. 877–896, 2008. [3] S. Blackman and R. Popoli, “Design and analysis of modern tracking systems. 1999,” Artech House, Norwood, MA, pp. 967–1068. [4] T. E. Fortmann, Y. Bar-Shalom, and M. Scheffe, “Multi-target tracking using joint probabilistic data association,” in Decision and Control including the Symposium on Adaptive Processes, 1980 19th IEEE Conference on. IEEE, 1980, pp. 807–812. [5] J. Roecker and G. Phillis, “Suboptimal joint probabilistic data association,” Aerospace and Electronic Systems, IEEE Transactions on, vol. 29, no. 2, pp. 510–517, 1993. [6] S. S. Blackman, “Multiple hypothesis tracking for multiple target tracking,” Aerospace and Electronic Systems Magazine, IEEE, vol. 19, no. 1, pp. 5–18, 2004. [7] R. L. Streit and T. E. Luginbuhl, “Maximum likelihood method for probabilistic multihypothesis tracking,” in SPIE’s International Symposium on Optical Engineering and Photonics in Aerospace Sensing. International Society for Optics and Photonics, 1994, pp. 394–405. [8] D. B. Reid, “An algorithm for tracking multiple targets,” Automatic Control, IEEE Transactions on, vol. 24, no. 6, pp. 843–854, 1979. [9] T. Quach and M. Farooq, “Maximum likelihood track formation with the Viterbi algorithm,” in Decision and Control, 1994., Proceedings of the 33rd IEEE Conference on, vol. 1. IEEE, 1994, pp. 271–276. [10] B. La Scala and G. Pulford, “Viterbi data association tracking for overthe-horizon radar,” Proc. Int. Radar Sym, vol. 3, pp. 155–164, 1998. [11] D. P. Bertsekas, D. P. Bertsekas, D. P. Bertsekas, and D. P. Bertsekas, Dynamic programming and optimal control. Athena Scientific Belmont, MA, 1995, vol. 1, no. 2.

[12] G. Pulford and B. La Scala, “Over-the-horizon radar tracking using the viterbi algorithm-third report to high frequency radar division,” CSSIP Report, Tech. Rep., 1995. [13] G. Pulford, “Taxonomy of multiple target tracking methods,” in Radar, Sonar and Navigation, IEE Proceedings-, vol. 152, no. 5. IET, 2005, pp. 291–304. [14] Y. Bar-Shalom, “Multitarget-multisensor tracking: Advanced applications,” Norwood, MA, Artech House, 1990, 391 p., vol. 1, 1990. [15] D. P. Bertsekas and D. A. Castanon, “Rollout algorithms for stochastic scheduling problems,” Journal of Heuristics, vol. 5, no. 1, pp. 89–108, 1999. [16] U. Orguner, “Target Tracking: Lecture 5 Multiple Target Tracking: Part I,” http://www.eee.metu.edu.tr/~umut/ee793/files/METULecture5.pdf, [Online; accessed 19-July-2015]. [17] A. R. Cassandra, Exact and approximate algorithms for partially observable Markov decision processes. Brown University, 1998. [18] A. Nedich, M. K. Schneider, and R. B. Washburn, “Farsighted sensor management strategies for move/stop tracking,” in Information Fusion, 2005 8th International Conference on, vol. 1. IEEE, 2005, pp. 8–pp. [19] A. A. Gorji, R. Tharmarasa, and T. Kirubarajan, “Performance measures for multiple target tracking problems,” in Information Fusion (FUSION), 2011 Proceedings of the 14th International Conference on. IEEE, 2011, pp. 1–8. [20] R. Danchick and G. Newnam, “Reformulating Reid’s MHT method with generalised Murty K-best ranked linear assignment algorithm,” in Radar, Sonar and Navigation, IEE Proceedings-, vol. 153, no. 1. IET, 2006, pp. 13–22.

target vs track trajectories GNN RMSE=14.1259 TCnt=0.43617

2500

2000

2000

1500

1500

y axis (m)

y axis (m)

2500

1000

500

0 800

target vs track trajectories rollout GNN RMSE=13.4042 TCnt=0.96454

1000

500

1000

1200

1400

1600

1800

2000

2200

2400

2600

0 800

2800

1000

1200

1400

1600

x axis (m)

(a) GNN 2500

2000

2000

1500

1500

1000

500

0 800

2600

2800

500

1000

1200

1400

1600

1800

2000

2200

2400

2600

0 800

2800

1000

1200

1400

1600

1800

2000

2200

2400

2600

2800

x axis (m)

(d) JPDA with Rollout

target vs track trajectories MHT RMSE=13.2263 TCnt=0.86525

2500

2000

2000

1500

1500

y axis (m)

y axis (m)

2400

1000

(c) JPDA

1000

500

0 800

2200

target vs track trajectories rollout JPDA RMSE=13.0925 TCnt=0.97872

x axis (m)

2500

2000

(b) GNN with Rollout

target vs track trajectories JPDA RMSE=15.2426 TCnt=0.95745

y axis (m)

y axis (m)

2500

1800 x axis (m)

target vs track trajectories rollout MHT RMSE=12.3262 TCnt=0.97872

1000

500

1000

1200

1400

1600

1800

2000

2200

2400

2600

0 800

2800

1000

1200

x axis (m)

1400

1600

1800

2000

2200

2400

2600

2800

x axis (m)

(e) K-best MHT

(f) K-best MHT with Rollout incremental score for the associated tracks

-3 -4

log likelihood ratios

-5 -6 -7 GNN JPDA MHT GNNR

-8 -9 -10

JPDAR

-11

MHTR

-12 0

50

100

150

time

(g) incremental score for different algorithms

Fig. 3: Graphical results for a maneuvering single target under high clutter (GNN, JPDA, MHT, GNNR , JPDAR , MHTR ). βF A = 10−5 , σ = 20 m, N = 7. The figures show the related part of the surveillance region.