Relative Performance of Single Scan Algorithms for ... - CiteSeerX

Acoustics 2002 - Innovation in Acoustics and Vibration Annual Conference of the Australian Acoustical Society 13-15 November 2002, Adelaide, Australia

RELATIVE PERFORMANCE OF SINGLE SCAN ALGORITHMS FOR PASSIVE SONAR TRACKING Graham W. Pulford Thales Underwater Systems Pty. Ltd., General Sonar Studies Group. 274 Victoria Rd, Rydalmere, N.S.W. 2116, Australia. E-mail: [email protected] 1 INTRODUCTION In a passive sonar system acoustic energy originating from a mechanical or biological source propagates through the ocean and impinges on an array of transducers. The resulting signals are digitised and processed through a beamforming stage that separates signals arriving from different angular directions. In narrowband processing, spectral analysis is applied to determine the signal’s frequency content while in broadband processing, the signal is integrated across a wide band to increase its signal-to-noise ratio (SNR). Normalisation of the noise background allows a detection threshold test to be applied to discriminate possible acoustic contacts. The available measurements from a single-sensor passive sonar system are therefore frequency, bearing and SNR (in the narrowband case) and bearing and SNR (in the broadband case). These measurements are processed over time to form tracks that may subsequently be analysed to determine contact range, speed and rest frequency [18]. In bearings-only tracking the receiver must perform own-ship manoeuvres to enable range estimation. Techniques for tracking multiple acoustic signals in frequency and/or bearing are under study at Thales Underwater Systems. Such automatic multiple target tracking (MTT) methods are an essential component of modern sonar systems, helping to reduce operator workload in situations where multi-dimensional information is received at high data rates. The underwater environment is a challenging arena for MTT for a number of reasons including the presence of multiple contacts (or targets), false alarms (due to clutter or reverberation), low signal-tonoise ratios, multipath interference and unmodelled signal dynamics. It is known that the optimal algorithms for tracking multiple targets in clutter have exponentially increasing computational and memory requirements (see Fig. 1). For this reason many suboptimal techniques have been proposed to solve the MTT problem that represent varying degrees of compromise between tracking performance and simplicity of implementation.

ISBN 0-909882-19-3 © 2002 AAS

212

Acoustics 2002 - Innovation in Acoustics and Vibration

Assume 2 measurements y(k) per scan k and 1 target

H0

k=4 H0

H1

k=3

H0 H0

H1 H1

k=2

H0

H0

H0

H1 H1 H0 H1

k=1 H0

H1

H0

H0

H1

H1

H1

H0 H1

H0: y(k) = target H1: y(k) = false alarm

H0

H1 H0 H1

H1

H0

UNDERWATER SYSTEMS

H1

H1

Figure 1: Exponential growth of association hypotheses in a target tracking problem. At each time k, each measurement may be the target (H1) or a false alarm (H0).

Several fundamentally different algorithm formulations are possible: single-scan (zero scanback) recursive, multiple-scan (N-scan-back) recursive and batch. The first two types allow the processing to advance one frame at a time, although in multiple-scan approaches, association alternatives across more than one scan are considered. Batch algorithms operate on a fixed number of frames at once and advance in multiples of the sampling time. The interested reader is referred to [2] for examples of these approaches. A comparative study of nearest neighbour association, probabilistic data association (PDA) and a hidden Markov model tracker (a discrete state-space approach) was carried out in [20]. Since the HMM is a batch algorithm, it has an advantage due to smoothing whereas the other approaches are filters in the information theoretic sense. Performance comparisons between PDA and the Viterbi data association algorithm [15] also highlight the gain achieved through data smoothing. Even when discussion is restricted to single-scan approaches to multiple target tracking, the question of performance prediction is unclear. To date, almost all target tracking techniques remain without analytical performance evaluation methods and the designer is obliged to perform Monte Carlo trials to predict system performance. Apparently simple questions such as which technique has the lowest average errors or the lowest false track rate can only be answered through extensive computer simulation. We have chosen to study the performance of the following 5 “single scan” MTT algorithms: (1) nearest neighbour Kalman filter (NNKF) [11]; (2) two-dimensional assignment algorithm (2-DA) [1]; (3) probabilistic data association (PDA) [4]; (4) joint PDA (JPDA) [13]; (5) nearest-neighbour joint probabilistic data association (NNJPDA).

ISBN 0-909882-19-3 © 2002 AAS

213


The current literature on MTT indicates that: (i) the NNKF has poorer performance in clutter than PDA [4]; (iv) JPDA has the best performance for crossing targets in the 2-D meaurement case [9, 13] although it can suffer from track coalescence [7, 12] in the 1-D measurement case. Note that the NNJPDA algorithm we present is not the same as the nearest-neighbour JPDA of Fitzgerald [12] since we use the best assignment from the 2-DA problem. There is however a paucity of performance results in regard to the overall performance of these algorithms when automatic track initiation and maintenance procedures are included in the assessment. Additionally, most of the radar tracking literature deals with the 2-D (x, y) measurement case, in which track crossings are easy to resolve and the clutter density is proportionally less than for a sonar problem with 1-D measurements. In the context of passive sonar tracking, we provide a ranking of all 5 algorithms in respect of the following performance metrics: (i) tracking error; (ii) number of confirmed true tracks; (iii) number of confirmed false tracks; (iv) true track length; (v) false track length. The results identify the strengths and weaknesses of the various algorithms and should help in the selection of tracking algorithms that are appropriate for specific passive sonar systems. The material is organised as follows. In section 2 we detail the 5 tracking algorithms chosen for this performance comparison. Section 3 deals with tracker implementation. Section 4 discusses the methodology for and results of the performance assessment. Some real data examples appear in section 5 and the paper is concluded in section 6. 2 TRACKING ALGORITHMS 2.1 Nearest Neighbour Kalman Filter

Although for computational reasons many operational passive sonar systems use the alphabeta filter [3], the NNKF is theoretically the most simple single-scan recursive tracking algorithm. In passive sonar the state consists of bearing and bearing rate, or frequency and frequency rate, or all four of these together. The NNKF consists of a discrete-time Kalman filter (KF) togeter with a measurement selection rule. The NNKF takes the KF’s state estimate xˆ (k | k ) and its error covariance P (k | k ) at time k and linearly predicts them to time k+1. The prediction is then used to determine a validation gate in the measurement space (bearing and/or frequency) based on the measurement prediction yˆ (k + 1 | k ) and its covariance S (k + 1) . When more than one measurement yi (k + 1) falls inside the gate, the closest one to the prediction is used to update the filter. The metric used is the chi-squared distance: d i2 = [ y i (k + 1) - yˆ (k + 1 | k )]¢ S -1 (k + 1) [ y i (k + 1) - yˆ (k + 1 | k )] .

The update corrects the state prediction by a time-varying gain multiplying the difference between the prediction and the actual measurement. The error covariance is also updated (see [3] for further details). This filter is only mean-square optimal when there are no false alarms and a single target is present. 2.2 2-D Assignment Algorithm

When multiple targets are present, the nearest neighbour rule can be modified to take target multiplicity into account. Suppose there are T tracks and M validated measurements between them. The single-scan measurement-to-track association problem may be posed as a 2-D

ISBN 0-909882-19-3 © 2002 AAS

214


assignment problem [1] in which the assignment cost between measurement i and track j is taken as the negative logarithm of: -1 g ij2 =| 2p S j (k + 1) |-1 / 2 exp{- 12 [ yi (k + 1) - yˆ j (k + 1 | k )]¢ S j (k + 1) [ yi (k + 1) - yˆ j (k + 1 | k )]} .

The resulting assignment problem, which may be non-square, may be solved by a number of techniques including the Munkres algorithm [16], the auction algorithm [6] and algorithms based on shortest augmenting paths [14]. The algorithm yields associations that enable tracks to be updated with their assigned measurement. Tracks not receiving a measurement are predicted but not updated. 2.3 Probabilistic Data Association Filter

The PDA filter is a single-scan approximation to the optimal Bayesian filter for tracking a single target in clutter [3]. The PDA filter is described in [4]. PDA is an “all-neighbours” approach that considers each of the n gated measurement y i (k ) in turn as if it were the target, using the Kalman filter to generate a conditional state update xˆ i (k | k ) . For each one of these association hypotheses, the association probability b i (k ) is computed based on statistical models of the number and spatial distribution of false alarms, and also the detection and gate probabilities. The overall state update is then a weighted combination of all the conditional updates, taking into account the hypothesis that none of the measurements may correspond to the target (i=0): n

xˆ (k | k ) = å b i (k ) xˆ i (k | k ) i =0

While it has been demonstrated that this strategy is more robust to track loss than the NNKF, it is also known that PDA does not correctly handle closely-spaced multiple targets. 2.4 Joint PDA Filter

Joint PDA is an extension of PDA to the case of multiple targets in clutter [13]. In this approach all possible associations between tracks and measurements are considered in the association probability calculations. The state estimation is the same as for PDA. To illustrate the concept, consider Fig. 2. In the PDA filter, each target track is independently updated; thus track 1 would be updated with y1 and y 2 whereas track 2 would use y 2 and y 3 . This ignores the fact that if y 2 is due to target 1 then it cannot be due to target 2. In JPDA we enumerate all possible permutations of all combinations of targets and false alarms. Omitting commas, the full list of association hypotheses consists of the 13 events {000}, {100}, {010}, {001}, {200}, {020}, {002}, {120}, {102}, {210}, {012}, {201}, {021}. Note that, for instance, {210} means that measurement 1 is from target 2, measurement 2 is from target 1 and measurement 3 is a false alarm. The all-clutter hypothesis {000} specifies that none of the measurements is from either target. Some of the events are excluded since they are inconsistent with the gating, leaving 8 feasible events: {000}, {100}, {010}, {020}, {002}, {120}, {102}, {012}. The posterior probability of each of the events must be calculated and then combined to give the association probabilities that are used in the track updating stage. The reader is referred to [3] for further details.

ISBN 0-909882-19-3 © 2002 AAS

215


FREQUENCY TRACK 2 GATE

y3 TRACK 1 GATE

y2(k+1|k) y2

y1(k+1|k) y1 BEARING

Figure 2: Gating scenario for 2 tracks and 3 measurements to which JPDA is applicable.

2.5 Nearest Neighbour JPDA Filter

The JPDA algorithm has very high computational complexity and many attempts have been made to simplify the association probability calculations [12, 19, 21]. All of these methods are ad hoc modifications to the original algorithm. A further such algorithm is proposed here, which we will refer to as NNJPDA. In this method the single best measurement-to-track association from the 2-D assignment algorithm is selected. This hypothesis along with the allclutter hypothesis are the only two used in the update process. The resulting algorithm is effectively a multitarget version of the a posteriori nearest neighbour filter [9, 10] that accounts for uncertainty in measurement origin. 3 TRACKER IMPLEMENTATION 3.1 Track Clustering

Validation gating excludes unlikely associations from the track updating process, providing a computational saving. A further step to reduce the load in dense multitarget environments is a process called track clustering. Any tracks that share measurements in the gating process are placed in the same track cluster. Each track in a cluster shares a measurement with at least one other track in the cluster. Clusters are non-overlapping and are processed as independent MTT subproblems. Particularly for JPDA, logic is also needed to limit both the number of tracks in a cluster and the total number of validated measurements to manageable computational levels. 3.2 Automatic Track Maintenance

Mechanisms are required in order to initiate tracks, confirm “good” tracks and delete “bad” tracks. Track initiation may be effected by selecting pairs of candidate measurements from successive scans of data, as long as these measurements have not been used for updating other tracks. If the implied rate of change of bearing (say) for a pair is not too great, a track may be started using the two measurements to estimate the initial bearing and bearing rate, and by selecting the initial track covariance. This two-point initiation strategy is discussed in [1, 3].

ISBN 0-909882-19-3 © 2002 AAS

216


Track maintenance is achieved by monitoring a track confidence measure: if track confidence reaches a threshold, the track is confirmed; if the confidence is too low, or if a maximum number of misses is exceeded, the track is deleted. In between these limits, the track is considered as pending, unless it has recently been initiated, in which case it is tentative. A confirmed track that becomes deleted is considered lost. These notions are summarised in Fig. 3. Various confidence measures may be tried [5, 8, 17], but not all of these are applicable to all 5 algorithm types since some of the algorithms are not Bayesian. In our case, the track confidence is taken as a smoothed version of the SNR’s from the history of gated measurements with the least innovations. This choice ensures that all algorithms use a common track maintenance framework and thus permits a fair comparison. INIT

TENT

Length> Init Delay

PEND or CONF

N

C £ Cdel

Y

N

CONF

TENT

Y

Y

Prev Conf

Y

LOST

N

DEL

C ³ Ccon N

PEND

Figure 3: Track status transitions for automatic track maintenance. Cdel = track deletion threshold; Ccon = confirmation threshold; init_delay = duration in the tentative state and prev_conf = previously confirmed.

3.3 Track Copy Deletion

In a heavily cluttered environment, tracks that are initiated on false alarms in the vicinity of a true track may sometimes converge to the true track. A means to detect multiple copies of the same track is needed to improve the clarity of the track display. Two tracks are declared to be copies if they share a sufficient number of measurements in their recent track histories. Various rules apply as to which track is to be deleted according to track length and confidence. 3.4 Tracker Tuning

The tracker has around 30 tuning parameters, most of which are common to all 5 algorithms. Some parameters, like clutter density and gate probability, are only used by PDA, JPDA and NNJPDA. The tracker was tuned for reliable performance on a number of real passive sonar data sets containing multiple bearing tracks. Tuning was also performed on simulated data for frequency-only and frequency-bearing operation. Tests were performed for crossing tracks and parallel tracks in 1-D at different measurement noise and clutter levels. Difficulties encountered during tuning included the adjustment of track maintenance thresholds to ensure satisfactory true track acceptance and false track rejection, as well as the filtering and gating parameters for tracking over a range of measurement noise values and false alarm densities.

ISBN 0-909882-19-3 © 2002 AAS

217


4 SIMULATED DATA TESTING 4.1 Scenario Definition and Performance Metrics

One thousand Monte Carlo trials of length 100 s (50 scans) were performed for all 5 algorithms on a single target in clutter bearings-only scenario. This case was chosen in preference to the 2-D bearing and frequency case as the latter lessens the effective clutter density. All algorithms used the same tuning parameter values including those for filtering and track maintenance. In this test, the true target bearing varied linearly, starting at time 4 s and ending at time 86 s with sampling time T = 2 s. SNR was Rayleigh distributed with mean 6 dB and measurement errors were zero mean Gaussian with a standard deviation of 1 degree. False alarms were uniformly distributed on [10, 170] degrees with an average number of 10 (6.25%) or 15 (9.375%) per scan. Acceptance for confirmed tracks was based on a maximum mean absolute error of 0.7 degrees (suitable for this scenario). Tracks not satisfying the acceptance test were classed as false tracks. The performance metrics chosen for algorithm comparison included: (i) true track absolute bearing error; (ii) expected number of true tracks; (iii) expected number of false tracks; (iv) true track length; (v) false track length. Note that we do not measure the probability of declaring a true or false track since it is not clear how to normalise results when a single run may have an arbitrary number of true and false tracks. There is ambiguity in the interpretation of the expected number of true tracks, which is a measure of both the probability of track detection and also the probability of track breakage. Another important performance indicator is track confirmation time. However, since all algorithms used the same SNR-based track confirmation rule, no significant differences were found for this quantity. 4.2 Performance Assessment Results

Monte Carlo trial results are presented in Tables 1 and 2 for the light and heavy clutter densities respectively. The ranking column on the right attests to the reliability of the NNKF in these two scenarios (which are typical of passive sonar data). Figure 4 shows that the NNKF had generally the lowest errors, although JPDA and NNJPDA were not far behind. The NNKF had the best false track statistics. Track continuity was better for PDA and JPDA and the lower complexity NNJPDA, the latter representing a good compromise between the NNKF and JPDA. Subjectively, the best overall track display is obtained with the NNKF if some track breakage may be tolerated (see real data examples, Fig. 6). Algorithm # samples for true track errors 100% mean error (deg) 100% median error (deg) 90% percentile error (deg) Expected # true tracks per run Expected # false tracks per run Mean true track length (scans) Mean false track length (scans)

NNKF 29101 0.623 0.348 1.148 1.208 0.363 31.5 15.6

2-DA 26516 0.676 0.357 1.250 1.462 0.421 27.2 15.2

PDA 31369 0.881 0.381 1.394 0.944 3.21 37.25 23.2

JPDA 31787 0.778 0.381 1.332 0.891 2.39 38.7 20.8

NNJPDA 29724 0.628 0.351 1.171 1.068 0.894 39 16.3

Ranking N/A 15243 15234 15243 21534 12543 54312 21543

Table 1: Scenario at 6.25% false alarms/degree. In general if the number of true tracks/run is greater than 1, track breakage has necessarily occurred; if it is less than 1, track loss has necessarily occurred.

ISBN 0-909882-19-3 © 2002 AAS

218


Algorithm Number of samples for statistics 100% mean error (deg) 100% median error (deg) 90% percentile error (deg) Expected # true tracks per run Expected # false tracks per run Mean true track length (scans) Mean false track length (scans)

NNKF 27243 0.771 0.3765 1.482 1.265 0.933 30.6 15.6

2-DA 23492 0.941 0.404 2.049 1.601 0.942 25.1 15.8

PDA 20075 1.19 0.432 1.863 0.804 8.551 33.96 25.5

JPDA 31787 0.865 0.398 1.5215 0.697 7.23 37.1 23.05

NNJPDA 29724 0.950 0.385 1.617 1.033 2.333 34 17.8

Ranking N/A 14253 15423 14532 21534 12543 45312 12543

Table 2: Scenario at 9.375% false alarms/degree. The algorithms are numbered from 1 to 5.

Figure 4: Cumulative distribution curves for absolute bearing errors at two clutter densities.

Figure 5: Crossing targets at high SNR: the NNKF (left) outperforms JPDA (right), which suffers from track coalescence. JPDA does better than the NNKF on tracks 2 and 4 when the noise is increased to 1 degree rms.

ISBN 0-909882-19-3 © 2002 AAS

219


5 REAL DATA TRACKING EXAMPLES Unclassified real bearing data from a passive sonar sensor are presented in these examples. Data were processed through a reference model of a sonar signal processing chain to obtain normalised, thresholded event data for tracking. Figures 6 and 7 show results for the NNKF algorithm, which gave the cleanest display for these data sets.

Figure 6: Relative bearing-time waterfall containing a strong man-made signal with sidelobes undergoing several sensor manoeuvres. Weaker biological signals are also present. Data courtesy of DSTO Australia.

Figure 7: Panoramic relative bearing-time waterfall containing 1 man-made contact with weak sidelobes, 2 groups of biological signals and 1 very strong intermittent biological signal at –90 deg.

ISBN 0-909882-19-3 © 2002 AAS

220


6 CONCLUSIONS Five single-scan tracking algorithms have been tested on a passive sonar tracking problem. Automatic track initiation and maintenance (based on SNR) were included in the assessment. The results indicate that, although the high complexity solutions like JPDA tend to give the lowest track loss and good crossing performance in higher noise, they also tend to retain false tracks. The humble NNKF is an effective solution that also deals well with crossings and has the lowest false track rate and errors. The NNJPDA algorithm presented here is a good tradeoff between JPDA and the NNKF and, because it is a Bayesian algorithm, is better suited to extensions such as integrated track existence and SNR models. Results on higher performance multiple-scan and batch algorithms will be reported in a future paper. ACKNOWLEDGEMENTS The author appreciates the assistance of Cyril Sadr-Khanlou and Jeffrey Szeto in developing some of the Matlab® software for this project. Alain Maguer, Pierre Blanc-Benon and Patrick Cooper are acknowledged for helpful suggestions. This work was funded by Thales Underwater Systems Pty. Ltd. under project number 2002-P-GSS-06.1-R-RY. REFERENCES [1] S. Blackman, Multiple Target Tracking with Radar Applications, Artech House, MA, 1986. [2] S. Blackman and R. Popoli, Design and Analysis of Modern Tracking Systems, Artech House, MA, 1999. [3] Y. Bar-Shalom and T. E. Fortmann, Tracking and Data Association, Academic Press, 1988. [4] Y. Bar-Shalom, and E. Tse, “Tracking in a Cluttered Environment With Probabilistic Data Association”, Automatica, v. 11, pp. 451 – 460, 1975. [5] Y. Bar-Shalom, K. C. Chang, and H. A. P. Blom, “Automatic Track Formation in Clutter with a Recursive Algorithm”, Proc. 28th Conf. Dec. Control, Florida, pp. 1402 – 1408, Dec. 1989. [6] D. P. Bersetkas, “The Auction Algorithm for Assignment and other Network Flow Problems: A Tutorial”, Interfaces, 20, 1990, 133–149. [7] E. A. Bloem and H. A. P. Blom, “Joint Probabilistic Data Association Methods Avoiding Track Coalescence”, Proc. 34th Conf. Dec. Control, LA, pp. 2752 – 2757, Dec. 1995. [8] S. B. Colegrove, A. W. Davis, and J. K. Ayliffe, “Track Initiation and Nearest Neighbours Incorporated into Probabilistic Data Association”, J. Elec. and Electronic Eng., Australia, v. 6, no. 3, pp. 191 – 198, Sept. 1986. [9] J. Dézert, “Poursuite multi-cibles mono-senseur: Analyse des principales approches développées dans le domaine”, Technical Report 1988–10, Onéra, Châtillon, France, 1988. [10] A. Farina and S. Pardini, “Track while scan algorithm in a clutter environment”, IEEE Trans. Aerosp. Elec. Sys., v. 14, no. 15, pp. 769 – 779, Sep. 1978. [11] A. Farina and F. A. Studer, Radar Data Processing, v. I: Introduction and Tracking, J. Wiley, 1985. [12] R. J. Fitzgerald, “Development of Practical PDA Logic for Multitarget Tracking by Microprocessor”, in Multitarget-Multisensor Tracking: Advanced Applications, (Y. Bar-Shalom, ed.), pp. 1–23, Artech House, 1990. [13] T. E. Fortmann, Y. Bar-Shalom and M. Scheffe, “Sonar Tracking of Multiple Targets Using Joint Probabilistic Data Association”, IEEE J. Oceanic Eng., v. OE-8, no. 3, pp. 173 – 184, July 1983. [14] R. Jonker, A. Volgenant, “A Shortest Augmenting Path Algorithm for Dense and Sparse Linear Assignment Problems”, Computing, v. 38, 1987, 325–340. [15] B. F. La Scala and G. W. Pulford, ``Viterbi Data Association Tracking for Over-the-Horizon Radar'', Proc. International Radar Symposium, v. 3, pp. 1155 - 1164, Munich, Sept. 1998. [16] J. Munkres, “Algorithms for the Assignment and Transportation Problems”, J. SIAM, v. 5, pp. 32 – 38, Mar. 1957. [17] D. Musicki, R. J. Evans and S. Stankovic, “Integrated Probabilistic Data Association”, IEEE Trans. Auto. Control, v. 39, no. 6, pp. 1237 – 1241, June 1994. [18] D. Pillon, N. Giordana, P. Blanc-Benon and S. Sitbon, “Association of Narrow Band Sources in Passive Sonar”, Proc. 5th Int. Conf. on Info. Fusion, Annalpolis, Maryland, US, pp. 1141 – 1146, July 2002. [19] J. A. Roecker, “A Class of Near Optimal JPDA Algorithms”, IEEE Trans. Aerosp. Elec. Sys., v. 30, no. 2, pp. 504 – 510, Apr. 1994. [20] S. Sitbon, “Comparative Study Between a New HMM Tracker and Conventional Passive Tracking Algorithms”, Proc. Undersea Defence Technology, pp. 441 – 445, 1994. [21] B. Zhou and N. K. Bose, “Multitarget Tracking in Clutter: Fast Algorithms for Data Association”, IEEE Trans. Aerosp. Elec. Sys., v. 29, no. 2, pp. 352 – 363, Apr. 1993.

ISBN 0-909882-19-3 © 2002 AAS

221