c NEC Research Institute, Technical Report 95 ? 07
A Comparison Of Two Algorithms For Determining Ranked Assignments With Application to Multi-Target Tracking and Motion Correspondence Matt L. Miller Baltic Images P.O. Box 964 ARP-3 2300 Vilnius LITHUANIA
Ingemar J. Cox NEC Research Institute 4 Independence Way Princeton, NJ 08540 U.S.A.
[email protected]
[email protected]
Roy Danchick and G. E. Newnam TRW Systems Integration Group One Space Park, bR2-1104 Redondo Beach, CA 90278 U.S.A
Abstract
Recently, it has become clear that determining a ranked set of assignments allows computation of very good approximations to the data association problem. Several algorithms have been proposed but only two return the k-best assignments in reasonable time. One is Danchick and Newnams' [DN93] algorithm, which is based on the recognition that determining the best assignment is a classical assignment problem, and that determining a ranked set of assignments may be accomplished by solving a series of modi ed copies of the initial assignment problem. The other algorithm is originally due to Murty [Mur68] and was most recently described by [CM95] within the context of multi-target tracking. In this paper, we evaluate the two algorithms on simulated and random data and show that Murty's algorithm performs signi cantly better in all scenarios. We show the relationship between the two algorithms and how Danchick and Newnams' algorithm can be very easily modi ed to Murty's algorithm. Experimental results using Murty's algorithm suggest that a solution to the real-time data association problem is now feasible.
1 Introduction The problem of data association, also called the motion correspondence problem, has been extensively studied by the multi-target tracking and surveillance communities [BSF88]. Two popular algorithms have emerged; the multiple hypothesis lter (MHT) of [Rei79] and the joint probabilistic data association lter (JPDAF) of Bar-Shalom [BSF88]. While the two algorithms are conceptually quite di erent, both need to enumerate all legal assignments of an ambiguity matrix.1 Each assignment, also called a hypothesis, has an associated 1 An ambiguity matrix is a concise representation of the ambiguity present in a scene. Columns of the matrix denote existing traks and rows denote measurements for a particular iteration. A non-zero entry in a row i and column j denotes the fact that measurement zi validated to track yj . A legal assignment of measurements to targets must satisfy the constraints that (1) a measurement can originate from only a single source target (often called the uniqueness constraints [Ull79]) and that (2) a target can give rise to no more than one measurement per cycle. These constraints require that a legal set of assignments, represented as a hypothesis matrix, have no more than a single non-zero entry in any row and/or column.
1
cost, computed from a probability value, which is used in subsequent calculations. While enumeration of the set of legal hypothesis is straightforward, it is also NP-hard. It is therefore clear that an exhaustive enumeration of all legal hypotheses is computationally infeasible. However, many such hypotheses have very small probability and consequently very little a ect on the overall calculation. This observation has led several researchers to investigate approximate multi-target tracking algorithms in which only a relatively small number of hypothesis with high probabilities are examined. Both Nagarajan et al [NCS87] and Brogan [Bro89] have suggested approximate solutions to this problem but Brogan's algorithm is not guaranteed to nd the -best hypotheses and both algorithms have exponential worst case run-times. In the context of multi-target tracking, Danchick and Newnam [DN93] were the rst to propose an algorithm guaranteed to nd the -best hypotheses. Danchick and Newnam recognize that nding the best hypothesis can be formulated as a classical linear assignment problem, a class of problems that has been extensively studied by the operations research and computer science communities and for which low order polynomial time algorithms have been developed. They then proposed to nd the second best etc, by modifying the cost matrix and solving the resulting assignment problems. Independent of Danchick and Newnam, Cox and Miller [CM95] also recognized that a legal hypothesis was a bipartite graph match, and could therefore be determined by classical assignment algorithms. To nd the -best hypotheses, they suggested using an algorithm rst due to Murty [Mur68]. This algorithm determines the -best hypotheses in low order, ( 4 ), time and is linear in . By contrast, the worst-case execution time of Danchick and Newnams' algorithm is exponential in . Nevertheless, the two algorithms are surprisingly similar. In Section (2) we rst describe the Danchick and Newnam algorithm, and then show how it can be converted to Murty's algorithm by a simple modi cation. In Section (3), we present experimental results which show the computational advantage of Murty's algorithm over that of Danchick and Newnam. k
k
k
k
O N
k
k
2 Algorithm Figures (1) and (2) show pseudocode for Danchick and Newnams' and Murty's algorithms respectively. Similar data structures and variable names have been used in both cases to highlight the similarity between the two algorithms. Both algorithms are given an assignment problem, 0 , for which they are to nd the -best solutions. The problem is expressed with a bipartite graph, represented as a list of triples, . In a multi-target tracking application, each represents a hypothesized target, each represents a measurement, and is the negative log-likelihood that is a measurement of . A solution, , to such an assignment problem is a list of triples in which each and each appears exactly once. The cost of a solution is the sum of the 's in its triples. The -best solutions are those with the lowest costs. In Danchick and Newnam, ANSWER-LIST is a list of pairs, , in which each is a set of assignment problems that all have the same best solution, . ANSWER-LIST is initialized with the single P
k
< y; z; l >
y
z
z
y
k
y
l
S
z
l
k
< Q; S >
S
2
Q
Find the best solution,
S0
, to
Initialize ANSWER-LIST with
P0
f 0g
P
P0
as UNPARTITIONED
While some problem in ANSWER-LIST is UNPARTITIONED
f Clear NEW-LIST Find the UNPARTITIONED pair, < ANSWER-LIST
,
Q
S
>, that has the lowest-cost solution in the
For each UNPARTITIONED problem, , in , do P
Q
f
For each triple,
from
Find the best solution, If
f
0
S
0
, to
P
0
0
P
Add
as UNPARTITIONED
P0
While some problem in ANSWER-LIST is UNPARTITIONED
f Clear NEW-LIST Find the UNPARTITIONED pair, ANSWER-LIST For each triple,
f
Let
P
0
=
Remove
f
S
0
the
, found in
S
, do
from
P
0
S
0
, to
P
0
exists, and its cost is less than or equal to the cost of the solution in k 'th-best problem/solution pair on ANSWER-LIST
Add
P
as UNPARTITIONED
0 0
< P ;S
0
>
to NEW-LIST
g From , remove all triples that include , and all triples that include , except for itself P
Mark
>
P
< y; z; l >
Mark
g
, that has the lowest-cost solution in
P; S
< y; z; l >
Find the best solution, If
P
as PARTITIONED
Append NEW-LIST to ANSWER-LIST Sort ANSWER-LIST into order of increasing solution cost Truncate ANSWER-LIST to the best
k
problem/solution pairs
g Figure 2: Psedocode describing Murty's lagorithm.
4
z
pair whose set of problems contains only 0 , and whose solution, 0 is the single best solution to 0 . After initialization, Danchick and Newnam go through a series of \sweeps" which increase the length of ANSWER-LIST. Sweeps proceed by a process called partitioning, which takes a problem, , and its best solution, , and , where is the number of triples in . In the DN algorithm, each new creates new problems 0 1 problem is created by rst copying , and then removing from it one of the triples that appears in . The union of the sets of possible solutions to 0 through is exactly the set of possible solutions to minus the solution . This means that the second-best solution to will be the best solution to one or more of the problems 0 through . Next, we nd the best problem-set/solution pair, , that contains unpartitioned problems. We then partition all the unpartitioned problems in , creating new problem-set/solution pairs and placing them into NEW-LIST. Any problem whose best solution is not as good as the -th best solution so far can be discarded, and need not be placed on NEW-LIST. At the end of each sweep, NEW-LIST is added to ANSWER-LIST. This entails combining sets of problems that have the same solution, removing duplicate problems, and then truncating ANSWER-LIST to length . This truncation throws out any old solutions that are inferior to solutions found during the sweep. The algorithm terminates when all the problems in ANSWER-LIST have been partitioned. The solutions that appear in ANSWER-LIST are the best solutions to 0 . Murty's algorithm is very similar to that of Danchick and Newnam. In fact, there is only one substantial di erence, highlighted with bold type in Figure (2), which is part of the partitioning process. As in DN, each , from a copy of , new problem, , in Murty's partitioning is created by removing one triple, the problem being partitioned. But then, before creating the next new problem, we remove from all the triples that contain y and all the triples that contain except for itself. Every possible solution to this modi ed version of , then, must contain , since and must appear in the solution and is the only remaining triple which contains them. This means that the set of possible solutions to will be disjoint from the set of possible solutions to each of the other new problems, while the union of their solution sets will still be the same as the set of solutions to the original P, minus the solution . Thus no two problems in ANSWER-LIST will have the same best solution. Murty's method of partitioning gives three substantial advantages over that of Danchick and Newnam. The obvious advantage is that we no longer need to manage sets of problems that have the same solution. So we can work with problem/solution pairs instead of problem-set/solution pairs, and all the bold highlighted steps of Figure (1) are unnecessary. A more important advantage is that Murty makes far fewer calls to the routine for nding the single best solution to assignment problems. This is because the DN algorithm often nds solutions that it has already found. Each time it does so represents a redundant call to the assignment routine. Murty only nds any given solution once, eliminating all these redundant assignment calls. Finally, the third advantage of Murty is that, in the average case, the size of the problem partitioned P
S
P
P
0
0
0
n
P ; P ; : : : Pn
n
S
S
P
S
P
0
0
P
Pn
S
P
P
0
0
Pn
< Q; S >
Q
k
k
k
k
P
0
< y; z; l >
Pi
P
P
z
P
< y; z; l >
< y; z; l > y
z
< y; z; l > 0
Pi
S
5
diminishes rapidly with every sweep [CM95]. In DN, the size hardly diminishes at all. The size of the problem partitioned has a substantial e ect on execution time, because it determines the sizes of problems sent to the assignment routine.
3 Experimental Comparison The execution time for both DN or Murty's algorithms is dominated by the time taken to solve assignments. Many di erent methods exist for nding the single best solution to an assignment problem [GK93], and the performance of DN or Murty is highly dependent on which method is selected. However, for a problem of a given size, execution time is approximately proportional to the number of calls made to the assignment routine, since each such call takes roughly the same time. To compare the two algorithms we rst examined the total number of assigment calls made. The performance of the two algorithms was compared on two problem sets. The rst was a collection of 20 dense assignment problems of size 20 20 with weights randomly chosen between 100. The second set consisted of 4 problems, of sizes 10 10, 20 20, 30 30, and 40 40, obtained from an electrooptical sensor simulation. In the simulation, a geostationary infrared detector monitors the launch of ninety ballistic missiles. Figure (3) shows the average number of assignments performed by both DN and Murty's algorithm for the random problem set as a function of . Figure (4) shows the number of assignments performed for each of the four other problems we studied. These gures show that the number of calls is the same for nding the rst and second best solutions, but, for greater than two, Murty requires dramatically fewer calls. Figures (3) and (4) indicate that Murty's execution times should be substantially better. In addition, it should be remembered that not only does Murty's algorithm examine fewer assignment matrices, but the average size of these matrices diminishes with each iteration. Figure (5) shows how the average size of problems partitioned decreases as a function of in Murty, while staying nearly constant in DN. The e ect of this di erence on relative execution time depends on the complexity of the assignment algorithm being used. To examine the relative e ect of diminishing problem sizes on execution time, we conducted timing trials on two di erent implementations of each algorithm. DN(JV) is a Fortran implementation of DN, written by Danchick and Newnam and employing Jonker and Volgenant's optimal assignment algorithm [JV87]. This assignment algorithm has worst-case performance of ( 3 ), and average-case performance which we observe to be a little worse than ( 2 ) on random problems. M(JV) is a C implementation of Murty's algorithm, written by Miller, which also employs Jonker and Volgenant's assignment algorithm. M(JV) includes a number of optimizations that dramatically improve performance, and are reported in a companion correspondence [MSC95]. However, because of di erences in languages, data structures, and heuristics, the execution-times of DN(JV) and M(JV) are not directly k
k
k
O N
O N
6
20 Random 20x20 Problems
DN 3
Average number of assignments
10
Murty 2
10
1
10
0
10
2
4
6
8
10 12 14 Number of solutions
16
18
20
22
Figure 3: Number of hypotheses examined by DN and Murty's algorithms as a function of for a randomly weighted 20 20 matrix. k
comparable. For better direct comparison, we implemented the two algorithms in C++, using the original Hungarian method of Kuhn [Kuh55], as M(H) and DN(H). The code for these two implementations is almost identical; in fact, the code for M(H) is exactly the same as that for DN(H), except that a few lines have been added to implement Murty's partitioning. Note that the use of the ( 4 ) Hungarian method increases the time di erence which results from diminishing problem size. The execution times for the random problem set are shown in Figure (6). The results for the other four problems are shown in Figure (7). 2 It is interesting to examine the execution times of DN(H) and M(H) in relation to the number of assignment calls they make (Figures (4) and (3)). As we might expect, the constant problem size employed in DN gives DN(H) an execution time that's linear with respect to the number of assignment calls it makes. But M(H)'s time to compute the two best solutions is noticeably lower than would be expected if it were a O N
2
The programs were executed on a 150MHz MIPS R4400 microprocessor.
7
10x10 Problem
4
Number of assignments
Number of assignments
10
3
10
DN
2
10
Murty
1
10
0
10
DN Murty
2
10
1
10
0
5
10 15 20 25 Number of solutions 30x30 Problem
4
5
10 15 20 25 Number of solutions 40x40 Problem
4
10 Number of assignments
Number of assignments
3
10
10
10
DN 3
10
Murty 2
10
1
10
0
10
20x20 Problem
4
10
DN
3
10
Murty 2
10
1
10
0
5
10
10 15 20 25 Number of solutions
5
10 15 20 25 Number of solutions
Figure 4: Number of hypotheses examined by Murty's and Danchick and Newnam as a function of for problem sizes ofa 10 10, 20 20, 30 30, and 40 40. k
linear function of the number of assignment calls. This results from the dramatic drop in average problem size for the rst few values of . After has reached three or four, problem sizes begin decreasing at a much slower rate, and execution time resembles a linear function of assignment calls. k
k
4 Conclusion We have shown the close relationship between DN and Murty ranked assignment algorithms and the striking performance dominance of the latter when more than a trivial number of ranked hypotheses are to be found. By replacing the original Hungarian method with the faster Jonker Volgenant algorithm, and performing several other optimization discussed in a companion paper [MSC95], we demonstrated that Murthy's algorithm, M(JV), has the potential to produce hundreds of ranked hypotheses in a fraction of a second for measurement-to-target assignment matrices with row and column dimensions also in the hundreds. The results described in this paper provide a compelling argument that the realtime data association problem has been solved. The historical trajectory of the solution began with Reid's [Rei79] sequential 8
20 Random 20x20 Problems
Average size of assignment problems
25
20
DN
15
Murty
10
5
0
2
4
6
8
10 12 14 Number of solutions
16
18
20
22
Figure 5: Average size of assignment problem as a function of . k
reformulation of More eld's multiscan batch 0-1 integer programming problem approach [Mor77]. The next milestone was Nagarajan et al's [NCS87] insight of collapsing real and clutter targets into one category. Danchick and Newnam [DN93] recognized that the Nagarajan et al formulation was in essence a classical linear assignment problem. Independently, Cox and Miller [CM95] discovered a very ecient ranked assignment solution algorithm in the operations research literature of the late 1960's and subsequently implemented a highly optimized version thereof [MSC95]. Complementary work by Collins and Uhlmann [CU92] also reduced the overhead associated with the validation step of the MHT to ( log ). It should be noted that by creating dummy triples , i.e. rows and columns, in the assignment problems, it is possible to handle ( ) false alarms, ( ) new tracks, ( ) track continuation/termination, and the case where the number of tracks is not equal to the number of measurements. As a result of this collection of work, an accurate low order polynomial time approximation to Reid's original and complete MHT algorithm is now feasible. To provide some historical context, TRW's 1989 implementation of Reid's MHT method ran for a week on a DEC VAX 11 785 to produce the 20 best hypotheses in a modest scenario of 20 target tracks and 20 measurements per simulated electrooptical sensor scan. Our e orts represent an improvement of about O N
< y; z; l >
i
ii
iii
=
9
M
20 Random 20x20 Problems
2
10
DN(H)
Average time in seconds
1
10
DN(JV) 0
10
M(H)
-1
10
M(JV) -2
10
-3
10
2
4
6
8
10 12 14 Number of solutions
16
18
20
22
Figure 6: Average run-times of Murty's and Danchick and Newnam as a function of on randomly weighted 20 20 cost matrices. k
seven orders of magnitude in six years and support the claim that \algorithms are technology". To be fair, two orders of magnitude can be attributed to the increased computing speed and memory of today's workstations; the remaining ve orders of magnitude are attributable to improved \algorithm technology".
Acknowledgements The authors thank Andrew V. Goldberg of Stanford University, and Satish B. Rao and Harold S. Stone of NECI for useful discussions.
References [Bro89] W. L. Brogan. Algorithm for ranked assignments with applications to multiobject tracking. IEEE J. of Guidance, 12(3):357{364, 1989. [BSF88] Y. Bar-Shalom and T. E. Fortmann. Tracking and Data Association. Academic Press, 1988. 10
20x20 Problem
2
Time in seconds
Time in seconds
10x10 Problem
10
DN(H) DN(JV)
0
10
M(H) M(JV)
-2
10
5
2
10
DN(H) DN(JV)
0
M(H)
10
M(JV)
-2
10
10 15 20 25 Number of solutions
5
2
DN(H)
0
DN(JV) M(H)
10
10
40x40 Problem
Time in seconds
Time in seconds
30x30 Problem
M(JV)
-2
10 15 20 25 Number of solutions
10
DN(H) DN(JV)
2
10
M(H) 0
10
M(JV) -2
10 5
10 15 20 25 Number of solutions
5
10 15 20 25 Number of solutions
Figure 7: Average run-time of Murty's and Danchick and Newnam as a function of for four problems of size 10 10, 20 20, 30 30, 40 40. k
[CM95] I. J. Cox and M. L. Miller. On nding ranked assignments with application to multi-target tracking and motion correspondence. IEEE Trans. on Aerospace and Electronic Systems, 32(1):486{489, 1995. [CU92] J. B. Collins and J. K. Uhlmann. Ecient gating in data association with multivariate distributed states. IEEE Trans. Aerospace and Electronic Systems, 28(3), 1992. [DN93] R. Danchick and G. E. Newnam. A fast method for nding the exact -best hypotheses for multitarget tracking. IEEE Trans. Aerospace and Electronic Systems, 29(2):555{560, 1993. N
[GK93] A. V. Golberg and R. Kennedy. An ecient cost scaling algorithm for the assignment problem. Technical report, Stanford University, 1993. [JV87]
R. Jonker and A. Volgenant. A shortest path algorithm for dense and sparse linear assignment problems. Computing, 38:325{340, 1987. 11
[Kuh55] H. W. Kuhn. The Hungarian method for the assignment problem. Naval Research Logistical Quarterly, 2:83{97, 1955. [Mor77] C. L. More eld. Application of 0-1 integer programming to multitarget tracking problems. IEEE Transactions on Automatic Control, AC-22(6), June 1977. [MSC95] M. L. Miller, H. S. Stone, and I. J. Cox. Optimizations to Murty's algorithm for determining a ranked set of assignments. Technical report, NEC Research Institute, 1995. [Mur68] K. G. Murty. An algorithm for ranking all the assignments in order of increasing cost. Operations Research, 16:682{687, 1968. [NCS87] V. Nagarajan, M. R. Chidambara, and R. N. Sharma. Combinatorial problems in multitarget tracking - a comprehensive survey. IEE Proceedings, Pt F, 134(1):113{118, 1987. [Rei79] D. B. Reid. An algorithm for tracking multiple targets. IEEE Transactions on Automatic Control, AC-24(6):843{854, December 1979. [Ull79] S. Ullman. The interpretation of visual motion. MIT Press, 1979.
12
