Tracking and identification algorithms have been developed to track moving targets using ... is turning, it may be temporarily stopped in a static-rotator case.
An Information-Theory Based Feature Aided Tracking and Identification Algorithm for Tracking Moving and Stationary targets through High Turn Maneuvers using Fusion of SAR and HRR Information Erik. P. Blasch Wright State University, Dayton, Ohio
ABSTRACT Tracking and identification algorithms have been developed to track moving targets using high-range resolution (HRR) radar. Likewise algorithms exist to link moving target indicator (MTI) hits with synthetic aperture radar (SAR) images to follow targets that are in a move-stop-move scenario. Each of these algorithms have limitations in their abilities to maintain a consistent track of an object that is in a move-stop-maneuver scenario. For example, (1) there is only spatial information from which to link MTI hits with SAR data and in the other case (2) HRR track and ID algorithm would not capture stationary targets. Fusing these modes can provide for track consistency. When multiple targets exist, such as in a group tracking scenario, the spatial information to link moving and stationary returns would be difficult. The incorporation of moving HRR classification and stationary target identification information would enhance the MTI-SAR linking algorithm for multiple targets. While HRR is better suited than MTI for linking object track information to spatially stationary SAR information, the difficulty with relying solely on HRR information is that when a target goes through a maneuver in which it is turning, it may be temporarily stopped in a static-rotator case. This paper discusses an information-theory approach for identification of targets in 1-D HRR and 2-D SAR modes and its incorporation into a feature-aided tracking and ID algorithm for tracking a target which goes through a stationary, moving, and maneuvering dynamics. Results are presented for a group of highly maneuvering targets which travel in one direction, turn, and travel in another direction from which typical kinematic tracking algorithms based on HRR information would break down. Keywords: Information Fusion, Group Tracking, Target Identity, Registration, MTI, HRR, SAR
1. INTRODUCTION Without successful adaptive multisensor fusion or online registration techniques, automatic target recognition (ATR) and tracking algorithms are prone to poor object classification and tracking. Multisensor fusion for a given situation assessment includes identifying measurement information for task completion and reducing measurement uncertainty in the presence of clutter. By extracting synthetic aperture radar (SAR) image and high range resolution (HRR) radar signature informational features, data registration, and target classification and tracking is achievable. This paper examines SAR and HRR information-theoretic features for target orientation and proposes a method for target classification and tracking for a movestop-maneuver scenario. Multisensor ATR algorithms include target classification as a subset of sensor management. Sensor management includes selecting sensors, sensor detection and recognition policies, and classification algorithms for a given set of mission requirements [1]. For example, a typical tactical aircraft carries an onboard active radar sensor that outputs physical measurements that can be used to 1) synthetically generate an image – a SAR profile, 2) collect blobs from and MTI for point-to-point connections, and 3) return HRR signatures from moving targets. These radar modes can be utilized for kinematic and identity estimates to detect, recognize, identify, classify, and track objects of interest while reducing pilot workload. In a complex environment, the onboard sensor manager must select the correct sensor or sensor mode to measure the correct object at a given time. Thus, the sensor manager must control the measurement sequencing process. A sensor manager process can be described as a problem in sequential decision making under uncertainty.[2] Prominent elements of the
256
Algorithms for Synthetic Aperture Radar Imagery IX, Edmund G. Zelnio, Editor, Proceedings of SPIE Vol. 4727 (2002) © 2002 SPIE · 0277-786X/02/$15.00
Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
problem include a knowledgeable competitor, a dynamic environment with uncertainties in target orientation and measurement clutter, and complexity arising from many possible sensor actions and outcomes [3]. From an ATR [4] point of view, geometric target information invariants are essential to linking features, geometry, and sensor to sensor registration. With invariants, image properties of geometric transformations on objects such as affine transformations or sensor projections from the 3D world to the sensor output space are intrinsic to the object's structure. These invariants are not dependent on the particular choice or realization of transformation parameters. Thus, we advance the 2D info-theoretic method [1] to 1D signatures over time. Additionally, by fusing multiple modalities, it is similar to human capabilities over 1D and 2D feature extraction and fusion.[5] Since the sensor manager can not control the geometric perspective of an object in the environment, it can simplify the object recognition task by determining information invariants. Most invariance work as applied to computer vision has focused on geometric invariants or 3D structure properties that are invariant to the sensing geometry. Typically these are expressed as theorems on distances or angles measured on point and line sets that are constant over the orientation space. Recent work has focused on photometric invariants, or invariants to the spatio-temporal object signature. Examples include color distribution and hyperspectral signature, EO/IR, or RF polarization invariants. The association analogy we explore is information features to imaging geometry for object orientation. We seek object properties, as measured by spatial/spectral intensity of a SAR, HRR, and tracking output, that are not dependent on the dynamics of the signature formation process. The information can thus be used for 2D, 3D, and possibly 4D registration [6]. This paper assesses uncertainty management for the target classification. In sensor-target classification, a sensor is directed to perform a sequence of measurements that isolate a target. The challenge is to guide the sensor so it identifies the orientation of the target “efficiently”. The work reported here follows that of Blasch [4] who investigated how learning can be used for searching and detection. The methodology follows that of information theory to determine the content of a SAR image or an HRR signature and the algorithm is evaluated using information-theoretic constraints to assist in identifying a target in a track and ID scenario where the groups of targets [7] can move, stop, or maneuver in place.
2. PROBLEM FORMULATION Feature extraction can be used for object tracking, identification, and classification [1]. For tracking, image content and registration are important for time and location referencing. Additionally, ATR algorithms are subject to uncertainty measurements and capacity constraints. For instance, if the image is thought to be a communication channel, then the desired output is to maximize the information available to the pilot, given bandwidth and time constraints. Consider Figure 1 as an illustration of an environment that the pilot is monitoring. By assumption, the aircraft carries a single sensor able to detect ground stationary and moving targets. Assume that the region of interest in the 2-D frame, is composed of K cells, which surround a single static target, shown in Figure 2 as the observation space. Any cell in the frame can be measured independently of the others, and the outcome of each measurement is a random variable indicating the magnitude of the cell. The probability density of each measurement depends on whether the target is actually present or not. Further assume that a fixed number of M measurements will be taken from an observation using entropy metrics. Likewise, we can assume the k cells are linearly connected
Detect
Target
Object
Recognize
Tank Classify Type
Identify T72
Allegiance
Intent
Behavior
Friend, Foe, Neutral
Threat - Shoot, Articulation
Movement
Figure 1. Object Classification. Proc. SPIE Vol. 4727
Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
257
in a signature framework. Thus, what can be done in 2D can be completed in 1D where one dimension is n and the other dimension is one. Combining these entropy metrics in 1D or 2D allow for a decision to be rendered as to which orientation the target is in. The assumption is that the target type is known a priori and the orientation information will further help reference target features for classification or target features can be registered from one measurement to the next measurement. Learned-observation information metrics are considered stored in memory and the ATR algorithm is to compare the returned SAR image or HRR signature measurements to a known database. Also, for moving targets, a measurement compared to trained data set can assist in the track heading angle. The target identification problem is to determine what sequence, minimal necessary number, and type of cells features [1] to measure. These actions should provide the highest probability that the target orientation will be isolated. After M measurements and comparisons to O observations, a measure of mutual information [8] will determine the informationtheoretic feature content. If a threshold is achieved, a preliminary orientation is determined which allows the classification routine to extract features for initial target matching and subsequent ID analysis. The steps are outlined in Figure 2.
Figure 2. Mutual Information for Target Recognition. The primary features extracted, shown in Figure 3, are mutual information on target orientation and identification. Orientation aides in tracking kinematics and identification helps in track-track data association. The motivation for mutual information is that 1) it utilizes the measured probability density function of cells, 2) it can easily be adapted for invariance assessment across different sensor modalities, and 3) orientation information can be used for image registration and the classification process. The mathematical model for static-target detection, feature recognition, and orientation determination, are presented in the next section. First, a selection needs to be done to determine which radar collection mode to prosecute a target. Since the decision for the sensor choice is determined a priori, a prediction has to be determined as to whether the Figure 3. Sensor mode selection. 258
Proc. SPIE Vol. 4727
Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
target is moving or stationary. If the target is moving then either a moving target indicator (MTI) mode or a high range resolution radar mode (HRR) should be selected. However, if the target is stationary, a synthetic aperture radar (SAR) map can be taken as the radar is moved around the target. In either case, the best results for target detection are when the correct radar mode is selected for the given target characteristics. If the incorrect choice is made, the entropy information will indicate that the wrong choice is made and the sensor manager can select another mode in the subsequent collection process. 2.1 An information-theoretic Model for Static-Target Feature Detection The identification of a target can be achieved by maximization of mutual information, which has been described by Viola and Wells [8]. The goal is to obtain a learned estimate of the association, A, that associates the target-measured orientation, T = f(k), and the detected target observation O by maximizing their mutual information over association estimate: ^ A = max A [I{T(x), O(A(x))} ]
(1)
where x is a random variable that ranges over the cropped visual image, shown in Figure 4, or a target signature Mutual Information, defined using entropy, is I{T(x);O(A(x)} = h(T(x)) + h(O(A(x)) - h(T(x),O(A(x))
(2)
where h( ) is the differential entropy of a continuous random variable, and is defined as: h(x) = - ó õ px(x)(log(px(x)) dx
(3)
Given the random variable measurements in an image, information on a referenced x, y information can be used as a feature of orientation, or independently as length and width. The joint entropy of two random variables x and y is h(x,y) = -ó õó õ p(x,y)(x,y)(log(p(x,y)(x,y)) dx dy
(4) Figure 4. Detection and Image Cropping for the Extraction of features.
Entropy can also be expressed as: I{T,O(A(x))} = h(O(A(x))) – h(O(A(x)) | T(x))
(5)
and h(y|x) is the conditional entropy which is interpreted as a measure of uncertainty, variability, or complexity. Information, in the association problem, is divided into three characteristic functions: 1) Entropy of the target, independent of A, 2) Entropy of the image which the target is associated with, and 3) The negative joint entropy of the observation O with the Target T. A large negative value exists when the target and the observation are functionally related. Basically, it learns associations where the observation O explains the target T orientation above a desired threshold. Hence, functions (2) and (3) are learned associations for complexity reduction.
40
35
30
25
20
Viola used a stochastic gradient descent method to seek a local maxima of the mutual information criterion. The method employs histograms to approximate entropies and their derivatives. For this paper, we utilize the histograms to approximate the entropies, as in Figure 5.
15
10
5
0 0
50
100
150
200
250
300
Figure 5. Histogram of Probabilities. Proc. SPIE Vol. 4727
Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
259
The sensor-target orientation-classification problem can be formulated as a multiple-hypothesis testing problem [2]. The mathematical algorithm for measurement processing is similar to a system with independent hypotheses. Each individual hypothesis test, denoted Hk, is referenced to the Oth image observation and simply states, “the observation contains a target orientation feature.” H hypotheses are postulated, one for each feature f = 1,…, F, as in Figure 6, of which we concentrate on orientation and classification in this analysis. In I -1 images, Hk is false; and in one image Hk is true. Let k denote the stage of the detection, where k = 0,1,…, K. At every stage k > 0, a sensor makes a measurement and compares to an observation in image I. By convention, let the feature measurement outcome y(t) = 1 denote a perfect orientation or pose correlation and y(t) = 0 denote no detection. Measurements, which are independent from stage to stage, have a probability density that is conditioned on the presence or absence of the feature and depends on the probabilities of false alarm and missed detection. Let I(t) = {(i(s), y(s)), s = 0,…,k} be the total information available at stage k, consisting of (i, y) measurement pairs, i(s) being the sample feature and y(s) the realized measurement at each epoch through stage k for the feature f.
5 10 15 20 25 30 35 5
10
15
20
T
Now let Bel(f) = [Belk(t)] = [f1(t), f2(t), … , fK(t)] denote the vector Figure 6. Features – Round Top. of conditional probability of feature estimates for the combination of cells in the frame. The summed element of Belk(t) is the total conditional probability that Hk is true given the accumulated feature measurements in cells k through stage K, i.e. Belk(t) = P(Hk | I(t)). Denote Bel(0) = [Fk(0)] as the vector of initial probabilities. Assuming that feature hypotheses are independent across images, values measured in cell k affect that image’s pose hypothesis and no other. The independent assumption allows multiple hypotheses to be true at the same time. Focusing on feature f, two cases present themselves corresponding to whether the measurement of feature f +1 is useful for classification or not. Bayes' Rule governs the assimilation of the measurements, where Belj(t) is our estimate for the conditional probability of feature f before the measurement in f is processed: Detection: Belo (f + 1) = P(Ho(f) | I (f + 1))
(6)
= P(Ho(f)|(i(f+1) = j, y(f+1) = detection), I(f)) = P(feature in o | detection of o, I(f)) =
P(detection of o | target in o)•P(target in j | I(f)) + P(detection of o | no target in o) • P(no target in o | I(t))) P(detection of o | target in o) • P(target in o | I(f))
By analogy with the above equation, Bayes' update of Belj(f) for the case when the sensor does not report a target is: No detection: Belo (f +1) = P(target in o | no detection of o, I(f))
(7)
Note in general that the sum of Belk(f) values across all I images is not unity. Likewise, we can do the same analysis for a 1 x n image or signature. 2.2 A Model for Moving-Target Feature Detection The HRR belief classification algorithm is based on the statistical behavior of extracted features and is similar to the statistical feature-based belief processing by Blasch [1]. The features, falT, consist of, a, salient peak amplitudes, l, feature
260
Proc. SPIE Vol. 4727
Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
peak location, and T the target profile as shown in Figure 7. Target class hypotheses are defined as the set Ω = {T1, T2,..., Tm} for an algorithm trained on m target classes. The location data are represented by L = {l1, l2,…, ln} and the peak amplitude data by A = {a1,a2,...,an} for n extracted peaks from an observed target signature. The extracted peak information from an observation is determined in real-time with priorities, αal, being placed 1.8 on the salient number of location features. The basic 1.6 statistical modeling concept is to estimate the probability 1.4 1.2 that a peak occurs in a specific location lj, given that the Magnitude 1 Peak (a) observation is from target Ti. Further, the probability 0.8 0.6 that the peak has amplitude aj, given that the peak is at 0.4 the location lj and that the observation is of target Ti , 0.2 00 50 100 150 200 must be determined. Using the peak information in an information theoretic analysis, we have a measure of Range Bins - (l) mutual information to fuse over time for target IDs. Figure 7. High Range Resolution Profile, showing amplitude(a) and location (l) of range bins.
3. ORIENTATION IDENTIFICATION/DETECTION METHODS Direct detection is an uninformed method which devotes equal attention to every image. The procedure is to choose a starting Image I* and advance through the frame in the same cell order each time, taking one noisy measurement per cell, processing it to update Belk(t) for that image, and then advancing to the next image in the predetermined cell sequence. When the frame is completed, the pattern is repeated, starting over with the image I*. A run is completed when O observations have been compared and processed. In order to ensure equal numbers of observations in each image, T is chosen to be a multiple of K so only complete frames of measurements occur. Likewise, the image may be a signature with an area of 1 x n. With the association rule procedure, it is likely that some images and cells will not be measured during an entire run since belief associations will exceed orientation and classification thresholds. The measurements not utilized in these images are expended in images where false alarms occur and in the target image which tends to absorb every measurement once its Bel(t) value gets large enough to surpass a confidence level. 3.1 A Model for Moving and Stationary -Target Feature Detection Figures 8 and 9 show the case where track information, assuming constant updates is confused with a static target. By using a mutual information measure, we can determine when a target is stopped and correctly take a SAR map and stop the track. Figure 8 shows the case in which two tracks are crossing and one of the targets goes from a moving state to a stopped state. In the case that only detection occurs, the filtering track algorithm would continue to predict the path of the target until there was no detection. In the case that a signature return would result, eventually, the track would lose detection capability and get lost. The solution to the problem, shown in Figure 9, shows that the we go from a detection mode to an identification mode, or at least a classification of the target. By using a measure of mutual information, we would be able to tell from the moving mutual information metric that the target might have stopped. When the target stops, the SAR mode is cued and the correct mutual information metric, below some threshold, we can verify that the target has stopped as having been identified with the correct radar mode. Thus, the track, defined over moving and stationary transitions, would be complete. There are also cases in which the target is doing a maneuver. In the case of a maneuvering target, such as a rotation, it would appear to stop according to the tracking algorithm. However, the rotation of the target would confuse the SAR map collection and the ID would be a low probability. Thus, the combination of the moving and stationary track and ID system would be able to detect the target, but not identify it until it continued moving. Thus, the methodology can not only do moving and stopping targets, but can incorporate maneuvering targets using a fusion of the tracking and ATR algorithms.
Proc. SPIE Vol. 4727
Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
261
Figure 8. Tracking Problem – Target Stops
Figure 9. Choose Correct Sensor
3.2 Invariant Feature Detection for stopped or maneuvering targets Invariance involves the problem where a feature is available but a specific match has not been concluded as to the target and the feature measured. As in the case when two objects are detected: the larger object (with an associated length to width ratio) and a signature (which appears with a smaller length-to-width ratio) can be assessed with the HRR modality to delineate the target movement. Since these two objects are detected together, they can be clustered together. Assuming that scaling has been performed correctly, it is a function of the invariance parameter to determine which affine transformation the object is in to choose the correct sensor for direct measurement for ID and indirectly determine the target speed. The mutual information methodology includes searching a known database for a match between the invariant feature and the detected feature. For example, the length of a target is size invariant with respect to SAR and HRR information in one dimension, such as length, but not in width. Additionally, since the pose and aspect angle of the target may vary, the algorithm must not only identify the target, but also the aspect ratio of the target when it is in different articulations. Insensitivity involves the correct association of the feature to the target type. For instance, when the aspect ratio of orientation is determined, the new domain of information is the aspect ratio as opposed to the measured data itself. Thus, the new information is insensitive to the measured 2D or 3D information. The goal is then to determine, within this new domain, what orientation the vehicle is in and a course measurement of speed.
4. RESULTS The dynamic detection and feature association methods are compared for the mutual-information content. The method for evaluating performance is Monte Carlo simulation and the performance metric is probability of error PE(k). Better performing methods will exhibit lower PE(k)’s for equal numbers of measurements, i.e., for equal expenditure of measurements. A designation error PE(k) occurs at time k whenever the probability for a target-pose in the image or signature is not the largest probability in observed or predicted set. Defining the image with the correct target pose as image-cell I(ktgt), the symbolism that prescribes the probability PE(k) of experiencing a designation error is:
PE = P(arg kmax Belk (T) ≠ ktgt)
(8)
PE(K) is a global error metric that looks at the entire set of detection-observation pairs to produce a single declaration decision of target classification.
262
Proc. SPIE Vol. 4727
Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
The standard test problem is to find a single stationary dim target placed randomly in a image with a given orientation. Here, we include moving data from groups of targets before a single target becomes stationary. Assuming that 1) a single sensor is available to detect each target, 2) pose truth information is available, and initially 3) each image measurement is equally likely to contain the correct orientation, a Monte Carlo run is conducted for T = 360 images with 128x128 measurements when cued by the tracker. During a run, an agent chooses the image where measurements occur guided by the classification policy of the method under study. Each study consists of a sufficient number of runs to allow PE(k) values to stabilize. Finally, both the classification and learned feature associations methods are implemented in MATLAB® as shown in Figure 10.
Figure 10. Level 5 – User Interface – That displays the HRR, SAR, and MTI tracking data. There are performance criteria where one metric is used for each target. PE(k) is the dependent variable and O the independent variable, O being the number of images or signatures. Results [1] show that the use of mutual information in the learned-association detection algorithm is an intelligent strategy for referencing the target so that recognized features can be used to track and classify the target. Fewer measurements are needed with orientation detection; however, a minimal set of features are needed to classify the target under the learned-association rule. In the table below, we show the results for both the 2D static and the 1D moving case. Note the 2D information has a higher resolution (more data points) by incorporating a larger separation between the entropy values in which to do a target identification. Below is a sample of information-theoretic orientation and classification features that can be used to determine a target’s orientation and point-numerical solutions. Table 1.Orientation information (Target Az = 240.8°) Stat H_x H_y H_xy H_x|y H_y|x I_xy D_p|q
SAR Processing 5.182 - Horizontal 4.567 - Vertical 9.534 – Joint Entropy 4.967 – Conditional Entropy 4.351 – Conditional Entropy 0.215 – Mutual Information 0.027 – Relative Entropy
HRR Processing 3.726 - Horizontal 2.154 - Vertical 6.738 – Joint Entropy 2.881 – Conditional Entropy 2.157 – Conditional Entropy 0.198 – Mutual Information 0.006 – Relative Entropy
Fused HRR and SAR 4.214 - Horizontal 3.769 - Vertical 8.249 – Joint Entropy 3.675 – Conditional Entropy 3.897 – Conditional Entropy 0.201 – Mutual Information 0.012 – Relative Entropy
Features, such as the target size, are determined as a collection of clustered high probability returns which are used to determine the target classification. Since only one training and testing data set was run, the probability of correct classification was high, although, it might diminish when different target images or signatures are used or clutter is increased.
Proc. SPIE Vol. 4727
Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
263
Using the moving and stationary data, as shown in Figure 11, a group tracking scenario was conducted [7]. In this case, we see that the left circle represents a move → stop scenario. The circle on the right shows a high maneuver case of the target. In each case, a consistent and stable track is maintained as well as a consistent and stable ID.
5. CONCLUSIONS The research included methods to orient and classify moving and stationary targets using an information theoretic measurement. In a series of simulation experiments, the learned-feature association uncertainty algorithm performed well, resulting in a desirable solution to maintain a consistent and stable track of the targets. The learned association method presented (i.e. reducing uncertainty) is more efficient than a direct detection method which is used in a target tracking analysis. Further research will focus on a combination of track techniques for classification problems and exploration in problems involving human selection of sensors and choice of uncertainty probability distribution functions to assess the fusion gain.
Figure 11. Group Tracking Move-Stop-Maneuver Results using the Mutual Information Metrics
6. REFERENCES [1] E. P. Blasch, Derivation of a Belief Filter for Simultaneous HRR Tracking and ID, Ph.D. Thesis, Wright State University, 1999. [2] D. A. Castañon "Optimal Detection Strategies in Dynamic Hypothesis Testing," IEEE Transactions on Systems, Man, And Cybernetics, Vol. 25, No. 7, July 1995, pg. 1130-1138. [3] E. Blasch, R. Malhotra, and S. Musick, "Using Reinforcement Learning for Target Search in Sensor Management," IRIS National Symposium on Sensor and Data Fusion, Boston, MA, April 12 - 18, 1997. [4 ] E. Zelnio and F. Garber, "Characterization of ATR performance Evaluation," SPIE Signal Processing, Sensor Fusion, and Target Recognition V, Orlando, FL, April 8 - 10, 1996. [5] E. Blasch and J. Gainey Jr., "Feature-Based Biological Sensor Fusion", 1st IEEE Conference on Sensor Fusion, Las Vegas, NV, July 6 - 9, 1998. [6] M. Leventon, M. Wells, and W. Grimson, “Multiple View 2D-3D Mutual Information Registration,” Proceedings 1997 Image Understanding Workshop, New Orleans, LA, May 11 – 14, 1997. pg. 625-629. [7] E. P. Blasch and T. Connare, “Group Tracking of Occluded Targets”, SPIE 4365, 2001. [8] P. Viola and M. Wells, "Alignment by Maximization of Mutual Information," The International Conference on Computer Vision, June, 1995.
264
Proc. SPIE Vol. 4727
Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
