Proceedings of EuroFusion98, M. Bedworth, J. O’Brien (eds.), pp. 55-60, Great Malvern, UK, 1998
Sensor Data Fusion for Anti-Personnel Land-Mine Detection Frank Cremer, Eric den Breejen and Klamer Schutte, TNO Physics and Electronics Laboratory, PO Box 96864, 2509 JG The Hague, The Netherlands, Email:
[email protected] Abstract - In this paper five different sensor data fusion options for the detection of anti-personnel land-mines are presented. The performance of these options are evaluated in a case study of estimated figures for mines and possible false alarm generating objects. Dempster-Shafer with proper mappings and rule based fusion provided the best result and an improvement over single sensor solutions. The data processing for each sensor is performed separately by the manufacturer. Each sensor returns (on a grid of 2.5 cm by 2.5 cm) a figure of merit (FOM) for detection of mines. This information is processed in the sensor data fusion process resulting in something similar to a probability map. This map, along with pre-processed data from different sensors, supports the operator to make the decision.
1. Introduction Sensor fusion is the process in which information from different sensors is combined. This can be especially advantageous, when sensors measure independent physical properties: weaknesses of one sensor are compensated by inherent strengths of other sensors, resulting in good performance of the complete sensor suite in a wide variety of conditions. In the GEODE (Ground Explosive Ordnance DEtection system) project, there are three sensor types based on different physical principles, all with their own strengths and weaknesses.
2. Sensor Fusion Options In the following section various sensor data fusion options studied in the GEODE project are introduced.
For evaluation of the various sensor fusion options realistic data is needed. Currently these data are not yet available so a case study of various mine-like and possible false alarm generating objects is presented instead. Estimates have been provided of sensors responses in various situations to various objects.
2.1 Dempster-Shafer For each of the sensors, three propositions are defined. The first one is the proposition that there is a mine (denoted by m); the second proposition states that the sensor senses background (denoted by b). The last proposition is that it is either a mine or background (denoted by m∪b). This last proposition expresses that there is something it cannot classify and thus gives the uncertainty of classification.
Within the GEODE project, TNO is responsible for the development of advanced fusion software for multi-sensor detection, localisation and classification of AntiPersonnel Landmines (APL). The partners in the GEODE project and the sensors they provide, are listed below:
To each of the propositions or to a union of propositions a probability mass is assigned. Probability mass is determined from the sensor’s ability to assign some certainty to a proposition on the basis of evidence, see also [2]. These masses are assigned as follows: msensor(proposition), where proposition is one of the above and sensor is either Ground Penetrating Radar (GPR), Metal Detector (MD) or InfraRed camera (IR). The sum of these three probability masses equals 1, since the propositions are exhaustive.
• EMRAD Ltd: Ground coupled ground penetrating radar. • Förster: Dual frequency, continuous wave metal detector. • IAI/ELTA: High clearance ground penetrating radar. • Marconi Spa: Two infrared cameras, wavebands 3-5 µm and 8-12 µm. • Dassault Electronique: provides a digital elevation map with a Sick laser telemeter.
55
Proceedings of EuroFusion98, M. Bedworth, J. O’Brien (eds.), pp. 55-60, Great Malvern, UK, 1998
The masses assigned to the propositions for the sensors are combined, resulting in overall probability masses. This combination is carried out by using the Dempster-Shafer rule: “The product of two masses is assigned to the intersection of the two propositions”.
on the theory of fuzzy numbers, which are fuzzy sets, see [4]. Fuzzy probabilities can be used for statements like “the probability of something is around 0.75”, indicating that this probability may be a little higher or a little lower than 0.75. By choosing the right membership function for this probability, one can define the certainty that the probability is 0.75. In Figure 2.1, the crisp probability and the fuzzy probabilities are plotted. 1
0.75
1
Crisp probability
0.5 0.75
1
Fuzzy probability
Figure 2.1: A crisp possibility of 0.75 on the left and a fuzzy probability of 0.75 on the right.
K2=1-mGPR(b) mMD(m) - mGPR(m) mMD(b).
Given the fuzzy probabilities on a mine, the fuzzy joint probability is defined as the intersection of the fuzzy probabilities. The result is a fuzzy probability, but a crisp decision is needed. One can “de-fuzzify” any fuzzy sets using a number of defuzzification methods. One method of defuzzification is the Centre Of Area (COA), dividing the area under the membership functions into two equal halves.
mGPR(m) m2(m) Ø m2(m) mGPR(b) Ø m2(b) m2(b) mGPR(m∪b) m2(m) m2(b) m2(m∪b) mMD(m) mMD(b) mMD(m∪b) Table 2.1: The mass product matrix of the GPR and the MD. The same procedure can be followed to combine the masses of m2 and mIR to derive the overall probability masses.
For GEODE, a translation of confidence levels to fuzzy probabilities is needed. The results of the translation from confidence level to masses needed for Dempster-Shafer is used. A Gaussian kernel as the membership function with its width equal to the uncertainty is used.
2.2 Bayesian methods The theory of decision making in context says that from all statistical methods the Bayes method performs best. However, proper use of Bayes means that joint probability density functions must be known, unfortunately these were not available in the GEODE project. Assuming independence of sensors, the joint probability on a mine given the observations from the GPR, MD and IR camera is given by: P(m| GPR, MD, IR) =
1 membership function
membership function
In Table 2.1, the masses assigned to propositions of the GPR and the MD are combined into m2. Some of the intersections between two propositions are empty (denoted by Ø), because they correspond to mutual exclusive hypothesis that are physical impossible combinations. This means that the total sum of the masses does not equal 1. To correct for this effect, all masses (except Ø) should be divided by K2. K2 is defined as one minus the sum of the masses otherwise assigned to Ø. In this case K2 is defined as:
2.4 Fuzzy rules The membership function of fuzzy sets can be interpreted as a possibility distribution. This differs from a probability distribution in the sense that it does not indicate the relative number of occurrences within a given measurement, see also [3].
P(m| GPR) P(m| MD ) P(m| IR) ⋅ ⋅ ⋅ P( m) P( m) P( m) P( m)
There are two ways of combining possibilities: using the “and” or “or” relationship, which are defined as the minimum and the maximum operator respectively. For detection of mines, a set of rules is derived, with each rule combined using the “or” relation. Each rule consists of the “and” combination of the FOMs of the sensors.
Note that the prior probabilities, P(m), are constants and do not influence the decision; they may be absorbed into the threshold.
2.3 Fuzzy probabilities A way to incorporate uncertainty into the sensor fusion is by the usage of fuzzy probabilities, see also [1]. This theory is based
Using this setup weighing factors need to be introduced for the FOMs. To obtain a rule set
56
Proceedings of EuroFusion98, M. Bedworth, J. O’Brien (eds.), pp. 55-60, Great Malvern, UK, 1998
2
Set 1 2
f(measurement)
masses
sensor dimension
FOM
Figure 2.2: From sensor measurement via FOM to probabilities for Bayes (top) and mass functions for Dempster-Shafer (bottom). Similar mapping should occur for other fusion paradigms. Each sensor has a different mapping function, which may influence the overall decision process. The mapping functions from confidence to fusion algorithm input are restricted in the sense that they are monotone rising or monotone descending. The exact shape of the mapping function is related to the performance of the sensor under varying conditions and the fusion algorithm used.
The rules for the cases in the case study are given in Table 2.2. The mapping types A and B are explained in the next section. Set 1
FOM
FOM
Sensor
1. Create an empty rule set. 2. Check for each mine in the list whether this mine is detected by an existing rule. 3. If no existing rule detects the current mine, add a new rule to the rule set. 4. Check for all rules whether the new added rule completely covers an already existing rule. 5. Delete all rules completely covered, by the new added rule, from the rule set. 6. Continue checking for each mine in the list.
measurement
probability
of detection, which minimises the number of false alarms, the following steps were made:
Mapping type A (PGPR(m) / 0.55) ∪ (PGPR(m) / 0.50 ∩ PIR(m) / 0.65) (PGPR(m) / 0.8) ∪ (PGPR(m) / 0.7 ∩ PIR(m) / 0.8) ∪ (PGPR(m) / 0.5 ∩ PMD(m) / 0.65) Mapping type B (PGPR(m) / 0.55) ∪ (PGPR(m) / 0.5 ∩ PIR(m) / 0.7) (PGPR(m) / 0.8) ∪ (PGPR(m) / 0.7 ∩ PIR(m) / 0.85) ∪ (PGPR(m) / 0.5 ∩ PMD(m) / 0.7)
In the following case study, estimates of the Dempster-Shafer probability masses are given. Two different mappings have been tested: A) use mass assigned to background as background and B) use this mass as uncertainty mass. Mapping B could be useful as MD and IR cannot determine that no mine is present: non-metal mines are impossible for MD to detect, and buried mines (without surface disturbance) cannot be detected with IR. As such, assigning mass to the background mass rather than to the uncertainty mass can give false indications to the fusion algorithm. For reasons of symmetry, one also could map the mass assigned to background of the GPR to the uncertainty mass. Experiments showed that this mapping decreases the detection performance.
Table 2.2: Rule sets 1 and 2 for mapping type A and B.
2.5 Mapping of figure of merit In the previous sections, probability masses, conditional probabilities and fuzzy probabilities were defined for the different sensor fusion paradigms. This raises the question how these can be related to sensor measurements. In the GEODE consortium it was agreed that, after processing, each sensor gives per grid cell a FOM. This FOM should be low when the grid cell most probably corresponds to background, and high when it is likely to correspond to a mine; it is a number between 0 and 1.
3. Case Study To evaluate the various sensor data fusion options a case study of expected sensor responses for mines and possible false alarm targets was performed.
Figure 2.2 depicts the process of mapping sensor data onto the required input for the fusion algorithm. The first step is to acquire raw measurement. Next, a function maps the raw data to a FOM. This FOM indicates the confidence of detecting a mine and is the agreed input from the sensors into the fusion process. This FOM is subsequently mapped to the specific type of input needed for the fusion algorithm used.
For the Bayesian approach and the fuzzy rules, the conditional probabilities for sensor S are calculated by: P(m|S)=mS(m)+0.5mS(m∪b). This means that the uncertainty mass is divided between the probability of a mine given an observation and the probability on background 57
Proceedings of EuroFusion98, M. Bedworth, J. O’Brien (eds.), pp. 55-60, Great Malvern, UK, 1998
depth influenced mostly the IR masses and to some degree the GPR masses.
given the same observation. No clear theoretical basis exists for this division. However, experiments performed with a different factor than 0.5 did not show significant different results.
3.2 Possible background
In Table 3.1, 18 different mines and conditions are given. Each mine is given an designation; for instance the first mine has the designation “SNN”.
SNN SNB SNS SLN SLB SLS SHN SHB SHS LNN LNB LNS LLN LLB LLS LHN LHB LHS
Debris Brick N Brick B Brick S Can N Can B Can S Nail N Nail S Print P.cone S P.cone B Backgr.N Backgr.B
GPR m(m m(b m(m∪b 0.2 0.1 0.7 0.2 0.1 0.7 0.1 0.1 0.8 0.2 0.1 0.7 0.2 0.1 0.7 0.1 0.1 0.8 0.2 0.1 0.7 0.2 0.1 0.7 0.1 0.1 0.8 0.7 0.1 0.2 0.7 0.1 0.2 0.5 0.1 0.4 0.7 0.1 0.2 0.7 0.1 0.2 0.5 0.1 0.4 0.7 0.1 0.2 0.7 0.1 0.2 0.5 0.1 0.4
MD IR m(m m(b m(m∪b) m(m m(b m(m∪b) 0.1 0.7 0.2 0.1 0.6 0.3 0.1 0.7 0.2 0.2 0.1 0.7 0.1 0.7 0.2 0.4 0.1 0.5 0.5 0.1 0.4 0.1 0.6 0.3 0.5 0.1 0.4 0.2 0.1 0.7 0.5 0.1 0.4 0.4 0.1 0.5 0.7 0.1 0.2 0.1 0.6 0.3 0.7 0.1 0.2 0.2 0.1 0.7 0.7 0.1 0.2 0.4 0.1 0.5 0.1 0.7 0.2 0.1 0.6 0.3 0.1 0.7 0.2 0.2 0.1 0.7 0.1 0.7 0.2 0.7 0.1 0.2 0.5 0.1 0.4 0.1 0.6 0.3 0.5 0.1 0.4 0.2 0.1 0.7 0.5 0.1 0.4 0.7 0.1 0.2 0.7 0.1 0.2 0.1 0.6 0.3 0.7 0.1 0.2 0.2 0.1 0.7 0.7 0.1 0.2 0.7 0.1 0.2
0.1 0.5 0.5 0.5 0.2 0.2 0.2 0.1 0.1 0.1 0.1 0.1 0.1 0.1
0.1 0.1 0.1 0.1 0.7 0.7 0.7 0.5 0.5 0.1 0.1 0.1 0.1 0.1
0.1 0.1 0.1 0.1 0.4 0.4 0.1 0.7 0.7 0.5 0.1 0.1 0.7 0.7
0.8 0.4 0.4 0.4 0.4 0.4 0.7 0.2 0.2 0.4 0.8 0.8 0.2 0.2
0.1 0.7 0.7 0.7 0.1 0.1 0.1 0.1 0.1 0.7 0.7 0.7 0.7 0.7
0.8 0.2 0.2 0.2 0.2 0.2 0.2 0.4 0.4 0.2 0.2 0.2 0.2 0.2
0.1 0.1 0.2 0.5 0.1 0.2 0.5 0.1 0.2 0.7 0.3 0.2 0.1 0.2
0.1 0.6 0.1 0.1 0.6 0.1 0.1 0.6 0.1 0.1 0.1 0.1 0.6 0.1
alarms
and
Objects with similar characteristics as mines for one or more sensors can result into false alarms for these sensors. In Table 3.1, these false alarm generating objects are given. In the list below, these objects are described along with the failing sensor(s) and the reason of failure:
3.1 Mines
Object Mine Mine Mine Mine Mine Mine Mine Mine Mine Mine Mine Mine Mine Mine Mine Mine Mine Mine
false
• Debris: For debris, all sensor have a large uncertainty mass, since none of the sensors give a clear signal. • Brick: The shape of brick has close resemblance to some type of mines, resulting into failure of GPR. If it is on the surface, or recently buried it also results into failure of IR. • Can: Any can has a large metal content and is detected by MD. It also has a circular shape and therefore IR is failing if it is on the surface. • Nail: Although nails do not contain much metal, they have some resemblance with detonators and thus failing MD. • (Foot)print: A footprint on the surface of horse or a person can have some resemblance with a circular or ellipsoid shaped mine failing IR detection. • Pine cone: The shape of a pine cone can be confused with a mine by the IR detector. • Background: When there is burial activity on an otherwise clean background the IR detector is slightly confused.
0.8 0.3 0.7 0.4 0.3 0.7 0.4 0.3 0.7 0.2 0.6 0.7 0.3 0.7
Table 3.1: Masses as used for Dempster-Shafer for several cases of mines and possible false alarm targets.
3.3 Discussion of results
The first character of this designation refers to the size and can be either small ‘S’ or large ‘N’. The second character of the designation refers to the metal content: using ‘N’ for non-metal, ‘L’ for low metal and ‘H’ for high metal. The third character of the designation refers to the depth of the mine: ‘N’ for non disturbed surface (i.e. buried deep), ‘B’ burial activity and ‘S’ for mines on the surface.
For Dempster-Shafer’s method, three different results are presented: the support for a mine, the plausibility of a mine and the intermediate between those values. The support is the mass assigned to the mine, and the plausibility the mass assigned to the mine plus the uncertainty mass. All the Receiver Operator Characteristics (ROC) curves given in this document are based on the case study with variation of +/-10% to sensor responses of the cases. In Figure 3.1, the ROC curves are given for the various options with Dempster-Shafer. In the ROC curve the
For each possible designation estimates of probability masses were given. The size has the most influence on the GPR masses and to some degree influence on the IR masses. The metal content influenced only the MD masses. The
58
Proceedings of EuroFusion98, M. Bedworth, J. O’Brien (eds.), pp. 55-60, Great Malvern, UK, 1998
Figure 3.3 shows the ROC curves for the different types of fuzzy logic.
1
1
0.9
0.9
0.8
0.8
0.7
0.7 Detection rate
Detection rate
detection rate is plotted against the false alarm rate for every possible threshold.
0.6 0.5 0.4 0.3
0.1 0 0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.5 0.4 0.3
Dempster-Shafer (support) A Dempster-Shafer (interm.) A Dempster-Shafer (plaus.) A Dempster-Shafer (support) B Dempster-Shafer (interm.) B Dempster-Shafer (plaus.) B
0.2
0.6
Rules I A Rules II A Rules I B Rules II B Fuzzy prob. A Fuzzy prob. B
0.2 0.1 0 0.9
1
0
0.1
0.2
0.3
False alarm rate
0.5
0.6
0.7
0.8
0.9
1
False alarm rate
Figure 3.1: The ROC curves of the case study for the various Dempster-Bayes options.
Figure 3.3: ROC curves for fuzzy logic Comparing Figure 3.3 with Figure 3.2, it can be seen that the fuzzy probabilities are performing worse than Bayes and Dempster-Shafer, with regards to detection and false alarm rate. One exception is the region where the detection rate is lower than 0.65.
This figure shows that Dempster-Shafer using mapping type B and plausibility gives the best performance (for the area where the detection rate is larger than 0.5). The variation seen in this figure between the ROC curves for mapping B are larger than the variation for mapping A; the reason for this is that the resulting uncertainty for mapping B is much larger than the resulting uncertainty for mapping A, and thus the differences are more significant when this uncertainty is handled differently.
Figure 3.3 clearly shows that both rule set I and rule set II perform very well at 100% respectively 83% detection rate, because the rules are optimised for these regions, but these rules do not perform very well outside these regions. Performance for the rule sets is (for false alarm rates below 50%) the same for mapping types A and B.
Figure 3.2 shows a comparison between the ROC curves for Bayes and the best DempsterShafer ROC curve.
Figure 3.4 shows the ROC curves when using only a single sensor compared to the best fusion methods. Using the MD, and especially the IR gave poor results on these difficult cases. GPR performed better, but the three fusion methods (Dempster-Shafer, fuzzy rules I, fuzzy rules II) all performed better than any single sensor solution.
1 0.9 0.8 0.7 Detection rate
0.4
0.6 0.5 0.4 0.3
1
0.2
0.9
Bayes A Bayes B Dempster-Shafer (plaus.) B
0 0
0.1
0.2
0.3
0.4 0.5 0.6 False alarm rate
0.7
0.8
0.8 0.9
0.7
1 Detection rate
0.1
Figure 3.2: ROC curves of Bayes and the best Dempster-Shafer
0.6 0.5 0.4 0.3
It can be seen that also for Bayes mapping type B performs better than mapping type A. Also, no large difference exists between the best Dempster-Shafer ROC curve and mapping type B for Bayes. On average the Dempster-Shafer ROC curve performs slightly better.
GPR MD B IR B Dempster-Shafer (plaus.) B Rules I B Rules II B
0.2 0.1 0 0
0.1
0.2
0.3
0.4 0.5 0.6 False alarm rate
0.7
0.8
Figure 3.4: ROC curves of bare sensors compared to fusion
59
0.9
1
Proceedings of EuroFusion98, M. Bedworth, J. O’Brien (eds.), pp. 55-60, Great Malvern, UK, 1998
4. Conclusions Two mapping types were considered by the case study, differing in the method they handle evidence for background for sensors not capable of detecting all mines. The results indicated that these sensors (MD and IR) should handle this evidence as uncertainty rather than as true background. For the presented case study, it was shown that sensor fusion improves detection rate and reduces false alarm rate over single sensor solutions. The sensor fusion algorithms with the best performance in this respect were the two rule sets. However, these require proper modelling of mines and background; something which may not be available in operational use. If proper modelling of mines and background is not possible the use of Dempster-Shafer is preferable.
5. Acknowledgement This research is partly funded by the European Union as ESPRIT project GEODE, number 26337.
6. References A.N.S. Freeling, "Possibilities versus [1] fuzzy probabilities - two alternative decision aids" in "Fuzzy Sets and Decision Analysis", H.-J. Zimmermann, L.A. Zadeh, B.R. Gaines (Eds.), Elsevier Science Publishers, Amsterdam, 1984. L.A. Klein, "Sensor and Data Fusion [2] Concepts and Applications", SPIE, Washington, 1993. L.A. Zadeh, "Fuzzy sets as a basis for [3] the theory of possibility" in "Fuzzy Sets and Systems", Volume 1, North-Holland Publishing Company, 1978 H.-J. Zimmermann, "Fuzzy sets, [4] decision making and expert systems", Kluwer Academic Publishers, Boston, 1987.
60
