A Distributed Information Fusion Testbed for Coastal ...

A Distributed Information Fusion Testbed for Coastal Surveillance Hans Wehna, Richard Yatesa, Pierre Valinb, Adel Guitounib, Éloi Bosséb, Andrew Dlugana, Harold Zwicka a MacDonald Dettwiler and Associates, 3800 Commerce Parkway, Richmond, BC, V6V 2J3, Canada b Defence R&D Canada (DRDC) Valcartier, 2459 Blvd. Pie XI Nord, Québec, QC, G3J 1X5, Canada Abstract - MacDonald Dettwiler is leading a PRECARN partnership project to develop an advanced simulation testbed for the evaluation of the effectiveness of Network Enabled Operations in a coastal large volume surveillance situation. The main focus of this testbed is to study concepts like distributed information fusion, dynamic resources and networks configuration management, and self synchronising units and agents. This article presents the system architecture with an emphasis on our approach for distributed information fusion. Keywords: surveillance, information fusion, distributed fusion, simulation, testbed

1

Introduction

The net-centric initiative of the Canadian Forces (CF) is dubbed Network Enabled Operations (NEOps). It is generally recognized that NEOps should increase the effectiveness of the CF by improving intelligence collection, analysis and information sharing between its various elements, including land, sea and air forces. Consequently, the implementation of NEOps is assumed to be key to achieve shared awareness, increased speed of command, higher tempo of operations and increased security of the CF in the field. Researching and developing NEOps will help the CFs investigating its concepts and remain interoperable with Allied forces, since many of them are also working on their respective applications. Indeed, Canada has already participated in NEOps in Afghanistan and the North Arabian Sea [1]. The large volume surveillance problem faced by Canada is characterized by the employment of mobile (maritime patrol aircraft, helicopters, UAVs, ships) and fixed surveillance assets (e.g. land radar) to a large geographic area in order to identify, assess and track the maximum number of moving, stopped or drifting objects. The observed objects are not necessarily aware of being observed and are cooperative or non-cooperative, and friendly or hostile. Coastal and Arctic Wide Area surveillance are good examples of large volume surveillance. The scarce surveillance (e.g. Electro-Optical (EO), Infra-Red (IR), and Synthetic Aperture Radar (SAR) sensors) and tracking capabilities (normal radar modes) make it very difficult to perform large volume surveillance and to keep track of all activities.

The CanCoastWatch (CCW) project will address the needs of the defence research group at DRDC Valcartier to provide forward-looking technology in support of the Canadian Forces. This project is a collaboration between industry and research (university and government) supported by Precarn funds. Precarn supports precompetitive collaborative research that promotes intelligent information and communications technology for the benefit of the Canadian economy. In the case of CCW, the industries are the prime MacDonald Dettwiler and Associates (MDA) and Actenum, the universities areCalgary and Simon Fraser. A key role in CCW is played by the government researchers from DRDC Valcartier who initiated the project, and who continue to provide valuable support and technical guidance. This paper provides a description of the CCW project (section 2), the multiagent architecture (section 3), the situation evidence (section 4), distributed information fusion (section 5), resource management (section 6), vignettes (section 7), and the future direction of the project and conclusions (section 8).

2

Description of the CCW Project

The primary purpose of the CCW system is to provide a testbed to evaluate Distributed Information Fusion (DIF) and Resource Management (RM) algorithms in the context of large volume littoral surveillance. The secondary purposes of the CCW system include the development of a simulator to explore concepts to improve maritime domain awareness, development of a distributed DIF and RM architecture, and development of a toolbox of distributed high-level DIF and RM algorithms The problem that the CCW system will address is the belief that significant enhancement of Canada’s ports security is not only necessary but also inevitable [2]. Threats may arise from unauthorized vessel traffic such as smugglers of goods and human cargo, terrorists and similar activities. In the current climate of a growing concern for homeland security and public safety, awareness of such situations requires the use of a broad and growing range of resources, the operation of which requires increasingly complex co-ordination.

The best strategy for information communication, fusion, resource management and scheduling in such dynamically changing environments is poorly understood. Traditional methods in AI and Mathematical Programming assume a static and non-distributed problem where the objective is to find an optimal assignment of resources to tasks ,given sufficient computing time. Unfortunately, often neither assumption is valid in the scenarios outlined above. The resource-scheduling problem is dynamically changing as the environment and requirements change due to continuing information updates. Moreover, not all information is always available to all parts of the network. There are communication and computing delays, bandwidth constraints, and communication loss to consider.

A node can Observe, Orient, Decide, and Act. All sensors and platforms belong to nodes. A node can sense via its built-in sensor(s), and it can actuate via its built-in platform. Broadly speaking: • The Observing function of a node corresponds to a Level-1 data fusion capability • The Orientation function corresponds to Level-2 information fusion • The Deciding function performs the Resource Management task • The Acting function implements the decisions made by the Deciding module. Moreover, nodes can exchange messages. In CCW, messages can be orders, requests, or information.

Centralized information systems can make actionable decisions sluggish. A distributed network ,providing all surveillance partners access to low volume but actionable information ,would enable rapid and semi-autonomous resource re-deployment as the operating picture clarifies over time. New management and decision support tools are required to assist distributed and semi-autonomous decision-makers.

The CCW architecture supports multiple nodes, each an OODA agent that interacts with its environment and with other agents. The following figure shows our system as a testbed to support multiple interacting OODA nodes.

With multiple surveillance platforms there is a need to network them, and keep each current on the evolving situation. Initial resource deployment starts a dynamically improving awareness picture of the current situation. Ancillary information will continually be supplied from external sources and contribute to the evolution of the situation assessment. The measurements taken by any one platform need to be distilled into information that improves the situation awareness. This is the role of data fusion in CanCoastWatch. While Level-1 fusion still holds many challenges, CanCoastWatch focuses on Level-2 fusion and higher. While traditional Level-1 fusion fuses sensor contacts into tracks, CanCoastWatch starts at the track level and focuses on higher-level information fusion. It attempts to assert high-level propositions about the targets: their intentions, their relationships to each other and their relationships to the environment. To do this it needs to employ higher-level reasoning. Some aspects of CCW have been reported in [3-4].

3

Multi-agent Architecture

The CanCoastWatch testbed is built to support a multiagent OODA architecture that derives from John R. Boyd's command and control decision loop [5-6].

Node

Node

Visualizer

Ediitor Node

TestbedManager

CommManager

Post Processor

Physical Model

Figure 1 High-Level Testbed Archtecture Each node encompasses the full functionality of OODA and interacts with the world through a feedback loop. The simulated physical world comes from the Testbed's Physical Model. The sensor data is translated into tracks which DF (Data Fusion) interprets into an assessment of the situation through level-2 fusion algorithms (the Orient phase of OODA). This situation is analyzed in terms of the mission goals by RM (Resource Management) which decides on a plan and schedule for the chosen course of action. It does so by altering platform trajectories and sensor parameters on-board the platforms. The following figure summarizes the feedback loop executed by each OODA node:

Platform Trajectories

A node is driven by goals. In CCW, a goal is a data structure that has the following three elements: 1. Proposition 2. Area 3. Time interval

Simulated World

Sensor Params

Sensor Data

Data Fusion (DF)

Fusion Controls

For example, a goal may state that the proposition isSmuggling is to be asserted in an area of northern Vancouver Island during the next 12 hours.

Situation Assessment Resource Management (RM)

Legend data store process

Mission Goals

Figure 2 Data Fusion Feedback Loop They nodes coordinate among themselves by passing goals that decompose a mission into the elements of the plan for each resource to be deployed to carry out the mission. To simplify the overall system, each node is built around a standard architecture. Figure 3 shows the internal structure of a node.

The nodes form a strict command and control hierarchy. Commander nodes can order subservient nodes, which may in turn represent groups of nodes, e.g. a squadron. For example, a commanding node may decide to further decompose a goal by handing subservient nodes sub-goals that cover smaller search areas, or that have subpropositions that serve as specialized evidence to support the proposition of the commander’s goal. CanCoastWatch uses two architectural approaches that will facilitate future research: •

Node Messages Tracks

•

Sensors

Goal

Tracks

Observing

Fused Tracks

Goal Fused Tracks

Node Manager

Orienting

Comm Layer

Situation Evidence

Comm Model

Goals Situation Evidence

Deciding

Decisions

Component-based standard node architecture allows researchers to design replacement components at whatever architectural level of interest to them and suits the new algorithms they wish to explore. This is achieved via a standard interface defined for each component in the architecture. Plug-and-play mechanism that provides standard component addition or replacement. This is achieved via Java JAR files and XML to construct the nested components during system initialization.

The flexibility of this architecture will allow researchers to easily update or replace components and hence investigate many different approaches to data/information fusion and resource management with no software development beyond the content of the component they wish to re-design.

Decision

Status

Acting

Orders

Tasking

Situation Knowledge

Platform, S ensor-Parms

4

Situation Evidence

The situation evidence data structure is a key interface of CCW. It captures what a node has learned by sensing its environment. The situation evidence is simply a collection of pieces of evidence.

Figure 3 Internal Structure of an OODA Node Besides the Observing, Orienting, Deciding and Acting functions, the node also has a “Comm-Layer” that allows it to communicate over communication links with other nodes. The node may also have a platform and sensors. In addition, a node has situation knowledge, which represents all that the node knows about the world. A particularly important part of the node’s situation knowledge is the measurement-derived situation evidence, which will be discussed in more detail in the next section.

A piece of evidence has four elements: 1. Time stamp 2. Proposition 3. Set of proposition qualifiers 4. Set of situation evidence objects For example, at time t1 the proposition isRendezvousing is asserted. There is a single proposition qualifier that records the probability with which this proposition is true, which is represented as a Dempster-Shafer value. Suppose

in our example there are two ships that are rendezvousing, then there are two situation evidence objects listed in the evidence structure, namely the track-IDs for the two ships. No distinction in principle is made between a Level-1 track and a Level-2 proposition. They are all treated as propositions for which there exists direct evidence at certain points in time. All evidence is treated as partially uncertain. For our current version of CCW, there are only two types of uncertainty: Gaussian covariances, and Dempster-Shafer (D-S). The former is used for properties such as “position”, “velocity”, “shape”, etc., and the latter is used for logical propositions such as “isSmugglingOperation”. For example, a piece of evidence can have a proposition called velocity with two qualifiers. One qualifier reports the value of the velocity and the other its covariance. In this example, there is only one situation evidence object, namely the trackID of the ship who’s velocity is reported. This situation evidence data structure stores in a rather raw form what is known from measurements about the situation. However, this information can relatively easily be accessed to provide useful information for decisionmaking. For example, if the goal was to find a fishing boat in distress, then all that is required is to query the situation evidence object for the piece of evidence that has asserted the proposition isFishingBoatInDistress with the highest D-S value. If the value is high enough, the search is declared complete. The ship location can be found by asking for the position value stored for the position proposition for the ship’s track-ID. The situation evidence structure could also be used to yield the probability distribution function (PDF) of a proposition as a function of space and time should this be required. Since all the known raw information is stored, the PDF or similar measures could be generated for a given propositions.

5

Figure 4 Orienting Function The OrientationManager takes one or more goals and a single SituationEvidence object as input. The latter contains situation propositions that were previously trackID-fused by the Observing box. The OrientationManager then tries to verify each goal proposition. In our implementation, the Orienting box is an expert system. The DIF Knowledge base, which is part of the node’s Situation Knowledge, contains rules that allow the ReasoningEngine to “prove” a goal proposition given a set of primitive propositions. The primitive propositions are logical statements such as “isDrifting”. Some of these logical propositions may exist already in the input SituationEvidence. Others need to be created. The creation requires a Classifier that, for example, compares the estimated velocity of a target with the estimated velocity of the ocean currents to assert the primitive proposition “isDrifting”. The ReasoningEngine can then, for example, infer from the propositions “isDrifting” and “isFishingBoat” that the goal proposition “isFishingBoatInDistress” can be asserted. The asserted propositions are stored in a single output SituationEvidence object. All propositions, all facts, and even all rules, are subject to uncertainty. In the current version of CCW, the SituationEvidence and the DIF knowledge base store logical uncertainty according to Dempster-Shafer Theory. The Classifier and the ReasoningEngine are expected to take this uncertainty into account. The OODA agents emulate distributed information fusion as they interact collaboratively. A mission is broken down into goals for nodes at different organizational levels to execute. These nodes share their understanding of the evolving situation through the SE (Situation Evidence), which they feed back up through the control hierarchy.

Information Fusion

In a CCW OODA node, the component responsible for higher-level information fusion is the Orienting box. The major components of this box are shown in Figure 4.

6

The details of the deciding box in the OODA node are shown in Figure 5.

Orienting Goal Fused Tracks

Classifier

Goal

DIF Knowledge Base

Primitive Propositions Fused Tracks

Node Manager

Situation Evidence

Ontology

Orientation Manager

Rules

Goal Primitive Propositions

Situation Evidence

Resource Management

Reasoning Engine

Facts

Deciding Goals Situation Evidence Dynamic Goal(s)

Dynamic Goal Generator

Goal Goals

Node Manager

Situation Evidence

Decision Manager

Capability Plan(s) Capability Plan Mode Plan

Decision

Planner

Dynamic Configuration Advisor

Mode Plan(s) Scheduled Plan(s)

Scheduler

Situation Knowledge

Figure 5 Deciding Function The DecisionManager takes goals and situation evidence as input and provides a decision as its output. A decision is a tasking order for a resource or a goal that is given to or removed from a subservient node. The DecisionManager coordinates the decision making process that is decomposed into four sub-functions that are briefly introduced in the following: The Dynamic Goal Generator (DGG) takes node goals and situation evidence provided by the Orienting function of the node as input. As output it may generate additional goals or remove old goals. The DGG interprets the SituationEvidence object and decides if the evidence warrants a new goal. For example, if a goal is to find a sinking fishing boat, and the SituationEvidence asserts that “isSinkingFishingBoat” is very likely true in a certain small region, then the DGG may decide to issue a highpriority goal to check out just this small region in more detail. A goal comes into the Planner module, which either splits it up into several new goals to be sent to subservient nodes, or it elaborates the goal by generating a capability plan. A capability plan breaks a goal into several subtasks. It also prescribes the order in which the tasks are to be executed. However, it does not specify the exact resource that is required to execute the task. Rather, the capability plan describes the required capabilities of the resource in a high-level manner. The Dynamic Configuration Advisor (DCA) takes a capability plan as input and outputs a mode plan. A mode plan is similar to a capability plan, except that for each task the capability table is replaced by a list of concrete resources that the DCA recommends that the scheduler try. The Scheduler takes all the mode plans as input and turns them into scheduled plans. A scheduled plan is similar to

a mode plan, except that each task has exactly one resource assigned to execute it. A scheduled plan also has absolute times assigned to all tasks. The scheduler resolves any conflicts between all the active plans. It also optimises its output according to a figure of merit, which is a weighted sum of criteria such as completion time and completion cost.

7

Vignettes

The fusion research is focused by “vignettes”, scenarios that are instantiated within the framework established by the vignette’s geographical location and set of available entity types. We looked at two different vignettes within CCW: a cooperative search and a non-coopertive search.

Cooperative Search This vignette addresses a simplified version of a cooperative-search scenario. A boat in distress is reported at 8 km southeast of Chrome Island in the Strait of Georgia in the inland waters off British Columbia. This triggers a cooperative search, i.e. a search where the target wants to be rescued and behaves in a transparent way to facilitate being found. At the time of the reception of the boat-in-distress call, an Aurora aircraft on surveillance task is 18 minutes from completing that task and arriving at Comox Air Force base. An Aurora is on the ground at Comox but needs 20 minutes prep time. This simple scenario is complex enough to highlight important basic features of DIF, RM, and the testbed. RM must choose between the Aurora busy on reconnaissance, or the Aurora on the ground that requires prep time to get airborne. This of course depends on the initial conditions of the vignette. The Aurora provides surveillance, while a Cormorant helicopter provides identification of the ship in distress. A Cormorant is ready close by at Comox, and another Cormorant is ready at Vancouver, which is further away. Both are dispatched and the timing of their arrival results in different scenarios playing out since the ship in distress drifts with the current, and a large number of active shipping and other objects in the area complicate the search mission. For this type of vignette, RM must evaluate which of the two available Auroras to utilize, choosing between one doing a routine background patrol or a dedicated search Aurora. DIF must then correctly use the sensor data from the selected Aurora to identify likely targets that need closer inspection by the helicopters. RM must react to this information the moment the DF provides it, and then dynamically re-schedule the helicopters to go and verify the likely targets based on their proximity to the current location of the helicopters. A screenshot of the CanCoastWatch visualizer is shown in Figure 6. The visualiser allows the researcher to step through each instant of the simulation run and display the

tracks of the targets and platforms together with helpful labels. When new likely targets are found, it marks their location in the display. Figure 6 displays an instant towards the end of the simulation run. The Aurora has already arrived in the search area and is flying a zigzag search pattern to scan the area for likely targets. The two helicopters have also already arrived. The helicopter from Comox arrived before the Aurora and started to fly a tightly knit zigzag search pattern. However, after the Aurora had arrived and found likely targets, the helicopter abandoned its search pattern and flew to investigate the likely targets. To do this it started a spiral search pattern around the reported location of the new target. The spiral pattern was abandoned the moment the target was declared false. Finally, the second helicopter arrived, and both shared the task of verifying likely targets thereby speeding up the search significantly.

other freighter scheduled to be in the area, in an attempt to confuse surveillance. It will use two land-based zodiacs to offload the illegal immigrants, by making multiple trips to/from the freighter to ferry persons to the coast. The intended drop point is either Guise Bay or Experiment Bight in the Cape Scott Provincial Park depending on conditions. The scenario is depicted in Figure 7.

Figure 7 Non-Cooperative Search Vignette

Figure 6 Cooperative Search Vignette

Non-Cooperative Search In contrast to the first vignette, this vignette features targets that don’t want to be found. It involves a freighter carrying illegal immigrants, which are off-loaded to zodiacs. This scenario will feature deceptive maneuvers and intentionally false sensor data, and thus will require a sophisticated DIF and an evolution of the RM capability over that used for the 1st vignette as the complexity of the following description shows. The vignette-2 mission will focus on a threat situation that develops off the northwest tip of Vancouver Island. A freighter coming from the eastern Pacific carrying illegal immigrants will arrive near Cape Scott on northern Vancouver Island. It will leave a known sea-lane off Cape Scott to begin a manoeuvre to off-load the illegal immigrants. It will not use the Automatic Identification System (AIS) to identify itself, rather - when it suspects it is being watched --- it will use an AIS identification of an

The freighter and zodiacs will attempt various elusive manoeuvres depending on how the situation develops. Among the elusive deceptions that the freighter can utilize to deceive DF and force RM to re-evaluate plans and schedules, the following are considered: • use a known commercial shipping route (sea lane) to mask its approach among other commercial freighters, • fail to provide an AIS identification or provide a false AIS corresponding to a scheduled freighter known to be scheduled to be in Cape Scott area around this time, • depart from a known commercial sea lane through Scott Channel when it approaches the intended drop point for its illegal cargo, • use deceptive cover of the presence of fishing boats open fishery off Experiment Bight to “hide” its presence amongst those boats, • respond to surveillance by active sensors (radar) by moving back into the commercial shipping route heading away from Vancouver Island, • use elusive maneuvers to periodically leave the fishing fleet to rendezvous with two island-based zodiacs The complexity of the non-cooperative search is captured by a large network of evidence with multiple possible confirmatory patterns. We will use a reasoning engine to evaluate the evidence to 'build up' a case to confirm an example of smuggling. For example, DIF may fuse information from various sensors on various platforms to identify a smuggling-operation by • Identifying an isFerrying activity that in turn requires confirmation of isLargeShip(s1) AND isShipNearShore(s1) AND isMovingSlowly(s1) AND

•

isSmallShip(s2) AND isMovingBetweenBeachAndLargeShip(s2,b,s1). Identifying an isRendezvousing activity that in turn requires confirmation of isLargeShip AND isSmallShip AND isTandemMotionBetweenShips where the tandem motion is defined by isShipsHaveSameHeading(s1,s2) AND isShipMovingSlowly(s1) AND isShipMovignSlowly(s2) AND areNear(s1,s2)

The reasoning is complex because the non-cooperative search requires reasoning about relationships and behaviour. These are properties developed in space and time and can be quite complex. The reasoning here will require working through threads of evidence where depending on the situation, when resource arrive, what can be sensed, etc. there will be different pathways through the evidence to make the conclusion isSmuggling. The testbed allows the evaluation and comparison of different fusion and resource management algorithms with respect to several criteria, for example, the elapsed time until the sinking fishing boat or the smuggling operation is discovered. Other criteria include the cost for the surveillance operation, or the weighted sum of the detection time and the cost of the operation.

8

Future Directions and Conclusions

The CanCoastWatch project is divided into three phases: •

•

•

Phase 1 built a simple implementation where the OODA control loop was centralized and nonautomomous resources were planned and dispatched. Each resource was mobile with simple sensors and could interact in a dynamic way to the environment. Phase 2 is currently building a multiagent OODAbased testbed where each node is a mobile and sensing node that interacts with other nodes through mission goals that are elaborated through multiple levels of hierarchy. This phase is restricting communication modeling and the complexity of interactions to focus on the goal of getting distributed OODA agents sensing, responding, and collaborating on a mission. Phase 3 will remove the limitations of Phase 3 and deliver the full testbed system.

We hope that CCW will serve the military research community by providing a flexible architecture for information fusion research.

References [1] General Ray Henault, NEOps and Future Operations, Presentation to the Network Enabled Operations Symposium, Nov 30 – Dec 1 2004 (http://www.drdcrddc.gc.ca/newsevents/events/neops_e.asp). [2] Coastal defence and maritime http://www.sfu.ca/casr/np-prev04.htm

security,

[3] Hans Wehn, Richard Yates, Andrew Dlugan, et. al., A Testbed Simulator to Evaluate the Efficiency of NetEnabled Surveillance, UVS Canada 2006, Montebello, Quebec, November 7-10 [4] Pierre Valin, Adel Guitouni, Éloi Bossé, et. al., Testbed for large volume surveillance through distributed fusion and resource management, SPIE Defense & Security Symposium, Orlando, Florida, USA, 2007, April 9-13 [5] Col. John R. Boyd, Patterns of Conflict, December 1986 [6] J.Roy, É. Bossé, Sensor Integration, Management and Data Fusion Concepts in a Naval Command and Control Perspective, DRDC-V, DREV-R-9816, Oct. 1998