Game Theoretic Sensor Management for Target Tracking

Game Theoretic Sensor Management for Target Tracking Dan Shena, Genshe Chen∗a, Erik Blaschb, Khanh Phamc, Philip Douvilleb, Chun Yangd, and Ivan Kadare a

b

DCM Research Resources, LLC, Germantown, MD, USA 20874 Air Force Research Laboratory, Sensors Directorate, Wright-Patterson AFB, OH, USA 45433 c Air Force Research Laboratory, Space Vehicles Directorate, Kirtland AFB, NM, USA 87117 d Sigtem Technology, Inc., San Mateo, CA, USA 94402 e Interlink Systems Sciences, Inc., Lake Success, New York, USA 11042

ABSTRACT This paper develops and evaluates a game-theoretic approach to distributed sensor-network management for target tracking via sensor-based negotiation. We present a distributed sensor-based negotiation game model for sensor management for multi-sensor multi-target tacking situations. In our negotiation framework, each negotiation agent represents a sensor and each sensor maximizes their utility using a game approach. The greediness of each sensor is limited by the fact that the sensor-to-target assignment efficiency will decrease if too many sensor resources are assigned to a same target. It is similar to the market concept in real world, such as agreements between buyers and sellers in an auction market. Sensors are willing to switch targets so that they can obtain their highest utility and the most efficient way of applying their resources. Our sub-game perfect equilibrium-based negotiation strategies dynamically and distributedly assign sensors to targets. Numerical simulations are performed to demonstrate our sensor-based negotiation approach for distributed sensor management. Keywords: Sensor assignment, negotiation game, agent, sub-game perfect, sensor gain, distributed sensor management, sensor network

1. INTRODUCTION The objective of developing a game-theoretic approach to distributed sensor-network management is to perform active management of distributed sensor fusion for situation awareness [1]. Game theory offers learning models for distributed allocations of surveillance resources and provides mechanisms to handle uncertainty of environments [2]. There are two major classes of sensor management algorithms: centralized and decentralized. In centralized sensor management (CSMgt) [3-6], a central processing node(s) would collect information from all other sensors, targets, and/or environments, and then assign sensors to different targets based on exploited information. In CSMgt, the assignment will be accepted unconditionally where the central nodes are a higher level than other sensor nodes dictating the sensor control. The advantages of use of a CSMgt strategy include a simple system design and less computational load in a small scale network. However, the centralized approaches are not always suitable for modern sensing/signal processing systems which become more and more complex and often require higher robustness (e.g. Net-centric warefare). When the scale of a system grows, the process of collecting information from all other sensors will be time consuming and undependable, which causes serious system synchronization and efficiency problems. In addition, when a CSMgt sensing system works in a severe environment, the failure of the central node would cause the failure of the whole system.

∗

[email protected]. This work was supported in part by the US Air Force under contracts FA8650-08-C-1407 Signal Processing, Sensor Fusion, and Target Recognition XIX, edited by Ivan Kadar, Proc. of SPIE Vol. 7697, 76970C · © 2010 SPIE · CCC code: 0277-786X/10/$18 · doi: 10.1117/12.850870

Proc. of SPIE Vol. 7697 76970C-1 Downloaded From: http://spiedigitallibrary.org/ on 11/26/2013 Terms of Use: http://spiedl.org/terms

To overcome shortcomings of the centralized approach, we need to develop a decentralized approach for robust realworld applications, such as in an environment with critically low signal-to-noise ratio, or even dangerous areas. In such cases, it is not easy for a centralized approach to obtain the information from all the sensor nodes as some communication links might be broken at unexpected time. Also, the information quality/correctness, such as the dependability and reliability of received information, would create network survivability issues. Furthermore, the communication delay will become more unpredictable when the system scales up with larger number of sensors. Most importantly, system failure caused by the failure of one single sensor is not allowed. Thus, we desire distributed and decentralized sensor-network management (DSMgt) approaches. In decentralized sensor management (DSMgt) approaches [8-10], coordination occurs locally (not globally) and there is no central node which will make any globally optimal decisions. No sensor node can “broadcast” information (e.g. availability, coordination proposal, negotiation results, etc.) to all other sensor nodes. In addition, no sensor node is supposed to have global knowledge of all sensors. The advantages of decentralized approaches include a scalable, modular, and survivable (robust) sensor network system. Sensor-to-target assignment aims to control the data acquisition process in a multi-sensor system to enhance the performance of target tracking. The problem of sensor assignment can be understood from the point of view of supply and demand analysis. By treating targets as “customers,” each target with explicit or implicit demand requirements is satisfied by supplying their needs with least amount of resources. The centralized sensor assignment has many limitations and impractical for situation awareness scenarios. Decentralized sensor assignment is required for future sensor assignment methods in sensor surveillance due to the number of targets and sensors. During a DSMgt scheme, to protect the benefits of all parties in the sensor-to-target assignment, negotiation among them is necessary. In this paper, we propose a negotiable game-theoretic based sensor management (NG-SMgt) approach to deal with the requirements of a dynamic sensor management and assignment. The rest of the paper is organized as follows. In section 2, the sensor management problem statement is presented. Section 3 describes the sensor-based negotiation game theoretic SMgt model. Numerical examples are simulated in Section 4. Finally we draw conclusions in Section 5.

2. PROBLEM STATEMENT To clearly understand sensor management for situational awareness, we need to first give a brief introduction of target tracking. Target tracking is the process of maintaining state estimates of one or several objects over a period of time. Target tracking algorithms are basically state estimation algorithms, where the state estimate is corrected by measurements from various sensors. The use of multiple sensors can dramatically improve tracking accuracy in a process known as sensor fusion. As the number of targets and sensors increases, centralized tracking systems can very quickly become overloaded by the large volume of incoming data. Furthermore, as the number of available sensors and sensor modes increases, it is easy to overwhelm human operators who are responsible for the sensor network. Sensor manager systems that balance tracking performance with system resources are often required to control information flow. Although functions controlled by sensor managers include a wide range of sensing activities, such as sensor modes or scanning regions, we focus on how best to assign sensors to the targets of interest to improve tracking system efficiency and effectiveness. In sensor management, each observer has to decide when to measure which object so that the performance gain in terms of a certain metric can be maximized. For a Kalman filter or its extension for the nonlinear dynamic state or measurement equations, namely, the recursive extended Kalman Filter (EKF), the state estimate error covariance has the following recursive form:

Pk−+11|k +1 = Pk−+11|k + H (x k +1 )T Rk−+11 H ( x k +1 )

(1)

Thus, the sensor gain from the sensor measurement at time tk +1 in terms of the inverse of the state estimation error covariance is

H (x k +1 )T Rk−+11 H (x k +1 ) where Rk +1 is the measurement error covariance. Consider S observers each of

which can measure at most one object at any sampling time. When there are T targets being tracked by S observers,


sensor assignment (S-T) is concerned with the sensor-to-target correspondence so that the total sensor gain can be maximized. For a decentralized system, one way for linking sensors and targets is to add a middle layer between them, called agents to perform dynamic sensor assignment and scheduling. An agent will represent a specific sensor. All desired performance matrix requirements of target tracking are saved in the agent and sent out via the agent. After the negotiation between agents, the available resources (e.g. sensors) are allocated to different targets for best system performance. Each agent plays a management role to the tracking tasks assigned to it and struggle to get the most available resources to solve the tracking tasks. There are three main reasons for use of agent-based negotiation. First, agent-based negotiation can scale well to deal with goal uncertainty (such as the case of an unclear goal in the current situation), which is a usual case in real applications. Second, sometimes we may have exact desired covariance levels for each target but do not have sufficient sensors to meet all these desired standards. Therefore, we have ambiguity on how to treat those target-specific requirements while making a global allocation decision. Agent-based negotiation provides a powerful means to deal with the local sensor interactions and affords a balancing mechanism arbitrating sensor needs. The last reason is that agentbased negotiation improves local tracking performance via requirement-oriented fashion rather through a global computation. Global computations are not practical in complex tracking scenarios where the diversity of targets and situations leads to distinct and time-varying demands across various targets. Agent-based negotiation triggered by requests offers a flexible way of updating sensor assignments to tune local performance whenever and wherever necessary. We identify three general issues, which need to be addressed for agent-based negotiation in a multi-target tracking system: (1) Representation. Most negotiations are about complex objects and situations that may require the support of a sophisticated representation scheme. Examples can range from targets and environments to sensors in which the operating conditions can vary over the situation. Thus, we need to establish the parameters and the representation of the parameters in the negotiation process. Instead of negotiating about money for goods in a market place; we can negotiate about reducing target uncertainty (demand) and sensor placement (supply) resulting in sensor for accuracies. (2) Problem solving. Many aspects of negotiation can be modeled as an exercise strategy in distributed sensing networking. There is a large body of work on algorithms and techniques for constraint solving that can be applied to negotiation and assignment problems. (3) Communication. In order to support meaningful communication among the negotiating parties, we need to have a common protocol and language for expressing primitive communicative acts that make up a negotiation (e.g., an agent request, a rejection of another agent, etc.) as well as a way to specify different negotiations that can take place. Compared with the centralized sensor management scheme, the decentralized management approaches rely on communication between nearby sensors. By doing so, the number of messages that each platform/sensor sends or receives is independents of the total number of nodes (platforms/sensors) in the system. The neighboring property ensures scalability of distributed systems with any number of nodes in any dynamic environments. The coordination occurs locally (not globally) and there is no central node which will make any global decisions. No node needs to “broadcast” information to all other nodes for management. In addition, no sensor node is supposed to have global knowledge of the entire sensor network. Negotiation will be performed between neighboring sensors and affords a sensor management scheme for efficient target tracking based on available resources. The advantages of using a decentralized sensor management can be summarized as follows: 1) system can be scalable without the bottlenecks caused by centralized nodes. No central node and any common communication facility ensures that the system is scalable as there are no limits imposed by centralized computational bottlenecks or lack of communication bandwidth; 2) system can be modular and distributed for the cases that no global information (prior information) is required. No node is central and no global knowledge of the network topology is required for fusion. The system fusion processing can be made survivable and robust to the on-line loss (or addition) of sensing nodes and to


dynamic changes in the network structure; 3) system can be survivable for the cases that no global information of system topology is required and the system does not critically rely on any single sensor for any stage of the whole sensing/coordinating process. As a result, failure of one sensor would not cause the failure of the whole sensor network, and it is easy to expand the sensor network via adding more sensor nodes. As all fusion processes must take place locally at each sensor site and no global knowledge of the network is required a priori, nodes can be constructed and programmed in a modular fashion.

3. SENSOR-BASED NEGOTIATION GAME MODEL FOR SENSOR MANAGEMENT In this section, we present a distributed sensor-based negotiation game model for sensor management in multi-sensor multi-target tacking situations. It is an extension of our previous work [12]. In our negotiation framework, each negotiation agent represents a sensor. The greediness of each sensor is limited by the fact that the sensor resource efficiency applied to a target will decrease if too much sensor resources assigned to a same target. It is similar to the market concept in real world. Sensors are willing to switch targets so that they can obtain the most efficient way of applying their resources. 3.1 Negotiation Game Model For each sensor (represented by an agent Ai), we create a negotiation game played by the sensor and its neighboring sensors Ni = {Neighboring sensors of sensor Ai}. We assume that there are communication links among the negotiation agent set {Ai, Ni} so that each agent in the set knows the current targets of other sensors. The sensor-based negotiation game is represented by a 5-tuple Gi = < Agents, Targets, H, P, C >, where • • • • •

Agents is the set of agents playing the game. Agents ={Ai, Ni}; Targets is the set of targets currently assigned to Agents. To enable sensors be in quiescence mode, we add 0 to Targets. H is the history sequence of the negotiation process including the offers and responses P is the function to obtain the agent to make an offer. P is defined on H. C is the utility function set. C = {Ci, i ∈ Agents}. The utility function is based on the following equation (The gain contributed by sensor i to target j at time k).

g (i, j; k ) = min eig ( M j ,k −1 + H ijT (k ) Rij−1 (d ij (k )) H ij (k )) − min eig ( M j ,k −1 )

(2)

where Mj, k-1 is the information matrix (the inverse of the P matrix) of target j at the previous time. mineig is the operator to calculate the minimal eigenvalue of a matrix. We make the following assumptions: • Rationality. All agents in the system are self-interested and rational. Each of them tries to maximize its own benefits in negotiations. • Initial Quiescence. After the negotiation begins, agents will not make any measurements until the end of the negotiation game. It seems not practical from an operational point of view. But it makes some sense for reassignment problems. Moreover, we will show later that an agreement will be obtained after first time step. Therefore the effects of initial quiescence are relatively small. • Sensor Capacity = 1. Generally, the set Di of sensor i can contain multiple targets. Here we first assume that the capacity of each sensor is 1, so that |Di| = 1. Therefore each sensor will select one target from Targets. Di = {di ∈ Targets}. However, it is possible that one target is selected by more than one sensor. This assumption can be relaxed and similar solutions can be obtained by following the definition of utility functions. • Agent Negotiation Capability = 1. It means at any time, each agent only involve one negotiation game. • Negotiation Game Enable Control. A negotiation game can be launched only after the host agent (who maintains the negotiation game) receives all acknowledge signals from its neighboring sensors Ni. The neighboring agent will send reschedule signal if it is involved in another negotiation or some of its status information is updated to request it to participate in the negotiation game. Then the host agent has to update the game information and resend the reschedule signal to enable a negotiation.


3.2 Subgame Perfect Equilibrium Based Negotiation Strategies A fundamental concept in game theory is the Nash Equilibrium [10], which is named after John Forbes Nash, who first proposed it. The concept is a solution concept of a game involving two or more players. In the game, each player is assumed to know the equilibrium strategies of the other players, and no player has anything to gain by changing only his or her own strategy (i.e., by changing unilaterally). If a player has chosen a strategy and no other player can benefit by changing his strategy while the other players keeps their’s unchanged, then the current set of strategy choices and the corresponding payoffs constitute a Nash Equilibrium. In a Nash Equilibrium, each player must answer questions from others and knowing the other player strategies, try to benefit himself from the other player strategies. For example, Tom and Jim are in game with Nash equilibrium, when Tom is making the best decision he can, and taking into account Jim's decision, at the same time, Jim is also making the best decision he can, and taking into account Tom's decision. In the same way, many players are in game with Nash equilibrium if each one is making the best decision that they can, and taking into account the others. Each strategy in a Nash Equilibrium is a best response to all other strategies in that equilibrium. The Nash Equilibrium may sometimes appear non-rational in a third-person perspective as a Nash Equilibrium is not Pareto optimal. The Nash Equilibrium may also have non-rational consequences in sequential games because players may "threaten" each other with non-rational moves. However, in many cases all the players might improve their payoffs if they could somehow agree on strategies different from the Nash equilibrium, which leads the concept of Subgame Equilibrium [11]. Subgame Equilibrium is an attempt to choose from the set of Nash Equilibria and in every subgame, Nash Equilibrium will be kept due to the structure of the game. Nash equilibrium is a normal-form concept, which ignores the sequential structure of play in extensive-form games. As a result it predicts some equilibria which appear problematic in the extensive form. But, Subgame Equilibrium can avoid these problems by reaching a local optimal. In a sbugame-equilibrium based negotiation, the bargain will be finished with agreement within two time steps. Using the bargaining strategy in negotiation, we develop a negotiable game-theoretic based sensor management (NG-SMgt) for sensor-to-target assignments that increase target tracking efficiency and effectiveness.

4. NUMERICAL SIMUATIONS We implement our sensor-based negotiation game for distributed sensor management. A scenario (Figure 1) with 8 targets and 6 sensors is used to demonstrate our algorithm.

Figure 1: A scenario with 8 moving targets and 6 moving sensors. In this scenario, the objective of the sensor management system is to dynamically and distributedly assign sensors to targets to improve the overall tracking performance. The 8 moving targets will follow the scripted moving paths. The end point of each path is marked by "o". Similarly, the sensors have their own paths with a “◊”.


We simulated the system for 300 time steps. Fig. 2 shows the results at time k = 90. The overall simulation is recorded in a video clip (demo.wmv, 1.2MB, 19s).

Figure 2: The sensor assignment at time k = 90. Fig. 3 shows the sensor assignment of each sensor. For example, sensor 4 first assigned to target 8, then target 2, finally target 1. The associated utilities obtained by each sensor are shown in Fig. 4.

Figure 3: Sensor assignment.


Figure 4: Utilities of each sensor.

For each target, the minimal eigenvalues of the information matrix at each time step are shown in Fig. 5.

Figure 5: Minimal eigenvalue of the information matrix of each target.


We obtained these preliminary results which shows that our game theoretic negotiation solution to distributed sensor management works in a dynamical and distributed way. From Fig. 5, we found some targets (target 1 and 4) are better tracked than target 7 and 8. This is because every agent (sensor) is greedy and only cares about its own utility. Further evaluations and development of the concept will be applied to different scenarios.

5. CONCLUSIONS We proposed a negotiation game approach for distributed sensor management. Our approach has several features. First, it is a Distributed agent system. Each agent represents a physical sensor. Each agent hosts and maintains a negotiation game. By playing the game with its neighboring agents, the host agent generated a self-enforced target assignment among the players of the game. All the decisions are made locally. Second, target tracking performance metrics are integrated in the utility (or objective) functions of agents. We build the utilities based on the sensor gains and target values, which can be straightforwardly replaced with different metrics. Third, there is no requirement on the number of targets and sensors. No pseudo sensors are needed to handle the cases with more targets than sensors. We implemented the sensor-based game negotiation algorithm and simulated a scenario with 8 moving targets and 6 moving sensors. Supportive results were obtained.

REFERENCES [1] E. Blasch, I. Kadar, K. Hintz, J. Bierman, C. Chong, and S. Das, “Resource Management Coordination with Level 2/3 Fusion”, IEEE AES Magazine , Mar. 2008. [2] G. Chen, D. Shen, C. Kwan, J. Cruz, M. Kruger, E. Blasch, “Game Theoretic Approach to Threat Prediction and Situation Awareness,” Advances in Journal for Information Fusion, July 2008. [3] Y. Bar-Shalom and X. Li, Multitarget-Multisensor Tracking: Principles and Techniques, YBS Publishing, 1995. [4] T. Kirubarajan, Y. Bar-shalom, W. D. Blair, and G. A. Watson, “LMMPDAF for Radar Management and Tracking Benchmark with ECM,” IEEE Transactions on Aerospace and Electronic Systems, Vol. 34, No. 4, pp. 2616-2620, 1995. [5] G. A. Watson, W. D. Blair, and T. R. Rice, “Enhanced Electronically Scanned Array Resources Management through Multisensor Integration,” Proceedings of SPIE Conference Signal Processing of Small Target, Vol. 3163, Orlando, FL, pp. 329-340, 1997. [6] J. Manyika, An Information Theoretic Approach to Data Fusion and Sensor Management, PhD thesis, The University of Oxford, 1993. [7] J. Manyikia and H. F. Durrant-Whyte, Data Fusion and Sensor Management: An Information-Theoretic Approach, Prentice Hall, 1994. [8] B. S. Y. Rao, H. Durrant-Whyte, and A. Sheen, “A Fully Decentralized Multisensor System for Tracking and Surveillance,” International Journal of Robotics Research, 12(1):20-45, 1991. [9] S. Utete, A Network Manager for a Decentralized Sensing System, Master’s thesis, University of Oxford, 1992. [10] J. Nash, “Two-person cooperative games,” Econometrica, Vol. 21, pp. 128-140, 1953. [11] M. Osborne and A. Rubinstein, A course in game theory, Cambridge, Massachusetts: MIT Press, 1994. [12] Li, X., Chen, G., Blasch, E., Patrick, J., Yang, C., and Kadar, I., "Multi-sensor management for data fusion in target tracking", Signal Processing, Sensor Fusion, and Target Recognition XVIII. Edited by Kadar, Ivan. Proceedings of the SPIE, Volume 7336 (2009), pp. 73360Y-73360Y-12 (2009).
