an ECHO 1, as observers, and 2 GEO satellites, namely EchoStar 10 and COSMOS 2350, as space objects. The orbits are generated by the General Mission ...
A Trust-based Sensor Allocation Algorithm in Cooperative Space Search Problems Dan Shena, Genshe Chena, Khanh Phamb, Erik Blaschc, a
Independent Consultant, 14163 Furlong Way, Germantown, MD, USA 20874 Air Force Research Laboratory, Space Vehicles Directorate, Kirtland AFB, NM, USA 87117; c Air Force Research Laboratory, Sensors Directorate, Wright-Patterson AFB, OH, USA 45433 b
ABSTRACT Sensor allocation is an important and challenging problem within the field of multi-agent systems. The sensor allocation problem involves deciding how to assign a number of targets or cells to a set of agents according to some allocation protocol. Generally, in order to make efficient allocations, we need to design mechanisms that consider both the task performers’ costs for the service and the associated probability of success (POS). In our problem, the costs are the used sensor resource, and the POS is the target tracking performance. Usually, POS may be perceived differently by different agents because they typically have different standards or means of evaluating the performance of their counterparts (other sensors in the search and tracking problem). Given this, we turn to the notion of trust to capture such subjective perceptions. In our approach, we develop a trust model to construct a novel mechanism that motivates sensor agents to limit their greediness or selfishness. Then we model the sensor allocation optimization problem with trust-in-loop negotiation game and solve it using a sub-game perfect equilibrium. Numerical simulations are performed to demonstrate the trust-based sensor allocation algorithm in cooperative space situation awareness (SSA) search problems. Keywords: sensor allocation, space search, trust-in-loop, negotiation game model.
1. INTRODUCTION There are two major classes of sensor management algorithms: centralized and decentralized. In centralized sensor management (CSMgt) [1-3], a central processing node(s) collects information from all other sensors, targets, and/or environments, and then assigns sensors to different targets based on the available exploited information. In CSMgt, the sensor-to-target assignment is accepted unconditionally as the central nodes assume a higher command level than the sensor nodes to dictate sensor control. The advantages of a CSMgt strategy include a simple system design and less computational load in a small scale network. However, centralized approaches are not always suitable for modern sensing/signal processing systems which become more and more complex and often require higher robustness (e.g. Netcentric warfare). When the size of a network grows, the process of collecting information from all other sensors will be time consuming and unreliable, which causes serious system synchronization and efficiency problems. In addition, when a CSMgt sensing system works in a severe environment, the failure of the central node would cause the failure of the whole system. To overcome shortcomings of the centralized approach, we need to develop a decentralized sensor management (DSMgt) approach for robust real-world applications. CSMgt sensor network system designs are not appropriate for sensors with critically low signal-to-noise ratios or sensors operating in dangerous areas. In such cases, it is not easy for a centralized approach to obtain the information from all the sensor nodes as some communication links might be broken at unexpected times. Also, the information quality/correctness, (i.e. dependability, sustainability, and reliability of received information), would create network survivability issues. Furthermore, the communication delay will become more unpredictable when the system scales up with larger number of sensors. Most importantly, a centralized approach typically requires all sensors to operate for system optimization and when one sensor fails, it could affect system-level performance. Thus, we desire distributed and decentralized sensor-network management (DSMgt) approaches.
Sensors and Systems for Space Applications IV, edited by Khanh D. Pham, Henry Zmuda, Joseph Lee Cox, Greg J. Meyer, Proc. of SPIE Vol. 8044, 80440C · © 2011 SPIE CCC code: 0277-786X/11/$18 · doi: 10.1117/12.882904 Proc. of SPIE Vol. 8044 80440C-1 Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
In decentralized sensor management (DSMgt) approaches [4-6], coordination occurs locally (not globally) and there is no central node that determines globally optimal decisions. No sensor node can “broadcast” information (e.g. availability, coordination proposal, negotiation results, etc.) to all other sensor nodes. In addition, no sensor node is supposed to have global knowledge of all sensors. The advantages of decentralized approaches include a scalable, modular, and survivable (robust) sensor network system. Sensor-to-target assignment aims to control the data acquisition process in a multi-sensor system to enhance the performance of target tracking. The problem of sensor assignment can be understood from the point of view of an economic supply and demand analysis. By treating targets as “customers,” each target with explicit or implicit demand requirements is satisfied by supplying their needs with least amount of resources. The centralized sensor assignment has many limitations and is impractical for situation awareness scenarios since collecting all demands and determining all supplies would create more of a time delay than is acceptable. Decentralized sensor assignment is required for future sensor assignment methods in sensor surveillance due to the large number of targets and sensors. During DSMgt sensorto-target assignment, negotiation is required to balance the costs and benefits of all participants. In this paper, we propose a negotiable game-theoretic based sensor management (NG-SMgt) approach to deal with the requirements of a dynamic sensor management and assignment. In [7], a game-theoretic approach to distributed sensor-network management for target tracking via sensor-based negotiation has been developed. If every agent or sensor is rational and cooperative (i.e. follow the negotiation result), the tracking performance is optimal. However, sometimes a sensor may choose a greedy tracking strategy to maximize its own performance which might degrade the team performance. So, in this paper, we revised the negotiation game model by integrating a trust model to limit the greediness of each sensor. In our research, we have looked at game-theoretic solutions for threat prediction and situation awareness [8, 9] in a multiplayer scenario. We utilized the methods for space threat detection [10], attack avoidance [11], and orbital evasive maneuvering through sensor management [12]. Using multi-agent modeling, we developed a framework for space situation awareness (SSA) [13]. To further the analysis, we compared several tracking methods [14] including delayed measurements [15]. Coupled with tracking, efforts of information metrics [16], search allocation game strategies [17], and service oriented architectures [18], improved the fidelity of the SSA modeling and system-level analysis. Our companion paper [19] provides a collision alert detection to provide operators with decision support for satellite protection. Using the detection of collision analysis, we investigate a distributed control track state using an open-loop feedback control rule with analysis in the NASA GMAT simulator. The rest of the paper is organized as follows. In section 2, we describe the revised sensor-based negotiation game theoretic sensor management approach with the proposed trust model. Space object tracking examples are simulated in Section 3. Finally we draw conclusions in Section 4.
2. TRUST-BASED NEGOTIATION GAME MODEL FOR SENSOR MANAGEMENT In this section, we present a distributed trust-based negotiation game model for sensor management in multi-sensor multi-target tacking situations. It is an extension of our previous work [7]. In our negotiation framework, each negotiation agent represents a sensor. The greediness of each sensor is limited by a trust value, which is large when a sensor node (player in a game) stays near in the sense of sub-game perfect equilibrium and small when the team optimal result deviates from the negotiation result. 2.1 Negotiation Game Model with Trust For each sensor (represented by an agent Ai), we create a negotiation game played by the sensor and its neighboring sensors Ni = {Neighboring sensors of sensor Ai}. We assume that there are communication links among the negotiation agent set {Ai, Ni} so that each agent in the set knows the current targets of other sensors. The sensor-based negotiation game is represented by a 5-tuple Gi = < Agents, Targets, H, P, C >, where • •
Agents are the set of agents playing the game. Agents ={Ai, Ni}; Targets are the set of targets currently assigned to Agents. To enable sensors be in quiescence mode, we add 0 to set of Targets.
Proc. of SPIE Vol. 8044 80440C-2 Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
Sensor assignment for space object 1 (EchoStar 10) 6 sensor index
Sensor 1
Sensor 2
Sensor 3
Sensor 4
4 2 0 0
200
400
600
800
1000
1200
1400
1600
1800
2000
Sensor assignment for space object 2 (COSMOS 2350) 6 sensor index
Sensor 1
Sensor 2
Sensor 3
Sensor 4
4 2 0 0
200
400
600
800
1000 1200 time (k)
1400
1600
1800
2000
Figure 3: Sensor assignment for each space object. (Sensor i = 0 means the sensor is not assignment to the target)
sensor 1
sensor 2 3 target index (1-2)
target index (1-2)
3 2
1 0
0
500
1000
1500
2
1 0
2000
0
500
sensor 3
2000
1000 1500 time (k)
2000
3 target index (1-2)
target index (1-2)
1500
sensor 4
3 2 1 0
1000
0
500
1000 1500 time (k)
2000
2 1 0
0
500
Figure 4: Target assignment for each sensor. Fig.5 shows the utilities of sensors at each time step. For each target, the minimal eigenvalue of the information matrix at each time step are show in Fig. 6.
Proc. of SPIE Vol. 8044 80440C-6 Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
• Rationality. All agents in the system are self-interested and rational. Each agent tries to maximize its own benefits in negotiations. • Initial Quiescence. After the negotiation begins, agents will not make any measurements until the end of the negotiation process which might seem impractical from an operational point of view. But it makes some sense for reassignment problems. Moreover, we will show later that a negotiation agreement will be obtained after a few time steps. Therefore the effects of initial quiescence are relatively small and repeated negations can be made for an entire game scenario • Sensor Capacity = 1. Generally, the set Di of sensor i can contain multiple targets. Here we first assume that the capacity of each sensor is 1, so that |Di| = 1. Therefore each sensor will select one target from Targets. Di = {di ∈ Targets}. However, it is possible that one target is selected by more than one sensor. This assumption can be relaxed and similar solutions can be obtained by following the definition of utility functions. • Agent Negotiation Capability = 1. It means at any time, each agent only involve one negotiation game. • Negotiation Game Enable Control. A negotiation game can be launched only after the host agent (who maintains the negotiation game) receives all acknowledge (ACK) signals from its neighboring sensors Ni. The neighboring agent will send reschedule signal if it is involved in another negotiation or some of its status information is updated to request it to participate in the negotiation game. Then the host agent has to update the game information and resend the reschedule signal to enable a negotiation. 2.2 Subgame Perfect Equilibrium Based Negotiation Strategies A fundamental concept in game theory is the Nash Equilibrium [23], which is named after John Forbes Nash, who first proposed it. The concept is a solution concept of a game involving two or more players. In the game, each player is assumed to know the equilibrium strategies of the other players, and no player has anything to gain by changing only his or her own strategy (i.e., by changing unilaterally). If a player has chosen a strategy and no other player can benefit by changing his strategy while the other players keep their strategies unchanged, then the current set of strategy choices and the corresponding payoffs constitute a Nash Equilibrium. In a Nash Equilibrium, each player must answer questions from others and knowing the other player strategies, try to benefit himself from the other player strategies. For example, Tom and Jim are in game with Nash equilibrium, when Tom is making the best decision he can, and taking into account Jim's decision, at the same time, Jim is also making the best decision he can, while taking into account Tom's decision. In the same way, many players are in game with Nash equilibrium if each one is making the best decision that they can, while taking into account the others. Each strategy in a Nash Equilibrium is a best response to all other strategies in that equilibrium. The Nash Equilibrium may sometimes appear non-rational in a third-person perspective as a Nash Equilibrium is not Pareto optimal. The Nash Equilibrium may also have non-rational consequences in sequential games because players may "threaten" each other with non-rational moves. However, in many cases all the players might improve their payoffs if they could somehow agree on strategies different from the Nash equilibrium (NE), which leads the concept of Subgame Equilibrium [24]. Subgame Equilibrium is an attempt to choose a result from the set of Nash Equilibria and a Nash Equilibrium structure will be maintained in each subgame. Nash equilibrium is a normal-form concept, which ignores the sequential structure of play in extensive-form games. As a result NE predicts some equilibria which appear problematic in the extensive form. But, Subgame Equilibrium can avoid these problems by reaching a local optimal. In a sbugame-equilibrium based negotiation, the negotiation bargain will be finished with agreement within two time steps. Using the bargaining strategy in negotiation, we develop a negotiable game-theoretic based sensor management (NG-SMgt) for sensor-to-target assignments that increase target tracking efficiency and effectiveness.
3. SIMUATIONS AND DISCUSSION We used the Low Earth Orbit (LEO) and Geosynchronous (GEO) satellite scenario (Fig. 1) to evaluate our negotiation game based sensor assignment algorithm. There are 4 LEO satellites, namely ARIANE 44L, OPS 0856, VANGUARD 1 an ECHO 1, as observers, and 2 GEO satellites, namely EchoStar 10 and COSMOS 2350, as space objects. The orbits are generated by the General Mission Analysis Tool (GMAT) [25] with the two-line elements from space-track.org.
Proc. of SPIE Vol. 8044 80440C-4 Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
Figure 1: A space object tracking scenario with 4 LEO observers and 2 GEO targets
Figure 2: The same scenario in the close view with details After obtaining the sensor-to-target assignment based on the negotiation game, we used various trackers, such as extended Kalman filter (EKF), unscented Kalman filter (UKF) , and linear minimum mean square error (LMMSE) filter to track GEO targets. We set Qi =0 (no rewards for quiescence mode) and Vi = 1 (both GEOs have same importance). We simulated the system for 1000 time steps and each time step is 50s. The sensor assignment is shown in Fig. 3 (from the point view of targets) and Fig. 4 (from the point view of sensors). The results show that negotiation results in varying the sensor-to-target assignment based on the availability and capability of the sensors to track the targets. Future research will compare the sensor-to-target assignment based on the scenario such as the speed of the targets, Earth blockages, and target importance.
Proc. of SPIE Vol. 8044 80440C-5 Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
Sensor assignment for space object 1 (EchoStar 10) 6 sensor index
Sensor 1
Sensor 2
Sensor 3
Sensor 4
4 2 0 0
200
400
600
800
1000
1200
1400
1600
1800
2000
Sensor assignment for space object 2 (COSMOS 2350) 6 sensor index
Sensor 1
Sensor 2
Sensor 3
Sensor 4
4 2 0 0
200
400
600
800
1000 1200 time (k)
1400
1600
1800
2000
Figure 3: Sensor assignment for each space object. (Sensor i = 0 means the sensor is not assignment to the target)
sensor 1
sensor 2 3 target index (1-2)
target index (1-2)
3 2
1 0
0
500
1000
1500
2
1 0
2000
0
500
sensor 3
2000
1000 1500 time (k)
2000
3 target index (1-2)
target index (1-2)
1500
sensor 4
3 2 1 0
1000
0
500
1000 1500 time (k)
2000
2 1 0
0
500
Figure 4: Target assignment for each sensor. Fig.5 shows the utilities of sensors at each time step. For each target, the minimal eigenvalue of the information matrix at each time step are show in Fig. 6.
Proc. of SPIE Vol. 8044 80440C-6 Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
sensor 1
sensor 2 30 utility of each step
utility of each step
30
20
10
0
0
500
1000
1500
20
10
0
2000
0
500
sensor 3
2000
1500
2000
30
20
utility of each step
utility of each step
1500
sensor 4
25
15 10 5 0
1000
0
500
1000 time (k)
1500
20
10
0
2000
0
500
1000 time (k)
Figure 5: Utilities of the 4 sensors after being assigned to the targets. The tracking results are shown in Fig. 7 and Fig. 8. We can see that LMMSE and UKF are better than EKF in terms of reducing the satellite tracking position errors. Since the system is nonlinear, the linearization used in EKF will produce large errors. More specifically, an ellipse orbit model is not very accurate to describe the GMAT orbital propagators from the target location error (TLEs). However, in the UKF and LMMSE tracker, sigma points based sampling methods are used the approximate the mean and covariance of the nonlinear system. Therefore, in the space object tracking scenario studied here, the UKF and LMMSE trackers are better than the EKF tracker. Future work will model the tracker covariance reduction needs as demands that can be satisfied by the appropriate sensor with the optimal perspective. minimal eigenvalue of the information matrix of space object 1 (Echostar 10)
eigne values
0.2 0.15 0.1 0.05 0
200
400
600
800
1000
1200
1400
1600
1800
2000
minimal eigenvalue of the information matrix of space object 2 (COSMOS 2350)
eigne values
0.2 0.15 0.1 0.05 0
200
400
600
800
1000 1200 time (k)
1400
1600
1800
2000
Figure 6: Minimal eigenvalue of the information matrix of each target.
Proc. of SPIE Vol. 8044 80440C-7 Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
Position errors (km) Position errors (km) Position errors (km)
EKF tracker for space object 1 (EchoStar 10) 100 DeltaX
DeltaY
DeltaZ
0 -100
200
400
600
800
1000
1200
UKF tracker for space object 1 (EchoStar 10) 100 DeltaX
DeltaY
DeltaZ
0 -100
200
400
600
800
1000
1200
LMMSE tracker for space object 1 (EchoStar 10) 100 DeltaX
DeltaY
DeltaZ
0 -100
200
400
600 time (k)
800
1000
1200
Position errors (km) Position errors (km) Position errors (km)
Figure 7: Tracking results for space object 1 (Echo Star 10). DeltaX, DeltaY and DeltaZ are tracking errors in X, Y and Z axis, respectively. EKF tracker for space object 2 (COSMOS 2350) 100 DeltaX
DeltaY
DeltaZ
0 -100
200
400
600
800
1000
1200
UKF tracker for space object 2 (COSMOS 2350) 100 DeltaX
DeltaY
DeltaZ
0 -100
200
400
600
800
1000
1200
LMMSE tracker for space object 2 (COSMOS 2350) 100 DeltaX
DeltaY
DeltaZ
0 -100
200
400
600 time (k)
800
1000
1200
Figure 8: Tracking results for space object 2 (COSMOS 2350). DeltaX, DeltaY and DeltaZ are tracking errors in X, Y and Z axis, respectively
4. CONCLUSIONS We proposed a negotiation game approach for distributed sensor management. Our approach has several features. First, it is a Distributed agent system. Each agent represents a physical sensor. Each agent hosts and maintains a negotiation game. By playing the game with its neighboring agents, the host agent generated a self-enforced target assignment among the players of the game. All the decisions are made locally. Second, target tracking performance metrics and trust values are integrated in the utility (or objective) functions of agents. We build the utilities based on the sensor gains and target values, which can be straightforwardly replaced with different metrics. Third, there is no requirement on the number of targets and sensors. No pseudo sensors are needed to handle the cases with more targets than sensors. We implemented the trust-based game negotiation algorithm and simulated a space scenario with 4 LEO observers and 2 GEO objects.
Proc. of SPIE Vol. 8044 80440C-8 Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
REFERENCES [1] Bar-Shalom, Y., and Li, X., [Multitarget-Multisensor Tracking: Principles and Techniques], YBS Publishing, (1995). [2] Kirubarajan, T., Bar-shalom, Y., Blair, W. D., and Watson, G. A., “LMMPDAF for Radar Management and Tracking Benchmark with ECM”, IEEE Transactions on Aerospace and Electronic Systems, 34 (4), 2616-2620, (1995). [3] Watson, G. A., Blair, W. D., and Rice, T. R., “Enhanced Electronically Scanned Array Resources Management through Multisensor Integration”, Proc. SPIE 3163, (1997). [4] Manyikia, J., and Durrant-Whyte, H. F., [Data Fusion and Sensor Management: A Decentralized InformationTheoretic Approach], Prentice Hall, (1994). [5] Rao, B. S. Y., Durrant-Whyte, H., and Sheen, A., “A Fully Decentralized Multisensor System for Tracking and Surveillance”, International Journal of Robotics Research, 12(1), 20-45, (1991). [6] Utete, S., [A Network Manager for a Decentralized Sensing System], Master’s thesis, University of Oxford, (1992). [7] Shen, D., Chen, G., Blasch, E., Pham, K., Douville, P., Yang, C., and Kadar, I., “Game Theoretic Sensor Management for Target Tracking,” Proc. SPIE 7697, (2010). [8] Wei, M., Chen, G., Cruz, J. B., Haynes, L. S., Pham, K., and Blasch, E., “Multi-Pursuer Multi-Evader PursuitEvasion Games with Jamming Confrontation”, AIAA Journal of Aerospace Computing, Information, and Communication, 4(3), 693–706, (2007). [9] Chen, G., Shen, D., Kwan, C., Cruz, J., Kruger, M., and Blasch, E., “Game Theoretic Approach to Threat Prediction and Situation Awareness,” Journal of Advances in Information Fusion, 2(1), 1-14, (2007). [10] Shen, D., Blasch, E., Chen, G., Pham, K., Cruz, Jose B., Schuck, T., “A Markov Game Model for Space Threat Prediction”, Proc. SPIE 6970, (2008). [11] Li, D., Chen, G., Blasch, E., and Pham, K., “Sensor Attack Avoidance: Linear Quadratic Game Approach”, Int. Conf. on Info Fusion - Fusion09, (2009). [12] Chen, H., Chen, G., Shen, D., Blasch, E. and Pham, K., “Orbital evasive target tracking and Sensor Management”, Ch 12 in [Dynamics of Information Systems: Theory and Applications], M Hirsch, P.P. Pardalos, and R. Murphy (Eds.), Springer (2009). [13] Chen, G., Blasch, E., Chen, H., and Pham, K., “Multi-Agent Modeling and Analysis Framework for Space Situational Awareness”, SPIE Newsroom, (2009). [14] Chen, H., Chen, G., Blasch, E., and Pham, K., “Comparison of several space target tracking filters”, Proc. SPIE 7730, (2009). [15] Chen, H., Shen, D., Chen, G., Blasch, E., and Pham, K., “Space Object Tracking with delayed Measurements”, Proc SPIE 7691, (2010). [16] Shen, D., Tian, Z., Chen, G., Pham, K., and Blasch, E., “Information-based awareness model for active sensing and resource management”, Proc. SPIE 7698, (2010). [17] Chen, H., Shen, D., Chen, G., and Blasch, E., “Tracking Evasive Objects via A Search Allocation Game,” IEEE ACC, (2010). [18] Blasch, E., Chen, G., Shen, D., Chen, H., and Pham, K., “Services Oriented Architecture (SOA) based Persistent ISR Simulation System”, Proc. SPIE 7694, (2010). [19] Xu, P., Chen, H., Charalampidis, D., Shen, D., Chen, G., Blasch, E. and Pham, K., “Sensor Management for Collision alert in Orbital Object Tracking”, Proc. SPIE 8044, (2011). [20] Mayer, R. C., Davis, J. H., and Schoorman, F.D., “An integrative model of organizational trust”, Academy of Management Review, 20(3), 709-734, (1995). [21] Sztompka, P., [A Sociological Theory], Cambridge University Press. (1999). [22] Gambetta, D., “Can we trust trust?,” in Gambetta, Diego (ed.) [Trust: Making and breaking cooperative relations], electronic edition, Department of Sociology, University of Oxford, 213-237, (2000). [23] Nash, J., “Two-person cooperative games,” Econometrica, 21,128-140, (1953). [24] Osborne, M. and Rubinstein, A., [A course in game theory], Cambridge, Massachusetts: MIT Press, (1994). [25] GMAT, http://gmat.gsfc.nasa.gov/
Proc. of SPIE Vol. 8044 80440C-9 Downloaded From: http://spiedigitallibrary.org/ on 10/22/2013 Terms of Use: http://spiedl.org/terms
