Abstract. The long term the goal of our research is to enable control of interactive wargames through use of high-level commands using agents to command ...
An Architecture to Support Autonomous Command Agents for OneSAF Testbed Simulations F. Lui, R. Connell, J.Vaughan DSTO Land Operations Division D.Jarvis and J.Jarvis Agent Oriented Software Pty. Ltd. Abstract. The long term the goal of our research is to enable control of interactive wargames through use of high-level commands using agents to command each subordinate level of the hierarchy in the simulation in accordance with military doctrine - including reacting to enemy actions, terrain, and reporting back up the command chain. The initial goal of the work is to demonstrate agents which reduce the need for intervention by human OneSAF Testbed Baseline (OTB) operators when running simulations. The command agent described will replace a single level of the command hierarchy, with the OTB engine being responsible for the low level behaviours. This paper describes the architecture adopted for the command agent, its implementation, capabilities and interface with OTB. Finally directions for further research are discussed. 1.
Introduction
The applications of Intelligent Agents1,2 in Synthetic Environments3 have attracted much attention over the last decade. Land Operations Division, DSTO, is developing Command Agents to model the behaviours of company commanders for use in wargames. The command agents will be external agents that can take control of the entities within the simulation. Like other agent architectures4, these agents will take an instruction, work out the context, plan how to carry out the operation and assign tasks to subordinate agents. In addition, they receive information back from the wargame — for example detection of enemy, fuel and ammunition status — and use this to build situation awareness and to respond to unforeseen situations. Hence, these agents will take over many functions of human operators in wargames. In this paper, we will outline the requirements for the agent, describe its design and implementation and describe its interface to the OTB. Information will be provided on how we elicited military domain knowledge for the agent’s rule base and on aspects of terrain awareness. Lastly, we will discuss our current and future work. 2.
Background
2.1 Simulation of a company commander using the command agent In this work, we are concentrating on a company attack scenario in which the agent is to control a motorised infantry company. The command
agent is intended to carry out tasks now performed by a human OTB operator. The role of the operator is to enter commands for the control units and entities through a Graphical User Interface (GUI). The OTB GUI comprises a map and icon display, a series of context-based editor windows and a series of pull-down menus. The operator and commander can use the graphical representation of the wargame on the GUI to acquire an adequate level of situation awareness, undertake terrain appreciation and monitor the progress of the battle. In current use, objectives given to the OTB operator who follows military doctrine and standard procedures to produce plans for courses of actions to be taken during the battle. With command agents, the objectives of the attack are inputs to the agents. The agents will then use their reasoning and rules, which are based on military doctrine, to produce plans and courses of action. Following this, the tasks and commands are sent to the simulation in a format that is understood by the simulation program. During execution of the task, the agent monitors progress and makes any necessary adjustments to the plan, thereby fulfilling the supervision and control function of the OTB operator. 2.2 Concept demonstration of the command agent software To demonstrate the functionality and capability of the command agent, we have chosen a deliberate attack scenario (see Figure 1). The objective is for a mounted infantry company located at Point A to attack an enemy formation occupying a position in the vicinity of Point B. The company commander agent is to produce a
plan and courses of action to carry out four phases for the attack, namely 1) preparatory, 2) assault, 3) exploitation and 4) reorganisation5. The company organisation consists of three platoons. Each platoon is comprised of three sections, each of which has nine soldiers and an armoured personnel carrier (APC). To prosecute the attack, the company splits into a fire support platoon and two assaulting platoons. The agent plans the routes, form-up positions and coordination parameters for the attack. The agent will monitor the location and status of its own troops and the enemy and will respond to situations which require changes to the plan. In planning and executing the attack, the agent will apply documented military doctrine5 and make appropriate use of terrain.
decisions relating to individual platoon member behaviour (eg maintain formation, contact drill) are made by the behaviour models within the simulation. The platoon command agents are aware of which simulation entities are under their control and monitor their position and status. This information is used to make local decisions. The platoon CAs interact with their company command agent; they receive commands and send aggregated reports in accordance with appropriate military doctrine. The company CA is responsible for planning and executing a mission; in this regard it determines the tasks that are required to be performed and allocates them to platoons. The command agent architecture is summarised in Figure 2. Note that the focus of this diagram is the command agent layer. The interface with OTB is discussed in Section 4 while the human player interface will be the subject of future research (Section 7).
A OTB
CA01
Fire support
CA11 CA02
B Dispatcher
CA03 CA12 CA04
Figure 1: The scenario for demonstrating the agent capabilities. The objective is for a mounted infantry company located at Point A to attack an enemy formation occupying a position in the vicinity of Point B.
3.
Software architecture of command agent
3.1 Command agents The command agents (CA) operate within a well-defined command and control structure which can be modelled as a hierarchy of teams6. This command and control structure is also expressed within the wargame simulation at the lower levels constructive entities and by human players at the upper levels. As an example, one may choose to model battalion behaviour using human players, company and platoon behaviour using command agents and platoon member behaviour using OTB entities. In this situation, the platoons are represented in both the simulation layer and the command agent layer. Decisions regarding platoon behaviour (eg move to form up point, retreat) are made by the command agents, but
Level0
Level1
Human Player
Figure 2: Command agent Layer Architecture. Levels 0 and 1 refer to the level within the command and control hierarchy (eg Platoon and Company)
In Figure 2, Company CA11, consists of two Platoons, CA01 and CA02. The members for CA01 and CA02 are modelled in the simulation layer and are not shown. Orders and reports are exchanged between platoon agents CA01 and CA02 and their members via the dispatcher over bi-directional links. Within each command agent, we identify three key capabilities: 1) Planning for action, 2) Control of action and 3) Reporting on action. These three capabilities operate on a shared belief structure that contains the command agent’s current beliefs regarding the world. The architecture is summarised in Figure 3.
4) extensive reasoning over plan failure,
The planning-for-action capability allows the command agents to take a command from the level above and by using the appropriate military doctrine, generates commands for the entities under its direct control. The progress of the resulting action is then monitored by the control of action capability. Implementation of the planning and control capabilities is described in Section 3.2. The reporting on action capability provides reports back to the command level. The content of these reports is derived from messages provided by the subordinates using the message aggregation process as described in Section 3.3.
5) team reformation and re-organisation, 6) connection of beliefs between teams up and down the command hierarchy; and 7) autonomy at each command hierarchy level. 3.3 Message aggregation Message aggregation (MA) performs the role of aggregating data derived from the entity models into reports issued at platoon or higher levels. MA Reporting serves two purposes:
Level+1
1) Reducing the volume of raw data on target acquisitions produced by the entity models which otherwise cannot be used by the operator or agent, and 2) Formating the data into accepted military reports.
Beliefs Reporting On Action
Planning For Action
The following reports are produced
Control Of Action
1) 2) 3) 4) 5)
Level-1
3.2 JACK Teams The command agent layer is implemented using JACK Teams6, which is a team modelling framework based upon the BDI (Belief, Desire, Intention) reasoning model and implemented as a model extension on JACK Intelligent Agents™. A key to the BDI approach is the emphasis on ‘intentionality’; that a reasoning entity adopts its plans of action with the intentions to satisfy its goals. JACK Teams builds on the Team Based Agent Technology (TeBAT) framework6 that was developed for use in computer simulations of military tactics and equipment, in particular those of the Australian Army5. Like TeBAT, JACK Teams provides mechanisms for modelling key aspects of team operations as they apply to land operations, including: 1) a hierarchical command structure,
4. Interface between OTB and command agents In distributed wargame simulations, such as OTB, data is shared between networked computers. OTB uses the Distributed Interactive Simulation (DIS) protocol for this purpose. In this protocol, Protocol Data Units (PDU) such as entity state, fire, detonation, collision, signal, event report etc are broadcast onto the network. The shared information allows each networked computer to generate a model of the battlespace. Enemy detections
Command Agents
OTB Sockets
Figure 3: Command agent Architecture. Level-1 may contain command agents or simulation entities; Level+1 may contain command agents or human players.
Situation Report, Location Status Report, Hostile Air Report, End of Hostility Report, and Contact/Incident Report.
DIS PDUs
Commands
Entity status PDU
DIS
2) team oriented activities, 3) team intentions,
Figure 4: Block diagram of the connections between OTB and command agents.
In this work, the command agents only require the location, heading and velocity data of controlled entities which can be derived from the Entity State PDU. In addition, they require information on enemy entities. This is generated by the entity target acquisition models within OTB, but the data is not available within DIS PDUs. Modifications to OTB source code have been made to output target acquisition data via a socket (see figure 4) so that an external agent can have access to the information. A second socket is used by the CA to to send commands back into OTB.
to the wargame scenario and so on. What is missing here is the impact that the enemy’s activities and doctrine have on the company commander’s decisions. In wargame simulations, this is classified as the enemy’s beliefs and intent. Very often, this information is very hard to determine using the rules that are built into the agents. In order to achieve a consistency and coherence in information flow throughout the simulation, it is necessary to extract only those rules which are relevant to the context in the wargame scenario, from the CWA model.
5.
6.
Military Doctrine: Knowledge Elicitation
Developing a command agent to undertake the required tasks necessitates an adequate knowledge of military doctrine in a form which can be built as rules for the operation of the agent. High fidelity knowledge elicitation methods7 can be used to extract rules and reasoning from specific military expert domains. This was achieved by using a Cognitive Wwork Analysis (CWA) approach8. This process requires several interviews with experienced company commanders. During the interviews, they are presented with questionnaires that are designed to elicit knowledge of how decisions are made during combat. The CWA model is comprised of work domain analysis, control task analysis, strategy analysis, social organisation analysis, worker competency analysis and the temporal responses at each hierarchy levels such as Platoon Leaders and Company, Battalion and Brigade Commanders. The advantage of CWA is that it takes into account the uncertainties in human decisionmaking. As we have mentioned in the above, this approach will create a high fidelity cognitive model, which describes how, in the real world, an initial event can lead to various courses of action which are comprised of a number of plans. However, one must bear in mind that wargames, such as OTB and CAEn, have limited fidelity. For instance, in real life, when an infantry unit is performing a reconnaissance mission, its tasks are to patrol a specified region and to report on the enemy’s status and position. The infantry unit tasks include report enemy detections, avoid line-of-sight (LOS) with its opposition and report the type of enemy weapons used, how they are used, the enemy’s strength and activities etc. The command agent’s tasks are, therefore, 1) to move the infantry unit to the destination, 2) check for LOS with enemy forces, 3) report on enemy detection, 4) collate the information and put them in context relevant
Terrain interactions
In the command agent, a rule-based reasoning capability is used to generate plans. The requirement for plans to follow established military doctrine has been discussed. The application of doctrine is influenced by terrain and the agent must be able to make decisions based on terrain awareness. Reasoning of the agent therefore combines military doctrine with terrain awareness and route-finding algorithms (see Figure 5). 4 phases •Preparatory •Assault •Exploitation •Reorganisation
Doctrine Terrain data
Deliberate Attack Route planning
Entity status Fire support
Command Agent
Commands
Figure 5: Block diagram of the scenario set up for a company attack. For example, in the preparatory phase, the command agent needs to work out routes on which the friendly troops are less likely to be detected by the enemy forces. For the implementation used here, this requires the route-finding algorithm to construct a matrix overlay for the terrain map by which a weighting system can be applied to determine the best path. Secondly, the command agent needs to determine the positions from which the friendly troops, namely two assault platoons and one fire-support platoon, are going to be launched during the assault phase. In the following sections, we will discuss the two processes, namely terrain appreciation and route-finding algorithm. 6.1 Terrain appreciation The command agent being developed is situated in a ground environment. As such, it needs
linkages to the terrain upon which it must act. In the real situation, the most successful commanders are able to read the terrain of the battlefield and make it work to their advantage; therefore the command agent must possess the same ability. This reasoning is part of the Military Appreciation Process (MAP), a process whereby the commander inspects the terrain upon which the battle will take place and creates overlays of different information; mobility corridors, key areas, high ground, that will influence the planning. The command agent will also follow this process and, to this end, we have been developing algorithms for calculating view sheds, planning routes and extracting features from the terrain data of relevance to the MAP. 6.2 Route finding The first terrain-aware application that the command agent will require is that of route finding. We have developed a tool that allows us to write specialised heuristics for navigating through terrain. These heuristics take into account the underlying terrain as well as any overlays that may be placed onto the map (see Figure 6). These overlays can consist of any conceivably useful information, mobility corridors, enemy weapon ranges, water sheds, anything that the specific heuristic may need to provide a useful route.
2) The Terrain data and route finding algorithm is being implemented and integrated into the command agent to determine routes to potential targets and objectives. From this, the command agent can assign tasks (such as move vehicles, attack a target, occupy an area etc) to the motorised infantry units. 3) Message aggregation is being incorporated into the command agent. This feature will give the command agent the capability of aggregating the raw data which is sent by the simulation, and to generate reports such as those mentioned in Section 3.3. 4) JACK-Teams6 is incorporated into the command agent. In doing so, we are utilising the features of teams and message aggregation in order to carry out the functions of a company commander during wargames. 5) Knowledge acquisition by consultation with military personnel is carried out. Rules for Preparatory Phase are acquired and they can be used in the command agent to demonstrate the functions and capabilities of a company commander agent. 6) The OTB interface, which provides the linkage to JACK agents, is being implemented. This completes the closed loop simulation, as depicted in Figure 4. JACK agents can receive DIS PDUs and detection messages as well as sending commands into OTB to take control of the constructive entities. Future work
Figure 6: An overlaid picture from path finding algorithm using the environmental data.
7.
Current and future work
Current work Currently, we are in the process of progressing the following: 1) Preparatory Phase in a combat attack scenario is being modeled with the assumption that it has preconceived beliefs and assumptions about the enemy’s positions. This will allow the command agents to produce plans and courses of action using the rule based reasoning capability.
In the future, it will be desirable to implement the assault, exploitation and reorganisation phases in order to fulfill further requirements of military doctrine. We will be devising the rules and temporal responses, which are obtained from CWA, to implement a high-fidelity model that incorporates the rule-based reasoning capabilities. Vigorous tests will be carried out in order to investigate the scalability of the command agent. This will provide us with a better understanding of the limits and constrains of the JACK-based architecture programming technique. 8.
Conclusion
In conclusion, we are currently integrating all the required components for the command agent namely; JACK-Teams, message aggregation, OTB interface and terrain appreciation. We have incorporated message aggregation function, terrain data and a route-finding
algorithm so that it is able to provide an overlay map which specifies the potential target locations given the raw target acquisition data which is sent in via the socket and Entity-State PDU. Although we have not fully implemented the human behaviours at this stage, a basic set of rules for determining the form-up positions has been incorporated. At the completion of this work, the command agent will be able to demonstrate human-like decision-making based on the BDI reasoning model in JACK-Teams. 9.
Acknowledgements
We would like to acknowledge Mr Ion Menadue for proof reading this paper. We would also like to thank Messrs Matthew Lyons and Peter Hanson for their efforts in implementing OTB’s interface which provides the linkage to the command agents. 10. References 1
Jörg P. Müller, “The Design of Intelligent Agents, A layered Approach”, 1996, (Lecture notes in computer science; vol 1177: Lecture notes in artificial intelligence), ISBN 3-54062003-6. 2 Ralph Rönnquist, Andrew Lucas and Nick Howden, “the simulation agent infrastructure (SAI) – Incorporating intelligent agents into the CAEn close action simulator”, SimTecT 2000 Conference Proceedings, Sydney Conventional and Exhibition Centre, Darling Harbour, Sydney, Australia, p83-87. 3 Anne-Marie Grisogono and Eyoel Teffera, “Towards a Synthetic Environment Design Methodology”, SimTecT 2000 Conference Proceedings, Sydney Conventional and Exhibition Centre, Darling Harbour, Sydney, Australia, p273-278. 4 http://www.soartech.com/ 5 Manual of land warfare part two infantry training, Vol. 1, Pamphlet 2, The Rifle Platoon, 1986, (MLW 2-1-2). 6 Lucas, A., Rönnquist, R., Howden, Hodgson, A., Connell, R., White, G. and Vaughan, J., “Towards Complex Team Behaviour in MultiAgent Systems”, Proc. SimTecT 2001. 7 Dan Diaper, “Knowledge elicitation principles, techniques and applications”, Ellis Horwood Limited, Publishers, Chichester, 1989. 8 Penelope M. Sanderson and Marcus Watson, “Cognitive work analysis and analysis design and evaluation of human-computer interactive systems”, OZCHI 98, designing the future, conference proceedings, 30th Nov to 4th Dec 1998, Adelaide, South Australia, p220-227.
