to develop agent applications by up to 40 percent. 1 Introduction. Itineraries are frequently used as a control structure in mobile agent systems, because.
Rapid Application Development Using Agent Itinerary Patterns Daria Chacón, John McCormick, Susan McGrath, and Craig Stoneking Lockheed Martin Advanced Technology Laboratories 1 Federal Street A&E 3W Camden, NJ 08102 {dchacon, jmccormi, smcgrath, cstoneki}@atl.lmco.com
Abstract. The behavior of mobile agents is often prescribed by a set of tasks represented in an itinerary. We have used a variety of agent itinerary styles, ranging from simple, sequential expressions to complex, finite state-based representations. The design and implementation of an itinerary can be a complex, time intensive task, particularly in mobile agent architectures, where resources and task execution occur in a distributed network. To address this issue, we have developed a set of reusable, flexible itinerary patterns that enable the rapid development of complex agent itineraries. Used in conjunction with a task library, these itinerary patterns have reduced our time to develop agent applications by up to 40 percent.
1 Introduction Itineraries are frequently used as a control structure in mobile agent systems, because some sort of checkpointing or discrete separation among tasks is useful in supporting weak mobility. Moreover, mobile agents perform tasks at locations, and itineraries very intuitively represent such behavior. A large number of Java-based, mobile agent architectures use itineraries to represent agent behavior, including the Voyager [1], Concordia [2], Aglets [3], and Jumping Beans [4] commercial architectures. These itineraries typically consist of a list of tasks to be executed sequentially and the locations where the tasks are to be performed. We found the sequential itinerary to be too inflexible for many of our applications, because it is difficult to program an agent to respond to environmental conditions when the sequential nature of the itinerary constrains it to performing a list of tasks in order. To address our need for more flexible agent behavior, we began developing itineraries based on finite state machine theory. In the finite state-based approach [2], the itinerary is treated as a metaprogram, allowing the programmer to design states at which tasks are performed and transitions to next states contingent on results obtained from the execution of the first state. While use of finite state-based itineraries furthered our goal of allowing agents to modify their behavior based on events in their environment, we found that designing and implementing finite state-based itineraries was often time consuming
and difficult. Consequently, we sought to simplify itinerary design and development by producing a library of reusable itinerary patterns and associated tasks. While formulating our approach for development of the pattern and task library, we saw that certain agent behaviors were required for many of our existing applications and would be required for future applications as well. Many of our previous projects required agents to persistently monitor for different conditions and react to their occurrence. This kind of agent activity has become known as “sentinel” behavior. We saw applications for this behavior in many military problems, including the time critical targeting process, air mobility operations, and discovery of mobile targets. With the need for sentinel-style persistent monitoring, we developed the Sentinel pattern, a behavior template superimposed onto a finite state itinerary. We also created a second, somewhat more general, variant of the Sentinel pattern. In this paper, we present the work related to the design and development of these patterns and we discuss in depth an application in which we have used them.
2 Extendable Mobile Agent Architecture We have based our agent work on the Extendable Mobile Agent Architecture (EMAA) [5], a Java-based mobile agent architecture developed in 1997 by Lockheed Martin Advanced Technology Laboratories (LM ATL). We have used EMAA on over a dozen projects. Figure 1 depicts EMAA’s basic architectural components. At EMAA’s core is an agent Dock that provides an execution environment for agents, handles incoming and outgoing agent transfers, and allows agents to obtain references to services. EMAA allows users to define agents, services, and events; agents may be mobile and may make use of stationary services, and both services and agents may generate and receive events. Services may implement connections to external systems, may themselves provide some complex functionality, or may carry out any number of other functions, so long as they are not themselves primary actors. Goal orientation and directed activity is generally reserved to be the function of agents.
3 Finite State Itineraries EMAA agents typically use itineraries based on the concept of finite state automata, if they must perform complex tasks. These flexible control structures allow agents to repeat tasks under certain conditions, to perform one task rather than another based on the previous task’s output, and in general to react to conditions of the computer network. We have modified the finite state-based itinerary concept slightly since it was originally described [6]. Rather than specifying with each task the machine on which the task must be performed, an agent can now make that decision at runtime based on a supplied list of hosts or required resources. Implementation of EMAA’s finite state-based itinerary requires the definition of several concepts and corresponding classes:
Agents
Agents
Events Events
Agent Agents
Servers
Local Resource
Servers
Local Resource
Fig. 1. The basic components of the EMAA architecture are the Dock, Servers, Agents, and Events
• Configuration—this is a convenience wrapper for a State and a set of Transitions from the state. • State—this generically represents an action to be performed; it may in reality be multiple actions (tasks) but, in terms of finite state automata, it is a single operation. • Transition—this is comprised of a Condition and a “next Configuration” to be executed if the Condition is fulfilled. • Condition—this piece of code must define a boolean Condition, and it must return the result of the Condition evaluation. • Task—this piece of code defines an operation to be performed, and it is wrapped by a State. Informally, a finite state-based itinerary consists of a set of States and a set of Transitions. After a State is executed, each associated Condition evaluates its output. When a Condition evaluates to “TRUE,” its corresponding State (encapsulated in a Configuration object) executes. It is possible that multiple Conditions could evaluate to “TRUE”; if so, then the first such Condition is accepted and followed. Figure 2 shows the relationships among Configurations, States, Tasks, Transitions, and Conditions. The result of the evaluation of a condition occurring in real-time causes an agent to enter a state and execute the program represented by that state.
4 Itinerary Patterns Patterns have often been used to describe interactions among agents; e.g., adapters, mediators, negotiators, brokers, reasoners, etc. [7]. These patterns provide well tested guidelines for designing agent relationships. However, they do not seem to be intended to speed the development of individual agents. Aridor et al [8] mentions the
Configuration Transitions
Configuration
State Task
Transitions State Task
Task result: TRUE
{
}
Condition: result = TRUE Next Configuration Condition: result = FALSE Next Configuration
Condition Next Configuration Condition Next Configuration
Configuration Transitions
State Task
Condition Next Configuration Condition Next Configuration
Fig. 2. The finite state-based itinerary components: A Task executes within a State and the Task execution results determine which Transition is followed. A Transition consists of a Condition to evaluate and a consequent next Configuration to execute
itinerary itself as a reusable pattern implementation, but it has been our experience that entire itineraries are rarely reusable; rather, one or more tasks can be reused but they must be combined with other tasks and performed at a different destination. Patterns for itineraries have not been well addressed in agent literature, largely because most agent systems use sequential itineraries that do not lend themselves well to abstraction and generalization. Tahara et. al. have recently proposed a development method for agent systems that incorporates behavior patterns into macro-architectural, micro-architectural, and object-level design [9]. Strategies for doing macroarchitecture-level design include describing agent behavior patterns using state transition diagrams. Finite state-based itinerary patterns allow general designs of macro-architectural agent behavior to be almost directly translated into runnable code. 4.1 The Sentinel Agent Behavior Pattern Figure 3 shows a Sentinel pattern that specifies an itinerary for persistent monitoring and reaction to the occurrence of a monitored condition. It supposes that a
Reaction TA T Start
Info Transport
Post Processing
M
Evaluation
MA
Continuation Check
F
F FA
T
Halt
Fig. 3. This Sentinel pattern describes persistent monitoring activity
prototypical persistent monitoring behavior will include retrieving information, postprocessing to place data in an evaluatable form, evaluating whether the condition has occurred, then reacting accordingly. For our Sentinel pattern, we expect that the evaluation state will tell us if the condition did or did not (yet) exist or can never exist. If the condition is not detected on a first pass, then it may be beneficial for the software process to continue those same monitoring behaviors, either indefinitely, until the condition has been satisfied, or until some other constraint has been met. The final state is a decision about whether or not to continue (i.e., rerun the itinerary pattern). The Sentinel pattern specifies a control flow for this process, which could be implemented by a single agent or, for situations where input data or needed processes are distributed, by multiple collaborating agents. 4.2 The Simplified Sentinel Agent Behavior Pattern In a number of situations, we found that the Sentinel pattern was not well suited for solving a persistent monitoring problem. We encountered situations in which one could not say that a single condition was true, false, or simply not yet true; rather, the condition was true to some degree, which necessitated an appropriate reaction. Recognizing this, we created the Simplified Sentinel pattern, shown in Figure 4, which is a variant of the Sentinel pattern. A programmer using the Simplified Sentinel pattern communicates the results of the Evaluation configuration through the encapsulating agent’s internal WhiteBoard; the Reaction configuration then analyzes and responds to those results, accordingly. 4.3 EMAA Support For Behavior Patterns To support agent patterns, it was necessary to overcome a limitation of the finite state automata construct. As stated theoretically, finite state automata, do not have memory; one state cannot obtain data processed by a previous state. To allow an agent to analyze and react upon data, however, one state must be permitted to pass on data
Start
Info Transport
Post Processing
Evaluation
Reaction
Continuation Check F T
Halt
Fig. 4. This Simplified Sentinel pattern handles monitoring situations where the Reaction state must handle many different possible Evaluation outputs
to succeeding states. Our implementation of pattern support uses a WhiteBoard inside the patterned agent to store data that must be transferred among states. Allowing a single State in a finite state itinerary to contain multiple Tasks can also be useful. For example, a single agent might need to retrieve data from two different databases on a single machine; each query to a different database represents a separate Task. We support this by allowing a single State to contain an arbitrary number of Tasks, providing that all Tasks may be performed at the same location. Moreover, it is necessary for an application to be able to exert some measure of control over agents; our implementation allows patterned agents to be easily controlled and monitored. Passing data among functional blocks. For agent-itinerary patterns to have any real power, an itinerary can no longer consist of a list of independent tasks; the itinerary must allow succeeding tasks to react to the output of preceding tasks. Using the Sentinel pattern as an example, the output of the information retrieval process must be available to post-processing and the output of post-processing must be available to the evaluation process, etc. Therefore, there must be some ability for tasks input by the user to pass data to other tasks within an itinerary. To solve this problem, a patterned agent carries with it a WhiteBoard that is invisible to Tasks but that supports interTask communication. The State object encapsulating the Task (or list of Tasks) acts as middleware among Tasks and a WhiteBoard contained in the agent. The WhiteBoard is used as a Hashtable, wherein tasks associate data to be passed around with a tag and other tasks may (indirectly) request the data associated with that tag from the WhiteBoard. Tags identifying the data to be passed among Configurations must be defined as the agent is being created. Tasks supplied by a programmer must conform to the IOTask interface; this requires a Task to implement the following three methods: public Vector getInputDataRequirements() public void setInputData(Hashtable) public Hashtable getOutputData()
The getInputDataRequirements method is called on the Task by the encapsulating State to elicit the WhiteBoard elements needed as input data. In response to this call, the Task should return a Vector containing the tags or keys identifying the data it will need. The State then retrieves those elements from the agent’s internal WhiteBoard and delivers them to the Task in the form of a Hashtable by calling setInputData.
After the Task’s execute method is called and all processing is performed, the State calls getOutputData to obtain a Hashtable containing the keys and values that the Task is passing on to succeeding Tasks (see Figure 5). Agent
Agent Internal WhiteBoard MOGASSIGNMENTS
execute() State getOutputData() {the QueryTask returns a State calls these Hashtable containing methods on task: key: QueryTask ÓMOGASSIGNMENTSÓ, value:Jtable}
{
Transitions
{JTable containing assignment records}
getInputDataRequirements() {the ProcessingTask returns a Vector containing one element: ÒMOGASSIGNMENTSÓ.} setInputData() {the state calls this method on the ProcessingTask with a Hashtable containing key:MOGASSIGNMENTS, value: JTable
}
State calls these methods on task: QueryTask State
Transitions
getOutputData() {the ProcessingTask returns a Hashtable containing key:ÓPROCRESULTSÓ, value:Jtable} Information Retrieval Configuration
Post-Processing Configuration
Fig. 5. Passing data from one functional block to the next: A State gets output data from its enclosed Task(s) and puts it on the Agent’s internal WhiteBoard so it is available for later States and their contained Tasks
Multiple Tasks Per State. The goal of a single functional block (e.g., Information Retrieval, Post-Processing, etc.) may require the execution of more than one separate action; for example, information retrieval may require multiple queries to different types of local data sources. One could program all of the needed queries into one task, but this severely diminishes task modularity and reusability. The State object developed to support agent patterns allows a programmer to supply multiple Tasks for execution. All Tasks supplied to a single State will be executed on the same machine. Agent Control. One of the key differences between a virus and an agent is that a virus is almost uncontrollable, while an agent should be controllable at all times by the force that created it. EMAA agents can be stopped, paused, suspended, and resumed during execution by means of distributed events. (Ref. [10] describes the Distributed Event Messaging System, an extension to EMAA that provides reliable event delivery to mobile agents.) When designing the components that support the Sentinel pattern, we particularly wished to enable users to retask pattern-based agents. Therefore, we built into the support components the ability to handle basic control events, including pause, suspend, resume, and stop. The classes that define the Sentinel and Simplified
Sentinel patterns add support for agent retasking; one or more Tasks within a given State may be replaced with Tasks of the same type. We added the typing constraint because permitting a Task class to be replaced by any other Task class can introduce runtime errors that will be difficult to debug. A later State may be expecting input from a Task; if it is replaced with a Task of a different type that generates no output or different output, then the agent’s internal WhiteBoard will be in an unexpected state and the agent will not function properly. Hence, we have essentially supported only parameter level retasking. In summary, finite state itineraries increase the decision-making power of agents. Itinerary patterns, a new research area, generalize certain finite state itineraries into itinerary templates that can be used for many applications. Tasks can be pulled from a library of reusable tasks and dropped into the pattern to create an application. We have developed a preliminary pattern development layer atop EMAA that allows programmers to quickly develop new itinerary patterns with certain built-in capabilities: memory among finite states, multiple tasks per state, and support for agent control, including retasking.
5 Applications To date, the Sentinel itinerary pattern has been used in multiple military applications requiring various forms of Sentinel-style persistent monitoring and alerting. For the Cooperating Agents for Specific Tasks (CAST) program, funded by the Defense Advanced Research Projects Agency (DARPA), we used patterned agents to monitor targets as they under went time critical targeting [11]. Agents monitor the progress of targets through the targeting pipeline, raising a flag if there is insufficient information to complete the targeting process. On DARPA’s Listen, Communicate, Show (LCS)—Marine program [12], we used sentinel patterned agents to monitor for changes in the status of logistics requests. We have used patterned agents in a prototype system for the U.S. Air Force Air Mobility Command (AMC) to ensure that air plans were not interrupted by last minute scheduling conflicts; we discuss this application below. The Air Mobility Operations project provides a clear example of a complex application that was implemented in an extremely short time with Sentinel patterns. We developed a functional prototype in two months; ordinarily, it would have taken three to four months. The use of patterns across several projects cut development time up to 40 percent. Patterned Agents in Air Mobility Operations. We have worked with AMC, using Sentinel patterned agents to discover and alert on conflicts in air mobility plans. The AMC’s mission is to provide airlift, air refueling, special air mission, and aeromedical evacuation for U.S. forces. The AMC manages several types of missions. Channel missions are standard supply routes that recur on a consistent basis; they are typically planned one to three months in advance. Contingency missions are critical emergency operations that take precedence over channel operations, and they may be planned only a few hours in advance.
Planners are currently unable to monitor for mission conflicts after missions have been submitted to the Air Plan Database. The most common conflicts encountered by planners are Maximum On Ground (MOG) violations. These situations often occur when existing air plans are not considered as contingency missions are planned, resulting in over-MOG conditions. This MOG Validation Agent is a variation on an agent used in the AMC application; it checks for MOG conflicts for a single mission flight plan. Figure 6 shows a state diagram of the MOG Validation Agent. Reaction Notify operator: mission valid
Info Transport Start
Query Air Plan DB
T Generate Reservations
Look for Conflicts
M
No Action
F Notify operator of mission conflicts
Air Plan Database
Continuation Check Halt upon conflict, else wait(N)
F T Halt
MOG Validation Query for routes, Agent Notify of parking conflict or reservations plan validity
Air Planner
Fig. 6. Maximum on Ground (MOG) Validation Agent state diagram
The following list describes the action performed by each functional block of the agent: • Information Retrieval Task—queries the Air Plan Database for all routes and parking reservations that may impact the current mission. • Post-Processing Task—generates reservations and impacting reservations from the retrieved data based on the current mission. • Evaluation Task—analyzes all retrieved reservations to determine if the current mission is still valid. This will be false if any conflicts have arisen, and it will be “maybe” if insufficient information is available to make a decision. • Reaction Tasks: — True Action Task—the agent notifies the operator that the mission is still valid.
— False Action Task—the agent notifies the operator that one or more conflicts have occurred. These conflicts are fully described to operators so that they may replan. — Maybe Action Task—no action is performed. • Continuation Check Task—the agent always continues until the mission begins execution. However, it waits for some predetermined amount of time before restarting pattern execution. In a complex application, it can also be useful to apply design patterns on a macro level, assigning one or more agents to act as information retrieval agents, one or more to do post-processing, etc. We implemented this strategy in the full application for the AMC because it was necessary to monitor for three factors that could impact a mission: MOG conflicts, airfield closures, and prior permission requests (PPRs). We have already discussed MOG conflicts. Prior permission requests are requests for permission to land that must be filed at an airfield some period of time before the mission date. This varies for different airfields, and there is typically a small window of time within which PPRs may be submitted. Previously, a human was responsible for ensuring that PPRs were filed at the correct time for missions; however, this process did not work well because many missions must be managed over time, and a single mission might have multiple PPRs to file at different times. Here we discuss the AMC application as we actually implemented it, with information retrieval tasks delegated to child agents by a Master Validation Agent. One Master Validation Agent exists for each mission. It creates three child agents to perform information retrieval and filtering for each basic potential conflict being monitored. • MOG Reservation Agent—retrieves data on parking Reservations from the Air Plan Database, sending back to the Master Validation Agent only those parking Reservations for a date near that of the monitored mission. • Notice To Airmen (NOTAM) Check Agent—retrieves NOTAM from the Internet: These notices alert pilots or air planners that airfields or runways are closed for maintenance, air shows, etc. • Prior Permission Request Check Agent—checks to see if a PPR must be submitted in the short term. The MOG Reservation Agent, NOTAM Check Agent, and PPR Check Agent send data they retrieve back to the Master Validation Agent in the form of events. In its Evaluation state, the Master Validation Agent determines if any of the data brought back poses a conflict to the mission. If insufficient data is available (perhaps because a child agent has not yet sent back data), then no decision is made and the Master Validation Agent waits for more data. If a conflict is found, the air planner is sent an alert and given a full description of the conflict. In this instance, application of the Sentinel pattern at a higher level ensures that less redundant data will be carried over heavily loaded networks. Rather than having three separate agents carry the mission plan to evaluate against MOG conflicts, NOTAM, and PPR constraints, very lightweight mobile agents transmit that information back to a stationary Master Validation Agent for analysis. Also,
delegating information retrieval to child agents—instead of having a single Master Validation Agent gather all data—allows information retrieval to occur in parallel. The patterned agents we developed for the AMC autonomously searched for mission conflicts from the time a mission was entered into the Air Plan Database until it was executed. This persistent monitoring enables very early notification of potential problems without direct operator involvement. Air planners are able to address problems before they become a crisis. Applicability. In general, we feel that agent itinerary patterns based on finite state itineraries can benefit any work, wherein: • Complex agent behavior is needed, but the development timeline is short. • Multiple agents perform similar behaviors. • The same basic tasks are reused in different configurations for different agents. The Sentinel behavior patterns, with their support for monitoring, can benefit situations, wherein: • An agent must persistently collect data, analyze it, and determine if some condition has occurred. • A collection of agents must collaborate to collect data, analyze it, and determine whether a condition has occurred.
6 Conclusions In our experience, itinerary patterns used in conjunction with libraries of frequently used tasks allow rapid development of complicated agents and agent interactions. Patterns based on a finite state-style itinerary are a simple but powerful means of programming highly complex, reactive, even adaptive agents. The Sentinel and Simplified Sentinel patterns have significantly reduced our time to develop agent applications and it has significantly increased our ability to reuse code. These benefits have allowed our developers to focus on creating new functionality—in the form of reusable tasks and services—rather than on laboring over complicated itinerary creation. In the future, we plan to extend our task library and our repertoire of patterns to increase the efficiency of our application development. We believe that additional agent behavior patterns will emerge as we continue to implement distributed, agentbased applications. We also plan to formalize our infrastructure for pattern support into a Pattern Development Layer, which will sit above core EMAA and the itinerary development layer. This will clarify and speed the process of developing new agent behavior patterns. Moreover, we are working to further simplify the EMAA-based, agent application configuration and execution process. We have already developed reusable user interfaces for controlling agents and monitoring their progress through an itinerary and across networks. In addition, we will develop interfaces that allow an end user to easily configure security levels, available services, and communications details in the EMAA Dock. We are considering the addition of agent creation
interfaces for nonprogrammers [13]; due to the high level of modularity introduced by itinerary patterns and task toolboxes, we feel that this would be a natural addition. In general, our work in the area of agent patterns is aimed at reducing development time and simplifying the creation and use of agents without losing power and flexibility.
7 Acknowledgements We wish to thank Julius Etzl, Martin Hofmann, and Ken Whitebread for their help in identifying the need across many agent applications for a sentinel monitoring capability. We are also grateful to Russ Lentini, who thoroughly tested and validated our pattern layer and to Virginia Garrison, who developed reusable graphical user interfaces for controlling and monitoring EMAA agents.
References 1. Lentini, R., Rao, G., Thies, J.: Agent Itineraries: An Alternative Data Structure For Agent Systems. Dr. Dobbs Journal of Software Tools, 24(5). (1999) 60-70. 2. Lentini, R., Rao, G., Thies, J., and Kay, J.: EMAA: An Extendable Mobile Agent Architecture, Fifteenth National Conference on Artificial Intelligence (AAAI ’98), Madison, Wisconsin, Technical Report WS-98-10: Software Tools For Developing Agents, ISBN 157735-063-4 (1997) 3. Chacón, D.: Cooperating Agents for Specific Tasks. Lockheed Martin Advanced Technology Laboratories. Contract Deliverable (1999) 4. Ad Astra Engineering. Jumping Beans White Paper. http://www.JumpingBeans.com/, October 1998. Sunnyvale, California. 5. Falchuk, B.and Karmouch, A.: Visual Modeling for Agent-Based Applications. Computer, Vol. 31. Institute of Electrical and Electronic Engineers. (1998) 31-38. 6. Kendall, E., Malkoun, M.: The Layered Agent Patterns. Pattern Languages of Programs, Illinois, (1996) 7. Daniels, J.: Integrating a Spoken Language System with Agents for Operational Information Access. Submitted to AAAI (1999) 8. Aridor, Y., and Lange, D.: Agent Design Patterns: Elements of Agent Applications Design. Proceedings of the Second International Conference on Autonomous Agents (Agents '98). (1998)108-115 9. Tahara, Y., Irie, Y., Ohsuga, A., Honiden, S.: Agent System Development Method Based on Agent Patterns. Proceedings of the 21s t International Conference on Software Engineering. ACM Press, (1999) 10. McCormick, J., Chacón, D., Lentini, R., McGrath, S., and Stoneking, C.: A Distributed Event Messaging System for Mobile Agent Communication. In submission. (1999) 11. Mitsubishi Electric Information Technology Center. Concordia. http://www.meitca.com/ HSL/Projects/Concordia/Welcome.html 12. ObjectSpace. Voyager ORB. http://www.objectspace.com/products/prodVoyager.asp 13. Clements, P., Papaioannou, T., and Edwards, J.: Aglets: Enabling the Virtual Enterprise. (1997)
