module insuring agents compliance. This technique is ... information provided by the definition of system rules. ... closer, in the system's law definition. The goal ...
Towards an Integration Test Architecture for Open MAS Luiz Fernando Rodrigues, Gustavo Robichez de Carvalho, Rodrigo de Barros Paes, Carlos José Pereira de Lucena Departamento de Informática Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio) {lfr,guga,rbp,lucena}@inf.puc-rio.br
Abstract. In open multi-agent systems, agents must accord social conventions in order to maintain the predictable integration. Usually, these social conventions are hard coded, leading to unsuitable systems. A solution to hard coded conventions is separate the system’s social convention into a separate module insuring agents compliance. This technique is called law enforcement. In this paper, we propose an approach for integration tests in open multiagent systems. This approach supports the creation of test cases based on the information provided by the definition of system rules. We propose to use XMLaw, a language for the specification of open multi-agent systems.
1. Introduction Advances on agent technologies rely on the development of processes, models, mechanisms and tools to build high quality systems. Multi-agent systems are systems composed of many autonomous agents that interact to achieve their goals [Wooldridge, 2002]. Despite their autonomy, agents must follow certain social conventions in order to maintain the global order. Traditional approaches to implement these social conventions, or laws, hard code them into the code of each agent. However in open multi-agent systems, where agents may enter and leave the environment at their will and there is little control over the behavior of all agents [Agha, 1997], this approach becomes unsuitable. A law enforcement approach [Minsky and Ungureanu, 2000; Paes et al., 2005; Esteva, 2003; Dignum 2002], separates the system’s rules from the raw source code, making them explicit so that the development and maintainability may become more efficient. Even though the current approaches deal at some extent with the monitoring and enforcement of such laws, it is not clear how it is possible to perform integration tests over such systems, i.e., how we can write test cases and receive reports about the results of them. In a systemic way, in open multi-agent systems, integration tests become very important once the behavior of the involved agents may be unpredictable. We can define an open system as a composition of several subsystems where rules must be obeyed. In open multi-agent systems, we shall consider that a subsystem must be treated as an agent, meaning that it will provide several properties that describe agents such as: autonomy, social ability and reasoning [Odell et al., 2000]. In this kind of system, we consider that agents must communicate with each other. They can also cooperate or compete while interacting. It is responsibility of the environment to provide support to agents interaction. However it is difficult to control the interaction
60
among agents since we do not have access to agents’ source code, for this reason, a law enforcement approach should separate the rules from the agents’ source code, making them explicit. Several works have been based upon law enforcement, each one presents its own way of representing and evaluating rules: [Martín et al., 1999]; [Esteva, 2003]; [Dignum, 2002]; [Kollingbaum and Norman, 2003]; [Minsky and Ungureanu, 2000] and [Paes et al., 2005]. Even with the benefits brought from the law enforcement, developers are still concerned with agent’s integration in the system. It is natural to ask how it is possible to know if an agent is acting according to the rules of the system, in other words, how can we find out if an agent is acting accordingly to the system’s laws? Or even more, is it possible to detect where and when an agent disrespect them? We believe that the answer is really closer, in the system’s law definition. The goal of this paper is to provide an overview of a test case based approach and an architecture to generate test reports by the analysis of the rule definitions representation. In order to achieve that goal, XMLaw will be considered as the adopted enforcement mechanism and language; in account of its event-drive model and XML description language. The next Sections are organized as follow. Section 2 introduces XMLaw, followed by an analysis of test case concerning points from XML law description file in Section 3. In Section 4, it is presented the proposed approach to an integration test architecture. Finally Section 5 discusses some conclusions and future work.
2.XMLaw XMLaw is a law enforcement language and environment supplying a structural and dynamic model. The former describes how all the law elements are related while the last shows how an event-driven architecture is implemented. The starting point of the structural model is the LawOrganization element. It represents the laws of a multi-agent organization and is composed by Scenes, Norms and Clocks elements, as shown in Figure 1. Scenes can be seem as an analogy to theater scenes, where actors play roles following scripts, but in this case we consider agents playing roles according to a protocol. Scene describes Protocols, Norms and Clocks elements which interact among them. Formally, a Protocol is a non-deterministic automaton composed of States and Transitions, where each transition is activated by a messagePattern. Norms prescribe how the agents are allowed or must behave and what their rights are. In other words, they prescribe the permissions, obligations and prohibitions of agents. As a consequence of the relationship between norms and transitions, it is possible to specify which norms must be active for firing a transition. For instance, a transition should only fire if the sender agent has a specific permission. As you can see, Norms can be defined into the Scene context and the LawOrganization context, being valid only in it definition context.
61
The Clock element describes time constraint. We argue that laws must be time sensitive. In other words, although an agent may be able to perform an action a1 at time t1, it might not be able to perform the same action at time t2 (t1 < t2).
Figure 1 - Law Conceptual Model
The dynamic model describes the behavior of the elements that compose a law specification. Events are the basis of the communication among law elements, that is, law elements dynamically relate with other elements through event notifications. Basically, we can understand the dynamic of the elements as a chain of causes and consequences, where an event can activate a law element; this law element could generate other events and so on. For example, the arrival of a message activates a transition, which activates a clock, which generates a clock tick, and the clock tick activates a norm. Taking advantage of this event-driven architecture, a mediator takes charge of the message enforcement. It subscribes itself to an observer structure and listen all events occurred in the environment, blocking then “out law” messages. The system implementation is made in a centralized server, where the mediator agent remains enforcing all messages, this approach is simpler and does not care about distributed synchronization, however it forces all agents to communicate to the same server.
3. Test Cases in XMLaw As described in the previous section, XMLaw is based on a mediator agent. The system laws are described in a XML file and our hypothesis is to reuse this language to observe system’s behavior. In Figure 2, the laws of a ping-pong example are shown. This example describes an organization with one scene and a very simple interaction definitions between the agent ping and the agent pong. The scene starts at state s0 and waits for a message m1 from ping, as described in the transition t0. After transmitting the ping message, the scene waits for the pong response, so the observation process can change to state s2 as describing transition t2, after that the scene finishes. Regarding the source code, some system’s assertives can be established, assertives are expressed as observation points. Observation points are an extensible feature that permits us to configure the test infrastructure to specific needs. The first one is the time to live observation point, it indicates that a scene can not stay alive for more than a stipulated time, if so, a treatment is activated, for instance, someone is notified.
62
Assertives like cardinality, agent role, agent type, entrance state, execution trace, execution context and evolution logic (all marked in the example) have the same behavior of the time to live assertive. If the constrains they express are not satisfied, it should activate all associated treatments. Cardinality observation point asserts for the allowed participant agents number in a scene, while agent role and agent type observation points are asserting the kind of agent and its roles. Entrance state observation point asserts for agents trying to join the scene in not allowed states. Execution trace evolution logic asserts for the transitions among the states fired by messages.
Figure 2 - Ping-Pong example observation points
There are more details in this example. For instance, we can analyze an execution trace by the messages definition and the evolution logic, reporting where faults in the message exchanging are. Since all messages are described in the XML file, it is possible to indicate the coverage of a test suite, indicating all the states and transitions activated during the execution. Another detail to be considered is stub agents generation. It is not clear yet how to construct a stub with some system logic built-in, but much of the information needed is described in the XML file, however this point is out of the scope of this paper.
4. Integration Test Architecture Even with an easier law manipulation and a control over agent’s behavior, the law enforcement approach does not solve problems of agent integration in open systems. Leading us to figure out how we should identify failures and errors in open system execution, how to execute tests cases and how to report non-compliant executions. As described before, the XMLaw environment provides a mediator agent, which enforces all messages exchanged by agents. All agents must use the same mediator, where non-compliant messages are always blocked. Also the environment is built with
63
an event-driven architecture, meaning that messages can fire events and observers can subscribe to them. Concerning testing and the XMLaw environment, we propose an integration test architecture providing means of executing test cases, block-unblock operation mode, allowing the execution of entire test cases, coverage evaluation of test cases and the generation of testing reports. The system will take advantage of the event-driven model used in XMLaw, all events generated in the system can be observed and be reported to a log database, Figure 3 illustrates the architecture.
Figure 3 - Integration Test Architecture
Actually the integration test architecture is an extension of the XMLaw environment. Since the environment uses a mediator agent which subscribes itself to all events in order to observe all messages, our approach will provide an agent, called Report Agent, which will also subscribes itself to all events (1), doing so, all generated events will be stored into a log data base (2). It is also necessary to implement a mechanism allowing non-compliant messages cross the system (3), if so, will be possible to execute test suits entirely. Another detail to consider is the creation of a test case agent, where all the test cases will be executed from. Since the report agent observes all messages in the system and analyses the system’s law, it can be responsible for indication the coverage of a test case. At the end of the execution of several test cases, the data-base contains a log of all events fired during the whole execution and also a trace of all messages exchanged. At this point the most important requirement is how to analyze the data. Since there are lots of confusing loggings lines in the data-base. Because of that, this integration testing approach provides an engine template that, from the log data, generates reports (4) conforming defined in templates (T1 and T2).
64
Even with the integration test architecture defined, the implementation process has not been started yet. Another technique, based in log instrumentation, had been drawn up but we have chosen this one presented. However, we believe there is still some fine tunings to be performed before the start of its implementation.
5. Conclusions We believe that integration tests are a real need when concerning open multi agent systems, since the chaotic behavior may happen and debugging methods are wasteful ways of solving problems, especially in distributed asynchronous systems. Our approach does not cover all points needed for a complete integration test, since it depends of report interpretation and analysis. However, the points addressed take advantages of already know system information, which were wasted before. Applying these information to testing aims to improve the system in the way that you do not have to redefine what your system must do in law clauses, and what it expect to do in test cases, since the information is already declared. Of course there will always be the test case definition, but in this case the test will be concerned exactly with the agent behavior, i.e., it will test how compliant the agent’s messages will be with the system’s law without worrying with unacceptable cases. The reporting analysis is also a good tool since it concerns several views of the system’s execution. Sometimes, reading log files is a unwise and wasteful task. The idea of focusing on the data to be observed permits us to go exactly to the problem cause, avoiding the tedious tasks as the analysis of log and trace files. Also, the template definition can be used in future analysis, so the user does not have to redefine a new report view when re-inspecting a log data base. Other future work is to well define the exact relationship between the observation points previously describe. By this understanding, we intend to be able to customize the integration test engine to specific needs.
6.References Dignum, F. (2002). Abstract norms and electronic institutions. In Proceedings of InternationalWorkshop on Regulated Agent-Based Social Systems: Theories and Applications (RASTA’02), at AAMAS, Bologna, Italy. Esteva, M. (2003). Electronic Institutions: from specification to development. PhD thesis,Institut d’Investigació en Intel.ligència Artificial, Catalonia - Spain. Kollingbaum, M. J. and Norman, T. J. (2003). Noa - a normative agent architecture. InProceedings of the Eighteenth International Joint Conference on Artificial Intelligence,pages 1465–1466. Morgan Kaufmann Publishers. Martín, F. J., Plaza, E., and Rodriguez-Aguilar, J. A. (1999). An infrastructure for agent-based systems: an interagent approach. International Journal of Intelligent Systems (IJIS), pages 217–240. Minsky, N. H. and Ungureanu, V. (2000). Law-governed interaction: a coordination and control mechanism for heterogeneous distributed systems. ACM Trans. Softw. Eng. Methodol., 9(3):273–305.
65
Odell, J., Kerr, D., Laamanen, H., Levine, D., Mack, G., Mattox, D., McCabe, F., McConnell, S., Raatikaine, K., Stout, K., and Thompson, C. (2000). Agent technology green paper. Technical report, Object Management Group. Paes, R., Carvalho, G., Lucena, C., Alencar, P., Almeida, H., and da Silva, V. T. (2005). Specifying laws in open multi-agent systems. In Agents, Norms and Institutions for Regulated Multiagent Systems - ANIREM, Utrecht, The Netherlands. Wooldridge, M; Weiss, G; Ciancarini, P. (Eds.) (2002) Agent-Oriented Software Engineering II, Second International Workshop, AOSE 2001, Montreal, Canada, May 29, 2001, Revised Papers and Invited Contributions, Vol. 2222 of Lecture Notes in Computer Science, Springer. Agha, G. A. (1997) Abstracting Interaction Patterns: A Programming Paradigm for Open Distributed Systems, In (Eds) E. Najm and J.-B. Stefani, Formal Methods for Open Object-based Distributed Systems IFIP Transactions, Chapman & Hall.
66
