Reliable Framework Architecture for Multi-Agents Systems Interaction Eugénio Oliveira
Guilherme Pereira
Cláudio Gomes
LIACC, Faculty of Engineering, University of Porto, Portugal
[email protected]
[email protected]
[email protected]
receivers). On the process, it is quite easy for one of these messages to be lost. Moreover, adding new agents to the system will require the system to be changed because the pair wise communication has to be reviewed. There is also a tendency to centralize the system by designing agents who are register agents, brokers or facilitators of all kinds. This poor design puts a great load on these agents, creating bottlenecks that degrade system performance and affect its reliability as well. In many systems there is not a clear distinction between data and knowledge, especially business-rule knowledge, therefore, for instance, changing an agent’s marketing strategy becomes a difficult task, as it requires reprogramming the agent. In a highly dynamic environment, as we find in today enterprises, this would mean that the response to a competitor’s new aggressive marketing campaign would require considerable time, hence loosing customers. As far as security is concerned, existing multi-agent systems either provide no security at all or boast complex security schemes on a per agent basis. This means that either every communication between two agents is prone to be listened by a third party or agents must use a secure channel and encrypted messages to communicate with one another. Unfortunately, this last solution is quite heavy and unnecessarily complex. Agent reputation assessment is also discussed as a key feature of e-commerce multi-agent systems. We hence suggest a different system’s architecture that is able to deal with these problems in a reliable way.
Abstract Multi-agents systems (MAS) are developed using a variety of architectures nowadays. These range from peer-to-peer to blackboard-like communication, from insecure distributed systems to complex security schemes, from those MAS using database to those using knowledge based information systems. Although a system’s architecture often has to be designed according to its specific application domain, we present in this paper a reliable MAS architecture suitable for most purposes in the e-business general domain. This framework combines the distributed nature of a multi-agent system with the security and encryption facilities provided by web servers, separating data from knowledge and encouraging the use of distributed data structures in a concurrent, scalable, transaction-safe and remote-event generator common area. Some notes on agent reputation assessment are also included. We also endow our agents with learning capabilities for two different aspects. Reinforcement Learning for bidding and a kind of Concept Learning technique for user profile. The main features of the proposed architecture are agent communication through a common area, remote user access through servlets with authentication and encryption, data/knowledge separation and agent reputation assessment using customer satisfaction values. The functionality of all the proposed multi-agent system features is illustrated through an agent-based Travel Services application (MASTAS).
2. Requirements for this class of application It is our belief that the architecture here presented is best suited to e-commerce applications. These applications include multiple agents having its own utility function, hence performing according to their best interest. However, in order to settle an agreement with other agents, these agents must relax their proposals making concessions to accommodate other agents’ interests as well. An agreement will possibly be reached if
1. Introduction Multi-agent systems present a variety of different architectures according to their application domain. Some of these architectures make use of message passing where agents communicate with each other in pairs (an agent has to send the same message as many times as its intended
1
access the system via browser, contacting the HTTP server who should dynamically launch a customer agent to represent this customer inside the system. Agents belonging to two different local area networks could also communicate by sending a HTTP request to the other agent’s machine HTTP server. This server could then dynamically create a temporary agent to write that message to that LAN’s common area.
all agents’ interests are taken into account up to a certain extent.
3. Proposed system architecture The proposed system architecture comprises one or more local area networks, every machine of which having a HTTP server for remote communication. Each local area network is considered to be secure and only remote or inter-LAN communications require special attention.
3.3. Data/knowledge separation We suggest that each agent should have its own database as well as its own knowledge base. It is essential that business rules lay inside the latter. Hence, if a company’s marketing strategy (for example, discount policies) is to change, the only operation needed is updating the agent’s knowledge base (updating the discount rule). The remain of the agent’s code stays unchanged. The knowledge base should also contain the utility function of the agent and its reasoning capabilities. Thus, altering an agent’s behavior is straightforward.
3.1. Inter-agent communication The communication between agents, belonging to a given local area network, is accomplished through a common area. This area should allow agents to exchange messages as well as other objects. It should allow multiple readers at the same time but should prevent multiple writers from writing the same object at any one time. A support for transactions and notification for remote events should be major assets. Operations executed under the same transaction either all succeed or all fail and in the latter case the previous state of the common area is restored. Thus, transactions ensure that the state of the common area is consistent after a set of operations if and only if that state was consistent before those operations took place. The ability to generate remote events notifying agents of state changes in the common area should play a central role in message delivery. The idea is that agents register their interest in some particular types of objects and they are notified whenever these objects are written to the common area. This feature makes it possible for an agent to ignore all messages that it is not interested in. Another important feature is assigning a duration time to every object written to the common area. This allows old messages to be garbage collected after a certain time if the system fails to do so. In a distributed environment, partial failure is prone to occur and hence there must be a way of ensuring that the state of the common area remains up to date.
3.4. Agent reputation assessment Agent reputation is a major asset in e-commerce applications. In this kind of applications there are customer agents trying to buy a certain good or service and supplier agents trying to sell it. Customer agents should use information about supplier agents’ reputation in order to choose which supplier agent to buy from. Our proposal is that this reputation associated to supplier agents be defined in terms of a subset of the cumulative function of customer agent satisfaction in every commercial transaction with that supplier agent. The agent satisfaction is defined as the difference between expected and real price of a certain good or service provided by a certain supplier agent in a commercial transaction. Hence, positive values for satisfaction mean that the customer agent was satisfied and negative values mean the opposite. When a transaction takes place, that customer agent writes to the common area its satisfaction value. Some of these values may be published in order to other customer agents take notice in future transactions. These values are the highest ones in modulo, meaning the best and worst values for that customer agent satisfaction.
3.2. Remote communication Communication from outside a local area network is done via HTTP. There should be a HTTP server on each machine belonging to a given local area network. Browsers or other software can make requests but the connection should be encrypted and should use a secure channel. The HTTP server should be able to create certain agents dynamically in order to represent external entities inside the local area network. For instance, in a multiagent system designed for e-commerce, customers should
3.5. Main features The main features of the proposed architecture are agent communication through a common area, remote user access through servlets with authentication and
2
encryption, data/knowledge separation and agent reputation assessment using customer satisfaction values. The common area should allow object storage and encourage the use of distributed data structures. It should also support transactions, remote event notification and garbage collection of old messages. These features will improve application reliability and scalability because agents are loosely coupled and they don’t need to care about other agent addresses, even less if the other agents received the message. Therefore, messages can outlive the agent who sent them. Another advantage of the common area is that one can exchange not only messages but also executable code inside objects. The use of servlets for user interaction provides security to the system from the outside. Hence agents should only encrypt messages sent outside their LAN. This feature will provide security to the system without the overhead of encryption of all messages sent. The separation between data and knowledge improves code maintenance because changing business rules will be kept aside from the main application code. Hence modifying a marketing strategy will be easily done. Agent reputation is very important to e-commerce applications as it constitutes perhaps the only way for a customer to trust a supplier, encouraging customeroriented marketing strategies.
4.2. Tools Software used to develop this system includes Java Development Toolkit (JDK), Jini and JavaSpaces from Sun Microsystems, Apache Jakarta Tomcat from The Apache Software Foundation, MySQL from MySQL AB Company and Amzi! Prolog from Amzi!. Agents were developed in Jini using JavaSpaces as a common area for communication. MySQL was used as database server and Amzi! was used for knowledge base support. Tomcat serves as HTTP server providing user interface with the multi-agent system through Java servlets. Tomcat/Jini integration was accomplished through writing a serialized object to a file. This file was identified by the IP address and the date and time of the request. Jini customer agents were then dynamically created by running another Java virtual machine as a sub process of Tomcat. Jini agents write back the reply to the same file in the serialized object reply fields. Tomcat servlet detects when the sub process terminates and retrieves the information from that file. Jini/MySQL integration is accomplished by JDBC using mm.mysql type 4 full Java JDBC driver by Mark Matthews. Jini/Amzi! connection is done by logic server.
4. Describing an application 4.1. General description A multi-agent system for travel services (MASTAS) was developed using these features. The system allows flight and hotel booking from local or remote customers. There are three main types of agents: customer, company and hotel agents who represent the respective entities. There is also a facilitator, a press agent and a softbot. The facilitator is responsible for storing travel packages bundled by company and hotel agents. The press agent is an agent who will publish customer complaints as well as good impressions. It serves as a mean to establish company and hotel agent reputation. The softbot is capable of extracting information from real world web pages updating company and hotel agent information systems. Each company and hotel agent has access to a database and to a knowledge base. The database is used to provide information about flight timetables, hotel seasons and bookings. The knowledge base contains the agent’s utility function as well as the discount policies and negotiation tactics.
Fig. 1. Multi-agent system architecture
4.3. MASTAS at work 4.3.1. User interface. Customers access one of the LAN’s machine’s servlet through a browser. They then fill a form about their preferences and constraints. These information items are the preferred date of departure, number of days of travel and price as well as the minimum and maximum possible values. Another page is shown requiring the introduction of the desired itinerary. Fields requested are origin, destination, number of adults, flight class, hotel localization, category, number of rooms, preferred, minimum and maximum number of days in that location and next destination.
3
customer agent. Several rounds can be necessary to choose the winner. When the utility of a certain bid approaches a predefined threshold, that bid is the winning one. One important aspect to bear in mind is that constraints must be satisfied. A customer agent will not accept a bid that will extend the travel beyond the maximum number of days, for example. So when the agent writes a request to the JavaSpace, it must have used a scheduling algorithm before in order to check those constraints: the previous scheduled destinations will influence the current constraints in this negotiation round. Eventually, the agent must renegotiate previous rounds because current constraints become impossible to satisfy.
Next, user priorities are read. Priorities are values that range from 1 to 7 (1 is highest priority) that are assigned to 7 different fields. These fields are the date of departure, total number of days of travel, number of days in each location, preferred price, flight class, hotel localization and hotel category. By assigning priorities, the relative importance of the listed fields is ranked. When the user confirms this information, the system executes a Jini customer agent and passes these values to it through a serialized object that is written to a file. The agent retrieves this information from the file and dynamically creates and compiles an Amzi! Prolog file containing the utility function of the agent plus a scheduling algorithm to schedule the travel. The utility function is established using the priorities given by the user. Upon a customer request, the facilitator can reply with a set of predefined travel packages. These packages are designed through bundling company and hotel agent products, as we will see later. If there is no suitable predefined travel package or if the user declines those offers, a negotiation process between the customer agent and company and hotel agents is to be carried out. The servlets that provide the user interface run on the HTTP server. User authentication is enforced and the communication is encrypted. The multi-agent system is protected from the outside in this fashion.
4.3.3. Reinforcement Learning. The learning method agents are using to improve their own bids is the QLearning algorithm. Each pair comment/action is a state. The comment is the reply to a company or hotel bid and the action is one of: maintain, increase or decrease that attribute’s value. The ranking included in the comment is used to calculate the Q value. Nevertheless, there are predefined constraints as to certain values: for instance, hotel prices for a given category and season must range from a minimum to a maximum value, depending on that country’s law. This information is inside the Prolog file of that hotel agent and is checked before bidding. Having scheduled the winning itinerary, that information is written to the serialized object’s file and retrieved by the servlet. The user can then accept that option or impose further constraints to any of the attributes. These constraints are of two different types: not acceptable or undesirable. If a given attribute is marked as not acceptable, the agent must provide an alternative itinerary with another value for that attribute. If it is marked as undesirable, the agent checks to see if another solution is possible but it can be that none is and that attribute’s value is preserved.
4.3.2. Negotiation protocol. As far as negotiation is concerned, the Q-Negotiation algorithm [1] is used. This algorithm allows for multi-criteria negotiation and includes Reinforcement Learning (Q-learning) capabilities. Initially, the customer agent writes requests to the JavaSpace with the information fed by the user. Each request is relative to a different destination in the user’s itinerary. Company and hotel agents can make bids to each one of these requests. However, they only reply when it is appropriate (for instance, a hotel in Paris will not answer a request for hotel in Lisbon). This behavior is implemented through remote events. Agents register their interest in certain types of objects (with certain values for certain fields) and whenever one of these objects is written to a JavaSpace, all registered agents will be notified (thus, a hotel in Paris will only register interest in requests for hotels in Paris and will not be notified of requests for hotels in other locations). Bids are evaluated according to the customer agent’s utility function. Customer agent replies to bids with a comment that indicates the relative ranking of that bid amongst all the others and with attribute by attribute information about the qualitative evaluation of that attribute compared to the best bid so far. Comparing current bids with the best bid so far is more realistic and convincing than comparing with the ideal and, besides, it helps to keep privileged information private to the
4.3.4. The facilitator. The facilitator’s role is to store travel packages that are created by company/hotel product bundling. The need for bundling is established by company agents. These agents analyze historical data and retrieve information about the most important destinations for that company. For instance, these agents could learn from their booking history that one of the most important destinations was Barcelona. This information is extracted using the ABC classification algorithm. This method divides products (in this case trips to specific destinations) into three categories according to its economic relevance or value, which is defined as the product of price by demand. Class A products have the greatest value whereas class C products have the least. Thresholds are defined in percentage of total value. Typical thresholds are: class A – 15 to 20% of total products represent 75 to 80% of total
4
value; class B – 20 to 25% of total products represent 10 to 15% of total value; class C – 60 to 65% of total products represent 5 to 10% of total value. This information is registered in the company’s knowledge base.
4.3.7. The softbot. The softbot is responsible for extracting real world information from web pages. This involves a definite clause grammar (DCG) for both syntactic and semantic analysis. Values are retrieved from the page and the context keywords are checked against those existing in the softbot knowledge base. The value itself is checked against its acceptable threshold. The softbot will write SQL scripts for database upload based on accepted values.
4.3.5. Concept Learning. Once the most important destinations are found, the company agent tries to establish a customer profile for each one of these destinations. This is accomplished using a Concept Learning algorithm: the Candidate-Elimination algorithm [2]. This method succeeds in learning the target concept provided that training data is not noisy and that the unknown target concept can be expressed in the considered hypothesis space. For each target concept found, the company agent will choose the best hotel agents in that location through a negotiation process. The customer profile that had been learnt before, will provide information to suggest which hotel attributes are to be preferred. For instance, if passengers to Barcelona booked first class tickets, then only 4 or 5 star hotels are to be considered for product bundling. The company agent books two different planes for this travel package on convenient dates. The first plane heads for the chosen destination and the second plane heads home. The number of rooms offered by this hotel set must be at least half of the total capacity of each plane. The chosen hotels must have a common set of attributes like category, localization and services offered, for example. These attributes are the ones to be advertised in that travel package.
5. Related work We were inspired by The Personal Travel Assistant developed at British Telecom by Nwana et al [3] as well as the Travel Agent Competition [4]. The Personal Travel Assistant is an agent capable of travel booking and monitoring. This agent will establish an itinerary for the traveler based on information like timetables provided by institutional agents like bus, railway or air company agents. The travel is booked and the agent monitors it suggesting alternative itineraries in case of bad weather or strikes, for example. Travel Agent Competition is a competition where agents buy and sell travel products like plane tickets, hotel accommodation and event tickets (like exhibitions’) in a dynamic marketplace. This competition is a reference to multi-agents systems’ experimental work due to the complexity of this application field. Different approaches to this competition range from a constraint solving approach by authors from the Swedish Institute of Computer Science (SICS) [6] to a determinant factor approach for price prediction by authors from the Southampton University [7].
4.3.6. Agent reputation assessment. The press agent’s function is to establish company and hotel agent reputation. Upon a successful transaction, the customer agent writes its satisfaction value to the JavaSpace. The press agent retrieves a certain number of these values for publication. The chosen values are the greatest in modulo, so only best and worst impressions will be considered. This is in contrast with the work of Sierra [5] who suggests that every agent should be associated with a reputation value by asking third party agents about their impressions. The number of these chosen values is limited to the amount of space available in the press agent’s newspaper, which is an object with a limited number of other objects inside called news. The news includes the name of every company and hotel agent and the cumulative satisfaction value for all transactions taken into account. Customer agents read the newspaper and take this information into account in their decision-making. The degree of influence that the newspaper has in the customer agents’ decision making is defined for each agent in its knowledge base.
6. Conclusions The research here described was motivated by the need to design a reliable architecture for multi-agent systems interaction. The key ideas here presented include agent communication through a common area, remote communication via servlets with user authentication, data/knowledge separation and agent reputation assessment by outputting customer satisfaction values. A multi-agent system’s application encompassing these features is described. It includes a press agent for publishing reputation on a newspaper and a softbot for real world information extraction based on syntactic and semantic analysis of web pages. Two kinds of learning techniques are used: Reinforcement Learning for negotiation bidding and Concept Learning for establishing user profile. This work is related to two major research experiments in the area of travel services: The Personal Travel Assistant from British Telecom and the Trading Agent
5
Competition. This is a complex application field where testing multi-agent systems’ theory is undoubtedly a challenging experience.
7. References 1.
2.
3. 4. 5. 6.
7.
Rocha, A. P. and Oliveira, E. C.; Electronic Institutions as a Framework for Agents’ Negotiation and Mutual Commitment; in the proceedings of the 10th Portuguese Conference on Artificial Intelligence, EPIA (2001). Mitchell T. M.; Machine Learning; McGraw-Hill International Editions, Computer Science Series (1997). Ndumu, D. T., Collis J. C. and Nwana H. S.; Towards Desktop Personal Travel Agents (1998). Trading Agent Competition, http://auction2.eecs.umich.edu/ Sierra, C.; Oral presentation in the 10th Portuguese Conference on Artificial Intelligence, EPIA (2001). Aurell, E., Boman, M., Carlsson, M., Eriksson, J., Finne, N., Janson, S., Kreuger, P. and Rasmusson, L., SICS, Swedish Institute of Computer Science AB; A constraint programming agent for automated trading (2002). He, M., Jennings, N. R.; Southampton TAC: Designing a Successful Trading Agent.
6
