How Third Trusted Parties Can Support Negotiations in Multi-Agent Systems for Virtual Enterprises Nick B. Szirbik Dept. of Information & Technology, TU Eindhoven, PB 513, 5600 MB, The Netherlands tel. +31-40-247 4379, fax. +31-40-243 2612 email:
[email protected] Abstract In a Virtual Enterprise (VE), there are three main functions for an IT infrastructure: support VE formation, track and monitor the execution of customer orders, and support the problem management (delays, re-configurations, re-routings). Formation and problem management implies complex interaction, and one of the crucial aspects of these two functions is the ability of the agents to perform negotiations in conflicting situations. The structure of the VE IT system can be designed to support negotiation. Our working system, called PROVE, enables negotiations by using a Third Trusted Party, that is providing mediator agents, and keeps historical information about previous agent based interaction in the VE. The data kept centrally was designed around the trust concept. Because trust is rather a vague concept, we identified some measures for capturing trust. The one investigated in this paper is the “tendency for exaggeration” during negotiations.
1. INTRODUCTION Regulations and the regulatory infrastructures on a free market are necessary besides other needs, to prevent the actors on the market to speculate during volatile periods and fast changing contexts. Changes can occur due to unexpected causes or because of deliberate intervention of law making bodies. Without change, a closed free-market system is reaching a state of equilibrium, but of course this is not a true possibility. In volatile periods, the speculative actors usually are worsening the situation, making the market more unstable, leading sometimes to market dislocation. The regulatory frameworks are complex systems that can be modelled at the highest level as a levelled map (as in figure 1). The first two levels are enacted in an organised and “formal” way. Also, these two have a large coverage (global or at state level) and their enforcement is done by specific institutions. Their change is difficult and it is performed with caution and intense deliberation. The two levels at the lower part of the map are only significant for specific groups with fuzzy boundaries (economic sectors, industries, alli-
ances, corporations, supply chains, etc.). These “soft regulations” are vague, they develop and change slowly in time, by becoming part of the actors’ behaviour. In most of the cases, it is difficult to capture and formalise them.. international law
} }
“hard” regulations
state laws and financial instruments a generic code of conduct (sub)-culture
“soft” regulations
Figure 1. The levels in the regulatory frameworks In this paper we show how the agent paradigm can be applied to capture some of the code of conduct and how the agent-oriented approach can help in the phase of conceptual design of the development of IT infrastructures for Virtual Enterprises. The paper is structured as: the following section is presenting the application domain, the third section is presenting the basic scenario for the intended negotiation protocol and how this one was chosen, section 4 is about the negotiation model, section 5 is showing a numerical example of a negotiation together with observations and arguments for future enhancement and the last section is concluding the paper. 2. THE APPLICATION DOMAIN In this paper, the application domain for the agentbased IT infrastructure is the Virtual Enterprise. Usually, agent systems are envisaged for very large markets, where the number of actors is large, the interaction patterns and chains cannot be predicted or they do not repeat, having more a probabilistic nature. The goal in such a market is to employ a regulatory infrastructure, which can enable that the main objective of welfare economics is satisfied, that is, the interaction of rational agents acting selfishly within an economy, results in an outcome that is Pareto efficient, i.e.,
no other solution exists that is better for all the actors [Kreps90]. Some of the approaches for symmetric (like B2B) agent based electronic commerce mechanisms are proposing as a policy regulator a neutral intermediary [Sarkar&al95]. The role of this intermediary is to arbitrate conflicts of interests. The comparative measure of the desirable property of the efficient global outcome is the “net global efficiency”: the sum of the individual gains of the actors, directly derived from the repeated interactions. The theoretical models for this kind of arbitrators between broker agents are inspired from game theory and micro-economics and are not applicable for systems where the number of economic actors is low and the interaction patterns are predictable and mostly repeatable. Another shortcoming of game theory inspired mechanisms is that the almost all (like the very classical Nash bargaining model [Nash50], and the Groves-Clarke tax [Clarke71]) assume that the agents will reveal their true preferences. In our approach, we accept that the agents can exaggerate their offers. There are some researchers following the same path, like Fankhauser and Tesch [Fankhauser&Tesch99] who propose a model in which the agents have weights associated with them, and the arbitrator is able to change these according to the lying behaviour, in a “prize or punish” scheme. But the main difference of our approach with classical arbitration is that the intermediary agents can refer to a centralised instance, as a Third Trusted Party, and can have historical information about the behaviour of the actors. In a Virtual Enterprise, short term goals are imposing a cooperative behaviour within a relatively closed society of self-interested actors. There are a multitude of interpretations about what a Virtual Enterprise is. Our definition is: “A VE is a group of enterprises, which has the capability of realising in a collaborative manner, items or services from a limited portfolio of products.“(see also [Kornelius99] for arguments for this definition). A basic assumption is that there is a great level of redundancy in terms of resources (materials and manufacturing and assembly capacities). This set of enterprises has unfortunately a very vague boundary. In order to define a crisp boundary, we consider only those enterprises that are linked together in a cooperative network, enabled by a specific communication framework. In the literature, this group is sometimes called a Network or an Enterprise Web [Fischer&al96]. In our previous work, we used the term Agent-Web. For simplicity, we will consider this network the Virtual Enterprise. When a VE has to execute a specific order or design a new product, only a small number of parties will participate in the process. We call this sub-group the Virtual Enterprise Cluster. The way the parties will co-
alesce into the group is opportunistic and based on the current context given by the distribution of available materials and capacities. When selecting partners, parties are also looking at previous performance indicators of the potential parties. Previous interaction is a good source for estimating the level of trust a party can have in others. But this is only a facet of the complex scenario of enacting a VE cluster (the rest are presented in figure 2).
domain culture and common language
relations, norms and codes of conduct
history of interaction and trust estimation
communication framework
Figure 2. Facets of interaction in a Virtual Enterprise The experience gained in Virtual Enterprising [Goranson99] shows that the speed of the selection of the parties in a VE cluster is strongly influenced by the existing regulations, laws and norms. A even stronger influence has the culture and the non-formal code. Also, the formation of the cluster is enabled in a decisive way by the communication framework. The current technology is providing a highly performant one. There are three views in this framework (figure 3): the hardware infrastructure view, the language view and the organisational view. Today, the hardware infrastructure view is dominated by the Internet and mobile communication means. From the language point of view, we have now EDI and more recently XML (with various dialects, such as ebXML). The organisational
hardware view
languages and meta-languages
communication framework
organisational view (role, relations)
Figure 3. The views related to the communication framework view of the existing frameworks shows metaphors known as B2B (business to business), B2C (business to customer), e-markets, e-hubs, inter-enterprise APS (advanced planning system), and inter-organisational workflows, to mention only a few. There is an emergence of new metaphors for the organisational view of
the communication (and collaboration) infrastructure. A relatively new trend is that the parties participating in Virtual Enterprises enact a third trusted party which is acting as an IT service provider. Two of the examples are from the aerospace industry and the airline sector (as MyAircraft.com, Aeroxchange.com). The tendency, from an organisational view, is to change the ad-hoc-archy of the party interaction in cluster formation to a “hyperarchy”, where the IT support infrastructure is not covering only the communication and coordination aspect, but also keeps track of the repetitive interactions, monitors the performance and measures trust. More than that, such an infrastructure imposes a kind of implicit regulatory mechanism, by adhering to mutually accepted norms. Also, to align the communication and coordination at the semantic level, common shared ontologies are needed, reflecting the specific culture of each VE. A powerful paradigm used especially in conceptual modelling and system analysis, but also in design and implementation of IT infrastructures for VEs is the agent paradigm. Our long term research attempts to identify opportunities how to apply the agent paradigm in all the above mentioned facets. In this paper, emphasis is on investigating some ideas related to the interaction history facet. We consider that this aspect has been neglected by current research. There are three main directions of research in the “software agents for business” area: - languages, meta-languages (e.g FIPA, XML and derivative mark-up languages) - ontological engineering (e-catalogues, XML based ontologies) - structural design (agent-oriented design methods, frameworks and tools) These are covering partially the communication framework (especially the organisational aspects and the communication language view) and to a lesser extent the semantic facet. The regulatory facet and the interaction history facet are not covered. We will show here, one way to use history data about previous interaction, and the problems posed by this approach.
with delays, re-routings, cluster changes and dynamic re-configurations of products during the manufacture/ assembly phase. A particularity of the PROVE system is that the agents are not owned by the participating enterprises in the cluster, but by a third trusted party which is an agent provider. An interesting conclusion was that any disturbance in the initial execution plans, had to be solved on the fly via negotiation. In a single enterprise case, mere replanning is sufficient, but in a loosely coupled and volatile multi-enterprise environment, any change leads to negotiations. The basic role of the agents in the PROVE system was to track and monitor the order execution across the VE. When the agents had to support negotiation, we discovered that the most appropriate role for them in this context (due to the fact that they are not owned by the parties) was to use them as mediators. In [Szirbik&al2000] it is shown how the mediator agents can solve a delay in a VE order execution. In this case we investigated some of the theoretical aspects of pair mediated price negotiation. An interesting outcome is that the agents evolved from simple software monitoring components to implicit regulatory mechanisms for inter-enterprise interactions. Hence, we decided to investigate how pair mediated negotiation (PMN) can be used in the VE cluster formation phase, when there is access to interaction history data. We first considered four price-based scenarios for how to enact a VE cluster. These scenarios are derived from various types of auctions. The auction mechanisms have inspired many agent-oriented e-market infrastructures. These mechanisms are reasonably simple to formalise, implement and control. Of course, in any form of automated bargaining there are loopholes and some clever designers of agents can take advantage for their own profit. This means that the regulatory mechanism must be flexible enough to be easily changed (or only slightly adjusted) to prevent any predatory tactics of the agents. “English”
3. THE SCENARIO A VE cluster has the following life-cycle: - Cluster formation - Work-effort execution in a collaborative manner (towards “win-win” situations) - Cluster dissolution In previous papers [Aerts&al00a], [Aerts&&al00b], it was presented how the monitoring and tracking of the execution segment can be supported via an agentbased infrastructure. In our department at TU Eindhoven, we built a prototype (called PROVE) to study and experiment various agent behaviours in the process of order execution in a VE. The agents had to cope
one buyer
“Dutch” the case investigated
one seller
many sellers
many buyers
Figure 4. The four basic auction types The nature of these basic scenarios is captured by the diagram in figure 4. In the classic “English” auction, there are many potential buyers and one seller and the price is rising during the auction process until all the bidders except one renounce to bid. In the
“Dutch” auction, the price is going down, and the first bidder is the winner. The other two scenarios, called sometimes “backward” auctions, imply one buyer and many sellers. This situation is similar to one in which the customer in a bazar is rounded by sellers who have no fixed prices. When the price is going down, it means that the buyer is silent and waits for a seller to reach his level of an affordable or convenient price. When the price is going up, it means that the sellers are silent, waiting for the buyer to reach their expected price. The first who is willing to sell at the buyer’s last announced price, is the winner. A combination of these two mechanism can be used, when the buyer and the sellers are repeatedly announcing new increased and decreased prices. All these mechanisms are prone to cheating. When groups of buyers and sellers know well each other, they can work together, in secret, to artificially inflate or deflate prices. In this way, they break the generally accepted code of conduct of auctions. It is very difficult for the auctioneers to prevent this kind of behaviour. We surveyed these mechanisms and many of their variations from an agent perspective, especially the case when mediator agents are used. In a VE the production is customer driven, and that means that the buyers have the initiative in the process of VE cluster formation. In our study, the model of a VE party is abstracted from the details involving manufacturing and distribution, leaving only two compartments, Sales and Purchases. In the VE cluster formation, the purchasing departments are looking for sellers, and the sellers are waiting to be contacted via their sales department. Each department can engage in price bargaining via a software agent that can be deployed using the specific IT infrastructure. We will have in this case buyer agents and seller agents. We are making two assumptions: A. When a party wants to buy something (product, service, outsourced capacity), there are many parties that can sell (the redundancy assumption) - or, for each buyer agent instance, there will be more than one seller agent. B. There is a regulatory body, similar to the Software Agent Common Provider (SACP), presented in [Szirbik&al00] which is creating the mediator agents.
Our protocol is derived from an auction mechanism which is in our opinion, one of the most secure. It exists in slightly different form in the business world and has the following description: if a buyer wants a product (usually a high value one - with a negotiable price), and (s)he has the capability to send human purchasing agents to many potential sellers, these agents will negotiate each one with a specific seller. The agents are in permanent contact with each other, but the sellers are considered not to be. The agent who is succeeding to negotiate the lowest price will pick his associate seller as the winner of the auction. This mechanism is probably the most costly in a humandriven setting, because it implies lots of resources and mobility. The main advantage is that if the identity of the buyer cannot be established easily by the sellers, it is difficult for them to use the colluding bidders’ practice (as it can happen in Vickrey auctions, see [Sandholm95]). By using software agents (which can be mobile agents, as in PROVE) and Internet based communication, the cost of applying this mechanism is low and can be used for low value items. Automating the process is quite straightforward, although many subtle details (presented later in the paper) show that the theoretical foundations of such a mechanism must still be investigated. When using agents, a party which has a purchase order to be executed, creates a software agent that is dispatched to the site of the third trusted party (TTP). Mobility is not essential, the agent can remain at the site of the party and only send messages to the TTP site. Here, for each buyer agent created in the VE, a mediator agent is created. The TTP has the capability to detect which parties are able to deliver according to the demand of the buyer. All these potential selling parties are contacted by the mediator agent, and they dispatch a seller agent to the TTP site. A main constraint is that the seller agents and the buyer agent cannot be in direct contact, they can interact only through the mediator. This is due to the fact that the announced prices of the seller and buyer are different from their walkaway prices (called “reservation prices” [Raiffa82]). There is more chance to reach an agreement if a mediator is trying to “convince” the buyer and seller agents to give in, little by little.
All the agents have the same common shared ontology and they can communicate with each other. The buyer and seller agents’ behaviours are created by the participating parties. These agents are self-interested rational agents, that is, they are trying to maximise their profit. Their rational behaviour (the way they bargain) is kept private, but they have to obey the protocols of the mediating agents, who are considered neutral.
4. THE NEGOTIATION MODEL As presented in [Szirbik2000] the simplest way to enact a mediated negotiation is the “simultaneous disclosure of reservation prices” technique. The buyer and the seller know exactly which are their reservation prices (the lowest price the seller will accept and the highest price the buyer will be willing to offer). If the mediator is able to convince the parties to give him the exact value of these prices, he can average these prices
ation tendency and this current value will update the centrally maintained value, which is accessible only for the mediator agents. We decided to measure this as a normalised value tend(party) in the interval [0,1]. This value is somehow similar to the weight assigned to a party in [Fankhauser&Tesch99], in the way that it will affect future negotiation, exaggerating (lying) agents being “punished”. The tendency can be measured in two ways. After each mediated negotiation (considering that it was successful), the mediator can compute the difference between the first announced price of the party and the agreed price.
(the “symmetric solution” [Harsanyi77]), and then each party will “win” half of the difference between the walkaway prices (like in figure 5). Of course, if the seller’s price is higher than the buyer’s, there is no room for negotiation. The problem with this schema is that the parties are not willing to disclose their walkaway prices even to a mediator who is considered neutral. The first announced prices are usually exaggerated, but this tendency is considered acceptable in any business culture. There cannot exist mediated average price
buyer’s price
firstprice – finalprice tend ( p ) n + 1 = -----------------------------------------------------------firstprice
price
seller’s price
buyer’s win
The difference is considered exaggeration and it is recorded using the overall average formula:
seller’s “win”
Figure 5. The ideal situation (when announced prices are the reservation prices)
n ⋅ tend ( p ) n + tend ( p ) n + 1 tend ( p ) n + 1 = ---------------------------------------------------------------n+1
the concept of real negotiation if the announced prices are the reservation ones. A negative outcome due to exaggeration can appear when the relation between the announced prices are in a different relation with the reservation prices, like in figure 6, where the reservation prices give room for negotiation, but the announced prices make it impossible for the mediation through symmetrical averaging. In a human driven negotiation, when the parties know that the other one is exaggerating, a “negotiating dance” [Raiffa82] is taking place. Step by step, each party is giving in a little bit, until the announced prices are meeting somewhere between the reservation prices. A mediator can lead and balance this “dance” in a neutral way, if he knows how much the parties are exaggerating. We consider that one of the measurable values for trust is the tendency for exaggeration of the parties in a centrally IT supported VE (by a SACP, for example). We propose that the third trusted party maintain and update a value for this tendency for each party. Each mediated interaction will yield a result for the exagger-
The sliding average can be used also, considering the only the interactions that took place in recent time (the window of time is to be selected by the mediator). The second way to measure the exaggeration tendency is before the negotiation (the estimation of the mediator). This estimation needs a fixed point of reference, and this can be a neutrally estimated price. How this can be obtained? Consider that many of the parties who can a deliver a product type cannot deliver it at the date (or date range) asked in a specific purchase order of a buyer (this is in accordance with the redundancy assumption). For this particular purchase order execution, these parties can be considered neutral. The mediator agent can ask these parties for an opinion about a “honest price”. Also, the mediator (without disclosing the identity of the buyer and seller) can ask only the neutral seller who has the best value (the lowest) for the exaggeration tendency in the “trust-database”. On the other hand, if the mediator has more values, he can infer (by simple average or by “maximum density distribution selection” for example) a honest price. Consider the following situation: the buyer announced to the mediator the price bp, the seller the
buyer’s exaggeration
enough room for agreement announced price of the seller
reservation price of the seller announced price of the buyer
reservation price of the buyer seller’s exaggeration
Figure 6. How negotiation can fail, even if there is a chance to succeed
price
price sp, and the mediator “discovered” the price honest price hp (as in figure 7 - there are other possibilities, where hp is to the left of bp or hp is to right of sp, but we are not investigating these). For simplicity, the mediator will consider that only one of the parties is exaggerating. For example if (sp -hp)(hp - bp), then the negotiator will consider that the seller is exaggerating. Following the same demonstration as for (3), the seller estimated exaggeration tendency will be: sp – 2hp + bp tend ( seller ) = ---------------------------------2 ⋅ hp – bp
and in this case, the tendency of the buyer will be considered 0. honest price
buyer’s announced price bp
hp
seller’s announced price price sp
Figure 7. The starting situation for the mediator In principle, the negotiation can be finished by the mediator agent, after one step by announcing a final price. The simplest resolution is to select hp as the final price. In reality, the mediator must consider that both parties are exaggerating, and the honest price is not necessarily the absolute reference. In order to compute the final price, the mediator can consider that exaggeration he measured is only half (the rest is considered to be done by the other party). The following formula will result in both cases (seller exaggerates, or buyers exaggerates): bp + sp + 2 ⋅ hp mp = -------------------------------------4
The formula above considers that the honest price is not the best price, but only an indication for where this price should approximately be. If this meditated price is within the boundaries of the reservation prices for both parties, an agreement is reached. If not, the nego-
tiation fails. The failure is given by the starting conditions (the reservation prices) and not by the negotiating technique. how much the buyer exaggerates bp
equal
bbp
hp
price sp
Figure 8. The buyer is exaggerating more with respect to the “honest price“ The mediator agent can take into account also the previous tendencies of the parties, tend(s)n and tend(b)n, and a corrected mediated price is a more fair one: ( 1 + tend ( b ) n ) ⋅ bp + ( 1 – tend ( s ) n ) ⋅ sp + 2 ⋅ hp mp = ---------------------------------------------------------------------------------------------------------------------4 – tend ( s ) n + tend ( b )n
Even if the prices are within the boundaries of the initial reservation prices, there is a great risk that the agents will not agree with the decision taken by the negotiator for the mp value. In automated negotiation (and in human negotiations as well), it is hard to asses by the parties that mp is a “good” price. Rational agents expect a “win” over the walkaway price. Internally, they set an expected price. In game theory, this price is called local maximum utility solution for that particular agent. Moreover, bargaining means to give in step by step, not to accept a single-step decision. We are applying two methods to increase the chance for agreement: i. There are more sellers in the process (each negotiation being carried by a separate mediator able to communicate with the other, or a concurrent thread in a single mediator). ii. The negotiation process is made in more steps. The negotiating agents will have in a multi-step approach more insight about the behaviour of the opposite party and they can change dynamically the parameters for their internal computation of the announced prices. Moreover, the buyer agent has a perspective over all the sellers, and he can compare their behaviour. Consider that the maximum number of steps is t. Each step i is started when the buyer and the seller announce both a price (bpi, spi). The mediator, who previously “discovered” the “honest price” is computing the tendencies for exaggeration using the formulas presented above. Also, he is computing his internal target price for the next step, mpi, but this value will not be disclosed to the parties. Although this price can be shown only as a proposal of the mediator, our observation was that during the steps, this target price is remaining relatively fixed,
what the buyer is announced
the target for the
what the seller is announced
mediator
bpi
mpi
spi
price i
msp
buyer has to give in
i
mbp
seller has to give in
Figure 9. The mediator creates the impression of “giving in” to the partiesgiving the impression to the parties that the other party (actually the mediator) is not “giving in” at all. Worse, the target price is “wobbling” around a fixed value (the first offer), showing a very strange behaviour to the buyer and the seller. i
i
( 1 + tend ( b ) n ) ⋅ bp + ( 1 – tend ( s ) n ) ⋅ sp + 2 ⋅ hp i mp = ------------------------------------------------------------------------------------------------------------------------4 – tend ( s ) n + tend ( b ) n
From a psychological point of view, it could be better to show to the buyer that the price offered by the negotiator is decreasing step by step, and the price offered to the seller is increasing with each step. In this way, the mediator offers different but converging prices to the seller and the buyer. This can be achieved by using the “margin” concept. The margin value, Mi, is computed each step, using: i
i
i sp – bp M = --------------------f( i)
where f is a linear function of i.
f
nounced prices and the offered prices must be like in figure 9. We used various values for s and t (e.g. s=5 and t=10) and a slope of (π/4) for the function f(i), and the offered prices evolved in the desired way for all the negotiation scenarios we tested. These values can also be selected dynamically by the negotiator, in order to preserve the above mentioned constraints. After the mediator has sent the offered prices to the parties (together with a remainder of how many steps are left), these have to respond in a limited time. The new announced prices have to converge, otherwise the negotiation is terminated. Also, if a party fails to respond in time, the negotiation is terminated without success. In this way, a seller or the buyer can terminate one of the negotiations that are taking place in parallel. If the allowed number of steps, t, is reached, and the announced prices did not converge yet, the negotiation is terminated. The single case when the negotiation succeeds, is when the announced prices are in the relation spimspi and spi
