Formal Verification of a Simple Automated Negotiation Protocol

Formal Verification of a Simple Automated Negotiation Protocol George Dimitoglou Dept. of Computer Science Hood College Frederick, MD 21701 [email protected]

Okan Duzyol Dept. of Computer Science The George Washington University Washington, DC 20052

Lawrence N. Owusu Dept. of Computer Science The George Washington University Washington, DC 20052

Abstract Negotiation is a complex human activity and even in its most basic form consists of three elements: communication, strategy and bid evaluation. The elements are exactly the same in automated negotiation. In this paper we present four possible approaches for bid evaluation and strategy but we mainly focus on the communication aspect of the negotiation. Communication lends itself well to be specified and modeled as a protocol, so we introduce a basic automated negotiation protocol which we simulate and verify using PROMELA and the SPIN model checker. Keywords —Protocols, formal methods, automated negotiation, SPIN

I. INTRODUCTION

N

and bargaining are complex human activities. They take place between at least two parties or groups trying to reach a consensus or a deal. The process and final outcome are affected by objective factors (i.e. proposed terms, time constraints) and other non-objective factors such as the cultural backgrounds of the participants, their perception of risk and their natural instincts. Automated negotiation faces the same challenges as negotiation between humans while posing a few new questions: Can machines negotiate effectively among each other and with humans? Are the outcomes of automated negotiation more efficient than human negotiations? What happens when the human factor is removed from the bargaining table? Can the outcomes be predicted? To answer any of these questions it is necessary to understand the mechanics of bargaining and to model the techniques used during negotiation sessions. Translating the negotiation process into computing terms, there must be a communication mechanism and infrastructure between the bargaining parties along with EGOTIATION

an established set of rules (a communication protocol) to carry out the negotiation. For this analysis, the focus is on the negotiation protocol, assuming the existence and treatment of the required infrastructure as a “black box”. In negotiation at least two parties (agents) participate. Adhering to the definition or “rational agents” [19][20] each party attempts to maximize their welfare. In general, for each participant the optimal outcome is to obtain a deal that satisfies (or exceeds) their maximum terms. However, it is unrealistic to expect that every participant will achieve their maximum goals, so each party must make concessions. An integral part of negotiation is understanding each agents’ value of preferences. By doing so agents know whether an offer is satisfying their preferences or not. Typically, a basic negotiation session includes the announcement of the asking terms and a series of counter-offers and partial concessions that may, or may not, lead to a deal. The process of participating in a negotiation is not an atomic event. There are a number of steps that each party takes during a negotiation. Evaluating the cost and value of the good or service at hand, issuing a first bid,

responding to a counter-offer can all be used to enhance the negotiation strategy by balking or sending multiple offers to project the notion of real interest. The use of strategy implies a number of other conditions beyond the scope of the basic entity’s rationality, and assumes additional capabilities such as coordination, group agreement or even goal-driven decision making to accommodate the notion of “social welfare”. All of these are very complex behaviors but there is a common action that plays a major role in determining their manifestation: the activity of bid evaluation. Bid evaluation is perhaps the most interesting and intricate part of the negotiation. It is the activity that forces the negotiating party to distill all of the information, evaluate the offer, its timing and compute its overall value within the context of the negotiation. This raises a plethora of interesting questions. How does the party evaluate the offer? What are the criteria? Are criteria pre-defined or dynamically established during each negotiation? If the latter, how are they evaluated?

II. RELATED WORK Currently the area of research closest to automated negotiation is in the field of intelligent agents. The work in this area is based primarily on techniques drawn from the convergence of Artificial Intelligence (AI) and Distributed Computing. From AI, intelligent agents use machine learning techniques, knowledge discovery and inference engines to enable them to make decisions [4][14][19][20]. From Distributed Computing, the agents borrow mobility and communication to operate in a distributed environment. A very common variant of the intelligent agent paradigm works on the World Wide Web (WWW) as a purchasing broker (shopping bot). It allows an agent to find the best deal (i.e. based on price) for a certain product on behalf of a user [5]. This activity is often referred to as negotiation, but in reality it is not. It is better described as a survey and rank operation rather than actual bargaining. We are interested in studying an interactive negotiation process where two entities (buyer-seller) try to reach an agreement by issuing offers and counter-offers. Unfortunately, almost forty years of theoretical research has still left many open areas when dealing with a general computational theory of bargaining and

negotiation [5]. There is a variety of reasons why this has happened, but it is mainly due to the existence of subjective and qualitative factors that are hard to represent mathematically. If instinct, intuition, or risk aversion were easily quantified, a set of rules could be derived to model bargaining and even predict the negotiation outcome.

A. Game Theoretic Approach Game theory provides an extensive set of mathematical tools to assist in the understanding and analysis of decision-making between interacting players. In a typical game, decision-making players have their own goals and try to determine their next move by predicting the other player’s decisions [1][6][17]. By taking into account the reasoning of other decision makers, the notion of strategy becomes a necessary factor, forcing future steps to be controlled by the players rather than pure chance. When applying game theory constructs in negotiations, the negotiating parties have the ability to consider every alternative ahead of time and decide what to do. Overall, the models of game theory are highly abstract and flexible enough to be adopted in a variety of negotiating situations (i.e. cooperative, noncooperative, two-person or group negotiations) [16]. In addition, by treating negotiations as games, we have the ability to mathematically evaluate the activities in terms of equilibrium points, stable strategies, and Pareto optimal solutions.

B. Consumer Theory/Utility Approach In Economics, a lot of work is done to understand consumer choice. Consumer theory attempts to describe the consumption behavior of a representative consumer who derives satisfaction from the consumption of goods or services [6]. This notion of “satisfaction” is also described as utility. Determining the appropriate utility function for the consumer can be non-trivial. But once discovered, finding consumer preferences is a matter of applying the function under the given conditions [13]. The theory is based on two assumptions: (a) the consumer is rational, hence all decisions being made aim towards the maximization of utility (satisfaction) and (b) the consumer is consistent, meaning that no contradictory decisions can be made [18]. With consumer theory in mind, any negotiation session can be executed by examining each participant’s

preferences against the given alternatives. Both players negotiate in order to improve, or at least maintain, their welfare. Each party evaluates the negotiation terms as the other party submits them. The terms are then evaluated through a utility function, ranked, and finally compared against the player’s objectives. The decision made will reflect the preference that maximizes the player’s satisfaction.

Using the same methodology, a population of strategies (sequences of decisions) is created, each of which represents an acceptable (deal-making) solution. The decisions represented by these strategies are rated, and based on these ratings, the algorithm will create a new population of strategies. The best strategies will be given more chance of being involved in this new generation and the process will repeat itself.

Consumer theory provides the tools to mathematically evaluate preferences assuming that the negotiation terms are quantifiable [18]. Approaching the negotiation process under the scope of maximizing each player’s utility fits well in an automated setting. Computers are “consistent” and can be programmed to be “rational”.

Genetic algorithms are flexible enough that even special features of genetic programming can be applied to negotiation sessions [7]. For example, combining two strategies can simulate genetic crossover and mutation can be simulated by randomly changing a small section of a small percentage of the strategies.

C. Regression Analysis Approach Regression analysis is a statistical technique used to find relationships between variables for the purpose of predicting future values. This type of analysis could be used as a supplement to compare the Game Theory and Consumer Theory approaches. Once an adequate number of test cases (negotiation sessions) are available, regression analysis can be used to formulate better decisions (i.e. counter-offers) by analyzing the historical trends of previous negotiations. The only disadvantages of this method are its strict dependence on the availability of historical data and the fact that the process may become computationally intensive.

D. Genetic Programming Approach Genetic algorithms model natural evolution in order to solve problems. Automated negotiation is assumed to be an interactive process with all activities driven by dynamic decision making. Instead of treating a negotiation session as a step-by-step interaction between participants, it may be beneficial to treat each participant’s series of decisions as a single strategy. With genetic programming, optimal strategies may be created given a certain series of decisions. In genetic programming a random population of strings is created, each of which represents a solution to the problem. The individuals represented by these strings are rated, and based on these ratings, the algorithm will create a new population. The best strings will be given more chance of being involved in this new generation [1][2].

Arguably, any one of these approaches may provide a complete answer to understanding how automated negotiation should work. Intelligent mobile agent technology has been argued as the most promising and most flexible infrastructure paradigm to carry out any of these approaches to facilitate automated negotiation. However, the questions raised so far are targeting the negotiation process as a mechanism rather than the required infrastructure paradigm. The introduction of agent technology in this examination only complicates the task at hand and introduces a number of agentspecific idiosyncrasies and red-herrings that plague this paradigm such as communication, mobility, security, authorization and ontology. By isolating the scope of the research to the negotiation process, the problem domain becomes more specific: How can we model the decision processes (mathematically or otherwise) to allow computers to negotiate? Can proven strategies be re-used? How can the necessary exchange of information be facilitated between the bargaining entities? The work should be focused in understanding the mechanics of negotiation and developing systems that can perform the activity. There are certain objective criteria such as stability, efficiency, simplicity, and symmetry [21]. If those criteria are met, an automated negotiation solution can be deemed successful.

III. A PROTOCOL-BASED APPROACH To better understand automated negotiation we develop an Automated Negotiation Protocol (ANP) as a generic mechanism. ANP is general enough that it can represent abstract items as long as value functions can be defined between cost and quantity for an item. It can

be used to facilitate the negotiation process between n entities with an intention to reach an agreement about the quantity and cost of a specific item/good by means of “offer/counter-offer”. The ANP-service-user is described as the protocol user that: (a) Invokes ANP by triggering any negotiation operation, interaction or activity such as a request for a bid and intend to bargain. (b) Performs a negotiation operation or interaction invoked by a peer ANP-service-user. (c) Computes some type of a value function that rationalizes and bounds the meaning of the negotiation within an equation between the cost and quantity of an item/good. (d) Makes use of the protocol even if encapsulated in an application.

(a) Communication is flawless and no packets are dropped or messages missed due to hardware errors. However, timeouts can occur at any time. The timeouts are necessary to allow either one of the participating entities to exercise balking or enforcing a temporal constraint on the negotiation. (b) The negotiation interaction is based on just two parameters: cost and quantity. Both parameters are quantifiable and domain-agnostic. The entities (n1, n2) enter the negotiation session with the following operational profiles: i. One entity (ni) possesses the quantity (q) and requires payment to relinquish it. This entity operates under a maximization function f where: f = max(f) when the value of the cost is the highest possible and quantity (q) is the lowest possible. ii. The other entity (nj) attempts to consume or, utilize the quantity. The entity operates under a maximization function z where: z = max(z) when the value of cost is the lowest possible and quantity is the highest possible. (c) Negotiated quantities (q) are not discrete. (d) Both entities can continuously make new or counter-offers.

Figure 1: Messaging interaction diagram

B. Vocabulary

We assume an environment with n-interacting entities (where n=2). The interaction is based on an exchange of information (messages) initiated by one or both of the entities. A protocol of interactions defines the rules of exchanges between the entities seeking an agreement over a specific subject. The subject can be anything: the cost of a good, the value for a service etc.

The protocol is defined using a bounded vocabulary. Limiting the vocabulary affords the entities to have a relatively “light” infrastructure to interpret and understand messages. To that extend, the vocabulary is intentionally terse to constrain any semantic overloading.

The entries n1 and n2 enter in negotiations in numerous ways. Minimally, they start by issuing messages for interaction either by asking if the other entity is willing to interact or by making an offer. The interactions continue with numerous other exchanges that facilitate the interactions.

{ Interact, Offer } Interact: May be used to initiate an interaction. The message payload consists of addressing information and one attribute. The attribute option is binary in the form of accept or reject (Table 1). It is assumed that the entities can interact directly without having to discover or establish each other.

A. Service Overview The protocol can be examined as a service with the following characteristics:

Message Interact Offer

Attributes Accept/Reject Cost, Quantity

Type boolean real

Table 1: ANP messages and their attributes

Offer: Used to make both an offer and counter-offer. There is no distinction between the two since the type of message payload is identical. The burden of tracking and identifying what is an initial offer from an entity, instead of a response (a counter-offer) to an earlier rejected bid is strictly placed on the entity. The message payload consists of a value pair (v, c) where v is the cost and q is the quantity. When an entity responds with an offer message and the values are different, then the offer is a counter-offer. If the reply is populated with identical values as the bid received from the other entity then it is an indication of offer acceptance.

C. Algorithm The entities n1 and n2 enter into negotiations in numerous ways. The sessions can start by issuing messages for interaction first by asking if the other entity is willing to interact or simply by making an offer. A possible interaction scenario (Figure 2) is as follows: (1) One party (seller) offers the other party (buyer) the quantity and the cost in the “offer” message. (2) The buyer can accept by sending back the same cost and quantity values as a response (using “offer” message) or make a counter-offer. (3) Both parties keep a “counter” semaphore to count the number of counter-offers to prevent the negotiation going indefinitely.

IV. PROTOCOL VALIDATION & VERIFICATION For our experimentation, we use the SPIN model checker [10][11][12] to verify our ANP protocols. The SPIN model checker is a mature product and includes a number of techniques to improve the efficiency of the model checking, such as partial-order reduction and state-compression. The software works with its own specification language called PROMELA (PROcess MEta-LAnguage) and verifies correctness claims specified as Linear Temporal Logic (LTL) formula. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

typedef msgtype { bool interact; int cost; int quantity; } int msg; mtype = {msg} chan ch1 = [0] of {mtype}; proctype Seller(int ainit,amin) { int x,b; int cnt=2; atomic { do :: ch1!ainit; if :: ch1?b; do :: (bch1!ainit-10; od ::b>amin->printf("Buyer offered more.Accepted.\n"); fi od; } } /* atomic */

Figure 3: Seller implementation code

Two service definitions are implemented, one for the seller (Figure 3) and one for the buyer (Figure 4). Both definitions are simple, handling the basic offer and messaging exchanges. 1 2 3 4 5 6 7 8 9 10 11

proctype Buyer(int cinit,cmax) { int c; int cnt=3; atomic { if ::ch1?c; do :: ( c >= cmax ) ->ch1!cinit+cnt; od :: cprintf("Seller offered less. Accepted.\n"); fi } } /*atomic*/

Figure 4: Buyer implementation code

Figure 2: Interaction scenario

The maximization functions max(f) and max(z) are also expressed very simply in lines 16 and 7 of Figures Figure 3 and Figure 4 respectively. The verification session begins with a triggering event that commences the negotiation. The two entities are represented and start similar to operating system processes, exchanging information.

The parameters provided to the Seller and Buyer (Figure 5) bootstrap the protocol by generating the initial cost and bid offers. 1 2 3

init { run Seller(70,40); run Buyer(30,50); }

Figure 5: ANP triggering event

From a computational perspective, the two entities go through identical states and transitions, depending on the nature of the message exchanged (Figure 7). Both entities begin from a similar initial state where no activity is taking place. The final state is reached after an offer is accepted, hence there is agreement and the negotiation has completed, or after an offer or counter-offer has been issued. Intuitively, a negotiation does not reach a final state after an offer is made but this transition enforces the notion that the two entities are independent and if one decides to withdraw from the negotiation, the other one is not left in a transition state, unable to join a new negotiation session.

The specific, detailed messages of the interaction are captured in a log similar to the one in Figure 8. Much of the description resembles the threading and message passing of operating system processes and threads. Starting :init: with pid 0 0: proc - (:root:) creates proc 0 (:init:) Starting Seller with pid 1 1: proc 0 (:init:) creates proc 1 (Seller) 1: proc 0 (:init:) line 67 "pan_in" (state 1) [(run Seller(70,40))] 2: proc 1 (Seller) line 33 "pan_in" (state 15) [ATOMIC] Starting Buyer with pid 2 3: proc 0 (:init:) creates proc 2 (Buyer) 3: proc 0 (:init:) line 67 "pan_in" (state 2) [(run Buyer(30,50))] 4: proc 1 (Seller) line 36 "pan_in" (state 1) [ch1!ainit] 4: proc 2 (Buyer) line 56 "pan_in" (state 1) [ch1?c] 4: proc 1 (Seller) line 36 "pan_in" (state -)[values: 1!70] 4: proc 2 (Buyer) line 56 "pan_in" (state -) [values: 1?70] 5: proc 2 (Buyer) line 57 "pan_in" (state 4) [((c>=cmax))] 6: proc 2 (Buyer) line 58 "pan_in" (state 3) [ch1!(cinit+cnt)] 6: proc 1 (Seller) line 38 "pan_in" (state 2) [ch1?b] 6: proc 2 (Buyer) line 58 "pan_in" (state -) [values: 1!33] 6: proc 1 (Seller) line 38 "pan_in" (state -)[values: 1?33] 7: proc 1 (Seller) line 39 "pan_in" (state 5) [((b=cmax))]timeout #processes: 3 8: proc 2 (Buyer) line 58 "pan_in" (state 3) 8: proc 1 (Seller) line 40 "pan_in" (state 4) 8: proc 0 (:init:) line 68 "pan_in" (state 3) 3 processes created

Figure 8: Log segment of the ANP simulation run

Figure 6: State Transition diagram

When the negotiation session begins, the communication channels and the exchanged messages are displayed in SPIN using a Message Sequence Chart (Figure 7). This graphical user interface (GUI) allows the visualization of message exchanges and value changes as the negotiation progresses.

Figure 7: Example of Message Sequence Chart for ANP simulation

Another interesting output from SPIN is the protocol’s verification phase results. There, additional elements of the protocol are examined such as the number of states and their transitions through a full statespace search, along with checks for cycles, invalid states or assertion violations. For ANP with two negotiating entities no extraneous conditions were observed during protocol verification.

V. CONCLUSION & FUTURE WORK Automated negotiation is a complex activity. In this paper we demonstrated that the mechanistic part of the activity such as the message exchanges and handling of the bids and counter-offers is not a difficult task. By employing a basic protocol it was shown that the basic interactions between two negotiating entities can be managed with relative confidence that a protocol verified for correctness contains no abnormal behavior such as deadlocks and infinite loops. Yet, the most interesting part lies with the evaluation method for bids which essentially determines how the negotiation progresses. For ANP, we used a simple maximization function which was sufficient to demonstrate the protocol. The

problem becomes significantly more complex when the number of participants increases along with the number of individual strategies and different goals in the negotiation session. Independent, self-interested entities negotiate differently than group-conscious entities. Similarly, even the number and type of negotiated attributes increases the complexity of bid evaluation since multiple attribute permutations may need to be computed to derive an optimal response. These complexities directly affect our protocol that has been verified for correctness for the restricted negotiation cases. Currently, ANP is defined for just two interacting entities. This is a limiting factor, given that it does not allow for legitimate coordination of group negotiations or provide protection for malicious collaborations (i.e. price fixing). Presumably, more work will be required to modify the protocol in order to accommodate a greater number of participating entities and address benevolent and malicious strategies that may manifest in the protocol as deadlocks, infinite loops or other erroneous conditions.

REFERENCES [1] Almgren, M. Communicating Agents Developed with Genetic Programming, Genetic Algorithms and Genetic Programming at Stanford 2000, pp. 25-32, Stanford Bookstore, June 2000. [2] Baber, V. Ananthanarayanan, R., Kummamuru, K. Evolutionary Algorithm Approach to Bilateral Negotiations, Genetic Programming, Proc. 5th European Conference, EuroGP 2002, LNCS, Vol. 2278, pp. 202211, Springer-Verlag, 3-5 April 2002. [3] Bartolini, C., Preist, C. A Framework for Automated Negotiation, Number HPL-2001-90, p. 25, Aug. 16 2001. [4] Bussmann, S., Müller, J. A Negotiation Framework for Cooperating Agents, 1992 Proc. of the Special Interest Group on Cooperating Knowledge-Based Systems, pp. 1-17, Dake Centre, University of Keele, 1993. [5] Chen, E., Vahidov R., Kersten, G.E. Agent-supported negotiations in the e-marketplace, Int. J. of Electronic Business, Vol. 3, pp. 28-49, Inderscience Publishers, March 2 2005. [6] Chipman, J. S. The Foundations of Utility. Econometrica. 1960 Apr; 28(2):193-224. [7] Choi, S.P.M., Liu, J., Chan, S-P, A genetic agent-based negotiation system, Computer Networks, 37(2), pp. 195204, 2001. [8] Hamzah, Zarul Girivendhen Dharmalingam. Automated Negotiation-Final Report. Surveys and Presentations in Info Systems Engineering., Imperial College of Science Technology and Medicine; 1996(SURPRISE 96; 4).

[9] Heitkoetter, J. Beasley D., Editor. Guide to Evolutionary Computation, Web Page. 1999 Nov 20. URL: ftp://ftp.cs.wayne.edu/pub/EC/FAQ/part1. [10] Holzmann. G.J. "The model checker Spin". IEEE Transactions on Software Engineering, Vol 23, No.5, May 1997. [11] Holzmann. G.J. Design and Validation of Computer Protocols. Prentice Hall, Englewood Cliffs, NJ 1991. [12] Holzmann. G.J. Tutorial: Design and Validation of Protocols. Computer Networks and ISDN Systems, 25(9):981, 1017, 1993. [13] Joshi, S., Micoreconomic Theory. Consumer Theory: Preferences. 2000. The George Washington University. Class Notes. [14] Kraus, S. Negotiation and Cooperation in Multi-Agent Environments. Artificial Intelligence, Vol. 94, Number 1-2, pp. 79-97, 1997. [15] Linhart, P. B., Radner, R., Satterthwaite, M.A.. Bargaining with Incomplete Information. San Diego: 1992. [16] Oborne, M. J., Rubinstein A.. A Course in Game Theory. MIT Press; 1994. [17] Raiffa, H. The Art and Science of Negotiation, Harvard University Press, Cambridge, USA, 1982. [18] Schmeidler, D. Subjective Probability and Expected Utility without Additivity. Econometrica. 1989 May; 57(3):571-587. [19] Simon, H. (1957). "A Behavioral Model of Rational Choice", in Models of Man. [20] Simon, H.A. (1990). A mechanism for social selection and successful altruism, Science 250 (4988): 1665-1668. [21] Zlotkin, G and Rosenchein J. S. Mechanisms for Automated Negotiation in State Oriented Domains. J. of Artificial Intelligence Research. 1996; 5:163-238.