A review of research literature on bilateral negotiations

Carnegie Mellon University

Research Showcase Robotics Institute

School of Computer Science

1-1-2003

A review of research literature on bilateral negotiations Cuihong Li Carnegie Mellon University

Joseph Giampapa Katia Sycara-Cyranski

Follow this and additional works at: http://repository.cmu.edu/robotics Recommended Citation Li, Cuihong; Giampapa, Joseph; and Sycara-Cyranski, Katia, "A review of research literature on bilateral negotiations" (2003). Robotics Institute. Paper 699. http://repository.cmu.edu/robotics/699

This Technical Report is brought to you for free and open access by the School of Computer Science at Research Showcase. It has been accepted for inclusion in Robotics Institute by an authorized administrator of Research Showcase. For more information, please contact [email protected].

A Review of Research Literature on Bilateral Negotiations Cuihong Li, Joseph Giampapa, and Katia Sycara CMU-RI-TR-03-41o 0»

A Review of Research Literature on Bilateral Negotiations A Carnegie Mellon University Robotics Institute Technical Report CMU-RI-TR-03-41 for the Navy Detailing Process, Cognitive Agents Technology Project Cuihong Li cuihong@andrew. emu. edu

Joseph Giampapa garof@cs. emu. edu

Katia Sycara [email protected] November 11, 2003

Abstract Automated bilateral negotiations are an important mechanism to realize efficient distributed matching in the Navy detailing system, and the presence of outside options is an outstanding feature of the negotiations. In this report we provide an extensive literature review on the research of bilateral negotiations in the fields of Economics and Artificial Intelligence. Three important dimensions are described to identify the negotiation environment and to build a research model. Preliminary considerations and suggestions are given in these dimensions on modelling the system. The reviewr suggests that negotiations with outside options is a new and important research problem, yet it can be addressed, based on the existing work.

1

Introduction

A bilateral bargaining (negotiation) situation is characterized by two agents - individuals, firms, governments, etc. - who have a common interest in cooperation, but wTho have conflicting interests concerning the particular way of doing so. Bilateral bargaining refers to the corresponding attempt to resolve a bargaining situation, i.e. to determine the particular form of cooperation and the corresponding payoffs for both [32] [29]. Bargaining is a prevalent form of interaction in human society. It is also an effective approach to resolve conflicts in a distributed manner, when a third party mediator is not available or trustable. Bilateral negotiation is a useful mechanism in the Navy detailing process. In the Navy detailing process, although most matches are decided through a centralized matching market mechanism, there are situations in wThich commands and sailors can negotiate with each other directly and decide the matches by themselves. Please refer to Section 2 for descriptions of these situations. The research wrork on bargaining has been conducted in the fields of economics, in particular game theory, and artificial intelligence (AI). The research in the economics community focuses on the outcome of a bargaining situation that satisfies some axioms1, or the strategy equilibrium2 of agents, based on some rigorous assumptions. Researchers in the field of AI contribute efforts to develop software agents wThich should be able to negotiate in an intelligent way on behalf of their users. The complex situations considered in an AI model prohibit the strategy equilibrium to be explicitly considered. Instead the AI research pays more attention to flexible and dynamic self-optimization of an agent by learning and adapting to the environment, which also includes other agents, and searching for a good decision heuristically. Research in economics and AI have different methodologies and concerns, yet their contributions complement each other. Insights and theoretical foundations developed in economics provide good heuristics for AI, and the AI approaches provide solutions to negotiations in realistic environments that usually cannot be solved by a game-theoretic model. The computationally feasible solutions provided by AI allow approximate implementations of theoretic results that are developed in a game-theoretic model and that may not be tractable to compute. To ground our research on bilateral negotiations, it is helpful to review the existing work and results in both the fields of economics and AI. The rest of the report is organized as follows: Before reviewing the literature we provide background knowledge, motivate our research work, and restate our research questions in Section 2. Section 3 reviews essential vocabulary, research work and results on x 2

See the definition of cooperative game theory in Section 3 See the definition of non-cooperative game theory in Section 3

bargaining theory in economics. The content is organized in three subjects: cooperative bargaining theory, non-cooperative bargaining theory, and bargaining with outside options. The research on bargaining in AI is reviewed in Section 4, focusing on different aspects of learning, heuristics, time issues and multi-attribute negotiations. Future work and discussions are provided in Section 5. Section 6 concludes.

2

Motivation

In the Navy detailing system a significant amount of matches are decided through a centralized matching market. In the centralized matching market sailors and commands submit their preference information (rankings, incentive bids, etc.), and the market replies with a matching that optimizes the overall quality across the matches. The matches are decided in a centralized way by the matching market, although sailors and commands can submit information to the market that influences the matching result. Howrever, there are some situations in wrhich commands and sailors can negotiate with each other and decide the matches by themselves. These situations include: • Special jobs and sailors: Some jobs with high priorities (such as the submarine jobs) and sailors with certain specialties do not get matched through centralized matching. Matching for these sailors and jobs can be performed through bilateral negotiations between a sailor and a command. • After centralized matching: The jobs and sailors that fail to be matched in the centralized matching market may negotiate directly with each other to reach a matching agreement. • Direct invitation: Some sailors may not apply for a job for which they are qualified. A command may directly contact such sailors that he desires and try to make a matching agreement. In these situations direct communication is established between a command and a sailor, and the communication may lead to an agreement between them without the mediation of a third party. We call this mechanism a matching and bargaining market (MBM), which can be regarded as a substitute for the centralized matching market. In a MBM a sailor may have multiple jobs that he qualifies for, and a command may find multiple sailors that qualify for his job. Instead of delegating the decision of matching to a centralized market, the sailors and commands negotiate one-by-one directly, and decide the matching and conditions of the matching. The matching is performed in

a distributed manner since it is a result of autonomous bilateral negotiations between commands and sailors, without the coordination of a central system. Compared to the centralized matching market, the MBM enjoys high flexibility at the expense of efficiency. The most critical problem for a bargainer during a negotiation is to decide how much to offer and how to respond (e.g., either accept or reject) to an offer. An effective negotiation decision solution is the key to automate the negotiation system supported by intelligent agents. To design an effective automated negotiation strategy, wre have to pay attention to he existence of outside options, an important feature of the bilateral negotiations in the MBM. A sailor may have more than one job that he can qualify for, and a job may find multiple sailors that are qualified for it. This feature differentiates the bilateral negotiations in MBM from common bargaining problems, in which the twro bargainers are monopolies. Outside options are an important factor to consider during the negotiation process because they can be used as a negotiation threat and have great impact on the negotiation strategy. With more promising outside options, a bargainer is in a more advantageous position of negotiation, and can employ a more aggressive negotiation policy. But the bargainer also takes certain risks at being aggressive, because a bargainer cannot be sure about the utility that she can achieve from the outside options. Therefore a bargainer has to tradeoff betwreen the expectation of a future deal and the probability of losing the current chance. The research we propose is to study the individual negotiation decisions in a matching and bargaining market by explicitly considering outside options. The task and output of this research work will be: To build an analytical model of the negotiation problem in a MBM, and to provide a negotiation decision model and solution that maximizes the expected utility of a bargainer. The questions that will be addressed by the solution are: How should a bargainer consider the outside options in negotiations? How much should a bargainer propose to the opponent? Given an offer from the opponent, should a bargainer accept it or reject it?

3

Bilateral bargaining theory in economics

Bargaining is a type of game. Bargaining theory is a part of game theory that studies bargaining games. We use the same taxonomy of game theory, shown in Figure 1, to organize our literature review of bargaining theory. Game theory can be divided into two branches: cooperative and non-cooperative game theory. Cooperative game theory abstracts away from specific rules of a game and is

Game theory

Cooperative game theory (complete information)

Non-cooperative game theory

complete information

incomplete information

one-sided incomplete information

two-sided incomplete information

Figure 1: Taxonomy of game theory mainly concerned with finding a solution given a set of possible outcomes. The solution is required to satisfy certain plausible properties, such as stability or fairness, which are called axioms. For example, an axiom can be that twro bargainers get the same share of the cake that they are negotiating for. Non-cooperative game theory, on the other hand, is concerned with specific games with a well-defined set of rules and game strategies, which are knowTn beforehand by the players. A bargaining strategy specifies the action of a player at each step given historical information3 of the negotiation. For example, the strategy of a buyer who bargains with a seller over the price can be to call the average of her last call and the seller's last call at each step, and accept an offer if it is better than the proposal she is about to submit. Non-cooperative game theory uses the notion of an equilibrium strategy to define rational behavior of players, which jointly decide the outcome of a game[32][34]. A strategy equilibrium is a profile of players' strategies so that no player could benefit by unilaterally deviating from her strategy in the profile, given that other players follow their strategies in the profile. For example, a strategy equilibrium for a buyer and seller who sequentially bargain over price without a time or budget limit is: the buyer initially calls the lowest allowable price and the seller calls the maximum allowable price. Both concede in each successive step at the minimum allowable pace, until their price calls match. No one can result in a better deal by conceding faster alone. An equilibrium facilitates the prediction of the players' behavior, and hence also the outcome of the game. Some 3

There is no historical information if the negotiation is a one-shot game, in which all players take an action simultaneously and then the game ends.

widely-used concepts of a strategy equilibrium include "dominant strategy*' equilibria, "Nash"7 equilibria and "subgame perfect" equilibria. A dominant strategy equilibrium consists of a dominant strategy of each player, which is optimal for a player irrespective of the other players' strategies. The strategies chosen by all players are said to be in Nash equilibrium if no player can benefit by unilaterally changing his strategy. A subgame perfect equilibrium refines the Nash equilibria in extensive-form games, i.e., games with a tree structure in which players act sequentially. In a subgame perfect equilibrium (SPE) the strategies for each subgame of the game tree constitute a Nash equilibrium. A game is with complete information if the preference information of a player is known to all other players, otherwise it is with incomplete information. For example, in salary negotiations between a command and a sailor, the negotiation is a complete information game if the sailor knows howT much the command is willing to pay for the job, and the command knows the minimum salary that the sailor is willing to accept; otherwise it is an incomplete information game if the maximum wTillingness-to-pay of the command or the minimum willingness-to-accept of the sailor is private information of the command and the sailor [14] [34]. If both sides have private information, it is called a two-sided incomplete information game, otherwise if only one side has private information, it is called a one-sided incomplete information game [1]. In multi-attribute negotiations the preference information also includes the relative importance of each attribute and howr they trade-off with each other. For an incomplete information game, "Bayes-Nash" equilibria is the equilibrium concept that is usually used. The strategies of players, which are associated with the private information of the players, compose a Bayes-Nash equilibrium if no player can get higher benefit on expectation by unilaterally changing his strategy [14] [34]. Cooperative games are all based on complete information, assuming that the input to the axiomatic solution is common knowledge or that players share true information with each other and the mediator, who regulates the solution given the information. In non-cooperative game theory players may withhold information or not be truthful with each other [34].

3.1

Cooperative bargaining theory

Cooperative bargaining theory is concerned with the question of what binding agreement two bargainers would reach in an unspecified negotiation process given the set of all possible agreements on the utility that each bargainer achieves. The path-breaking work of Nash [33] provides a unique solution that satisfies four properties, which are now called the "Nash axioms". The Nash axioms include: (1) The final outcome should not depend on how the players' utility scales are calibrated; (2) The agreed payoff pair

should always be individual rational4 and Pareto-efficient5] (3) The outcome should be independent of irrelevant alternatives; (4) In symmetric situations, both players get the same utility [16]. The Nash bargaining solution is characterized by the payoff pair s = (xi,x 2 ) which maximizes the so-called Nash product (xi — di)a(x2 — d2)^, where d\ and d2 are player l's and player 2's outcomes in case of a disagreement, a and /3 are the bargaining powers of player 1 and player 2. Some other solution concepts in cooperative bargaining theory include the Kalai-Smorodinsky bargaining solution [20] and weighted utilitarian (bargaining) solution [30] [32]. A cooperative bargaining model does not consider the negotiation process, but leaves the outcome to be determined by an axiom. We do not expect cooperative bargaining theory to be applicable to our research on bilateral negotiations in Navy detailing systems because the negotiation outcome is the result of specific interactions between commands and sailors.

3.2

Non-cooperative bargaining theory

Non-cooperative bargaining theory considers bargaining as a fully specified game. The game refers to the negotiation protocol that twro players follow during the bargaining process. A negotiation protocol is the set of rules that govern the interaction between negotiators. It covers the negotiation states (e.g. accepting proposals, negotiation closed), the events that cause negotiation states to change (e.g. no more bidders, bid accepted), and the valid actions of the participants in particular states (e.g. which messages can be sent by whom, to whom, at what stage) [19]. Some negotiation games or protocols are described below. These protocols have been applied mainly to evaluate negotiations over a single issue, such as the price of a good to be negotiated. They can be extended to the multi-attribute negotiations in which the attributes are negotiated either simultaneously or sequentially. In the following we first describe the negotiation protocols and present the subgame perfect equilibium (SPE) of the complete information bargaining game under these protocols. Then in the rest and most of the section we review7 the work in non-cooperative game theory with incomplete information. 3.2.1

Bargaining with complete information

We describe three non-cooperative bargaining games with complete information. These games can be used by two bargainers to divide a given bargaining "surplus" - the total profit resulting when the players reach an agreement. The total surplus resulting from 4 5

An agreement is individual rational if a player does not lose by participating in the game. An agreement is Pareto-efficient if no player can gain without causing a loss for the other player.

the match between a sailor and a job can be equated to the value of the sailor to the job, minus the minimum benefit required by the sailor. These games are different in the negotiation protocols, hence also the outcome at the subgame perfect equilibria. Such protocols can be candidates of the bargaining protocols used in the Navy detailing negotiation system. The subgame perfect equilibria of these games provide valuable references to design the autonomous negotiation strategies if corresponding protocols are used, and complete information is assumed. The ultimatum game One of the players proposes a split of the surplus and the other player has only two options: accept or refuse. In case of refusal, both players get nothing. In this game the proposer has overwhelming bargaining power. The subgame perfect equilibrium (SPE) specifies that the proposer asks for the entire surplus, and the responder accepts [32]. The alternating-offers game The alternating-offers game is a multi-stage extension of the ultimatum game. In this game player 1 starts by offering a fraction x of the surplus to player 2. If player 2 accepts player l's offer, he receives x and player 1 receives 1 — x of the surplus. Otherwise, the game moves to the second stage, and player 2 needs to make a counter offer. Player 1 can accept, or reject and make another proposal in the next stage. This process is repeated until one of the players agrees or until a finite deadline is reached. A unique SPE can be found with backward induction. The SPE reaches an immediate agreement on an efficient division of the surplus. The driving force behind this result is players' impatience to reach an agreement and one player's opportunity to make a take-it-or-leave-it offer in the final period [32]. Rubinstein [39] studies the alternating-offers game with infinite horizon, i.e., the bargaining can go on forever if no player accepts an offer of the other player. This model with infinite horizon can be used to approximate the situation with many negotiation rounds available. It shows that the strong result of a unique SPE prediction can be generalized from the finite case to that of infinitely many stages. The unique SPE approaches the outcome of the Nash bargaining solution, as described in Section 3.1, with the bargaining power of the two players a — ln^n