Game Theoretic Models for Strategic Bargaining

Chapter 2 Game Theoretic Models for Strategic Bargaining Nicola Gatti Politecnico di Milano Dipartimento di Elettronica e Informazione Piazza Leonardo da Vinci 32, 20133 Milano, Italy Email: [email protected]

Abstract. Bargaining is one of the most common negotiation situation. Agents must reach an agreement regarding how to distribute objects or a monetary amount. On the one side, each agent prefers to reach an agreement, rather than abstain from doing so. On the other side, each agent prefers that agreement which most favors her interests. This problem has been widely studied in the game theory literature, under the assumption that agents are intelligent (i.e., able to collect all the information over the opponents) and rational (i.e., able to maximize their gain). The most satisfactory models represent a bargaining situation as a non-cooperative (strategic) game, where a solution is a strategy profile, specifying a strategy per agent, that is somehow in equilibrium. In this chapter, I survey the game theoretic strategic models for bargaining and the corresponding solving algorithms. Although the bargaining problem has been studied in the literature for almost 30 years, no algorithm able to solve a general bargaining problem with uncertainty is known. I discuss also the critical issues behind the game theoretic approaches and some possible new research directions.

2.1

Introduction

Bargaining is one of the most common negotiation situation. In a bargaining, agents try to reach an agreement regarding how to distribute objects or a monetary amount. On the one side, each agent prefers to cooperate to reach F. Lopes and H. Coelho (Eds.), Negotiation and Argumentation in MAS c 2010 Bentham Science Publishers Ltd. All rights reserved −

1

2

N. Gatti

an agreement, rather than abstain from doing so. On the other side, each agent competes with the other, preferring that agreement which most favors her interests. As a result, a bargaining problem combines cooperative and non-cooperative aspects. Microeconomics/game theory provides several models to study bargaining. There are two main approaches. In both approaches, the preferences of the agents over the possible agreements are modeled by using von Neumann and Morgenstern utility functions [1]. The first approach is said cooperative, while the second is said non-cooperative or strategic. In the cooperative approach, a bargaining problem is a multi-objective optimization problem where the objectives are the utility functions of the agents. The attention is exclusively posed on the agreement without considering any procedure whereby the negotiation is carried out. Edgeworth in [2] and Hicks in [3] constrain the agreements to be individually rational (i.e., they provide each agent an utility that is not worse than the disagreement utility) and socially rational (i.e., they are Pareto efficient). Anyway, the space of agreements satisfying these constraints is usually infinite and no criterion to select a specific agreement was known. The bargaining problem was indeterminate by economics until the works by Nash [4, 5] where he provided two seminal papers that constitute the birth of the formal theory of bargaining, usually called axiomatic bargaining. Nash assumes some axioms over how agents should divide the surplus and shows that with these the agreement is unique. The axioms are: invariance to affine transformations, Pareto efficiency, independence of irrelevant alternatives, and symmetry. Essentially, Nash provided a criterion to select a specific Pareto efficient agreement, usually called Nash bargaining solution. Several works follow the Nash’s approach, considering a different set of axioms and thus selecting different agreements, e.g., the Kalai-Smorodinky’s solution discussed in [6]. Differently from the cooperative approach, the strategic approach is instead interested in specifying the procedure whereby the agents reach an agreement.1 Nash introduced the first bargaining model expressed as a strategic game. This model is named the Nash’s demand game [5]. Formally, a game is described by a pair: the mechanism, defining the rules (i.e., number and roles of the agents, actions available to the agents, sequential structure, agents’ preferences), and the strategies, defining the behavior of every agent along the whole game. Solving a game means finding the agents’ strategies that are somehow in equilibrium, where the most famous concept of equilibrium 1

Technically speaking, the problem is not a multi-objective optimization problem, but an equilibrium computation problem.

2. Game Theoretic Models of Strategic Bargaining

3

is the Nash equilibrium. In the Nash demand game, both bargainers must demand simultaneously a utility level. If the pair of utilities is feasible, it is implemented; otherwise, there is disagreement and both receive a utility of zero. This game admits a continuum of Nash equilibrium outcomes, including every point of the Pareto frontier, as well as disagreement. This leads to the indeterminacy of equilibrium outcomes. Only at 1982, thanks to Rubinstein in [7], a satisfactory strategic model for bargaining was provided. His model, called alternating-offers protocol, prescribes that two agents play alternately making offers or accepting the offer made by the opponent at the previous time point. Formerly, Rubinstein assumes that the agents’ utility functions depend on the time point at which the agreement is achieved. Exactly, time discounts exponentially the utility of an agreement. Rubinstein showed that his game admits a unique subgame perfect equilibrium [8], addressing thus the outcome indeterminacy problem. In the computer science literature, the strategic approach is commonly considered the most satisfactory. Indeed, software agents will interact according to a given protocol and, being selfish, they will try to maximize their revenue. The Rubinstein’s alternating-offers protocol is considered the most expressive negotiation protocol and has been widely studied and refined. In particular, agents’ utility functions have been enriched by introducing reservation prices and deadlines and the number of issue has been increased from one to many. A large number of works study the problem of finding the equilibrium strategies when there is uncertainty over the utility functions. It is worth remarking that no algorithm able to solve bargaining problems with arbitrary uncertainty is known. Only specific results for very narrow uncertainty (e.g., over one or two parameters) are known. Furthermore, no computational complexity result over the bargaining problem is known, leaving open the question whether a solution to the bargaining problem with uncertainty is computable or not. In this chapter, I provide a survey of the main results on strategic bargaining with discrete time2 , discussing also some critiques on the strategic approach. In Section 2.2, I present the main strategic bargaining models, describing the original alternating-offers and its extensions proposed in computer science (with reservation price and deadlines, multiple issues, bargaining in markets, and multilateral bargaining). In Section 2.3, I discuss the solution concepts appropriate for bargaining with complete information and I show how a solution to the bargaining problem can be found. In Section 2.4, I focus on the 2

Some models with continuous time can be found in literature, but they cannot be applied to real-world situations where the communication between software agents is not negligible.

4

N. Gatti

bargaining with uncertainty, discussing how the game problem is formulated and presenting the main contributions presented both in the microeconomics and computer science literature. In Section 2.5, I discuss the critical issues behind the strategic approach, i.e., common knowledge and computational hardness, that make its applicability hard. I show also how a solution concept that has not been considered in bargaining community so far can address the two previous critical issues. Finally, Section 2.6 concludes the chapter.

2.2 2.2.1

Strategic Bargaining Models Rubinstein’s Protocol

The seminal Rubinstein’s protocol models a bargaining problem over a single issue as a two-agent extensive-form game with perfect information and infinite horizon. The protocol prescribes that two agents, a buyer b and a seller s, play alternatively at discrete time points. Denote by t ∈ N a generic time point and by ι : N → {b, s} the function, called player function, that returns the agent that plays at time point t. Function ι(t) is defined as follows: ι(0) is a data of the problem and for t > 0 it is such that ι(t) 6= ι(t − 1). Agents negotiate on a real valued parameter, say x, whose value belongs to the range [0, 1]. The pure strategies σι(t) (t)s available to agent ι(t) at t > 0 are: • offer(x), where x ∈ [0, 1] is the offer for parameter x; • accept, that concludes the bargaining with an agreement, formally denoted by (x, t), where x is such that σι(t−1) (t − 1) = offer(x) (i.e., the value offered at t − 1), and t is the time point at which the offer is accepted. At t = 0 only actions offer(x) are available. Notice that agents can negotiate for a time indefinitely long. Seller’s and buyer’s utility functions, denoted by Us : [0, 1] × N → R and Ub : [0, 1] × N → R respectively, return the agents’ utility for each possible agreement. They are defined as Ub (x, t) = (1 − x) · (δb )t for the buyer and Us (x, t) = x · (δs )t for the seller. Parameters δb and δs are called discount factors and capture how the agents’ utility decrease as t increases. 2.2.2

Protocol Extensions

Bargaining with Deadlines and Reservation Prices The main extension of the Rubinstein’s model commonly used in the computer science literature captures reservation prices and deadlines. Reservation prices express, for the buyer, the largest acceptable offer and, for the seller, the smallest one.


5

Deadlines express the last time point by which agents are interested in reaching agreements. The protocol and agents’ utility functions described in the previous section are modified as follows. The protocol prescribes that the agents can accomplish an additional action, called exit, that allows agents to leave the negotiation. When this action is played, the negotiation stops and the outcome is N oAgreement. The utility of N oAgreement is zero for both agents. Denote by RPb , RPs ∈ R+ the reservation prices of buyer and seller respectively (we assume RPb ≥ RPs ) and by Tb , Ts ∈ N the deadlines of buyer and seller respectively. Agents’ utility functions are:

Ub (·) =

Us (·) =

8 > :(x, t) 8 > :(x, t)

0 (

(RPb − x) · δbt

if t ≤ Tb , otherwise

(x − RPs ) · δst

if t ≤ Ts , otherwise

where < 0. It is worth noting that, after the deadline Ti , the utility value of every agreement for agent i is strictly negative. This induces a rational agent to leave the game rather than reaching any agreement once her deadline expired. Notice that the deadline is not due to the protocol, but it is due to the preference of the agents. Multi-Issue Bargaining A very common extension aims at capturing the situation in which agents negotiate over multiple issues [?, 9], e.g., the price and quality of a service. A crucial point is the procedure with which the issues are negotiated: in-bundle, when the issues are negotiate simultaneously, and issue-by-issue, when the issues are negotiate sequentially. Each procedure presents a specific protocol. In the in-bundle case, called n the number of issues, agents negotiate over a vector x = [x1 , . . . , xn ] of real valued parameters xi ∈ R. Each action offer(x) specifies a vector of values x ∈ Rn , one for each parameter, while action accept will force an agent to accept the entire vectors of values. The utility functions over agreements (x, t) are defined in the literature in several ways. The most common ones are the linear-additive utility functions. For each parameter xj , each agent i has a specific reservation price RPij and discount factor δij , while the deadline is usually the same for all the issues. The utility of an agreement (x, t) before the deadline is easily given by the sum of

6

N. Gatti

the utility of the single parameters xi as defined in the previous section. Some works focus on models in which utility functions are non-linear and/or they are not additive. Usually, these utility functions are separable, being defined as the product of two components: the first one is a (non-linear) function defined over x and the second one is defined over t. The issue-by-issue case is easily the repetition of the single issue model per each single issue. The utility functions commonly used are the linearadditive ones. In this case, each issue has a specific deadline. The issue-byissue procedure is associated with the agenda problem. That is, the problem to determine the sequence of issues to be negotiated. Bargaining in Markets Here the situation where multiple buyers and multiple sellers is considered. Bargaining in markets is usually captured by providing agents with the option to leave the negotiation they are carrying on and starting a new one with a different agent. This option is called outside option. In this case, the bargaining problem is associated with the matching problem. That is, the problem to determine a matching for each agent. This is because, when outside option is available, the matching can change during time. Several models can be found in the literature. In [10] the authors do not explicitly model the presence of several opponents. When the outside option is played, an agent receives a given monetary value. In [11, 12] the authors explicitly model the presence of several opponents and assume the matching between the agents to be random. At each time point there is a positive value of probability that an agent will meet a different opponent. In this way, the outside option is not directly available to the agents, but it played by the nature. To the best of my knowledge, the unique work providing a bargaining model that explicitly capture the presence of opponents where outside option is available to the agents is discussed in [13]. This model divides each time point into two stages where, in the first stage, non-matched agents play matching actions proposing to be matched with specific agents, while, in the second stage, matched agents play an extension of the alternating-offers protocol where outside option is available. In this way, if agent i that is currently matched with agent j and there is a non-matched agent k that is interested to be matched with i, agent i can play the outside option stopping to negotiate with j and starting to negotiate with k at the next time point. In [13], possible delays between the match and the start of the negotiation are considered. Multilateral Bargaining Multilateral bargaining refers to situations in which more than two bargaining parties are present. The first extension is due Herrero


7

in [14]. One of the agents, say agent 1, begins by making an offer specifying the gain for each agent. Then, the other agents can accept or reject it. If all accept, the negotiation terminates with an agreement. If at least one agent rejects, time elapses and the next period another agent, say agent 2, makes a new offer, and so on. An alternative extension is given by Jun in [15], Chae and Yang in [16], Krishna and Serrano in [17]. The protocol is the same of Shaked and Herrero in [18] except that, when agents replies to an offer, all the agents that accept the offer leave the game, while the other continue to bargain with the proposer over the part of the surplus that has not been committed to any agent.

2.3 2.3.1

Solving a Bargaining Problem with Complete Information Solution Concept and Solving Algorithms

Game theory provides several solution concepts for studying games. The choice of the solution concept to adopt depends on two issues: the characteristics of the game (e.g., strategic-form vs. extensive-form) and the assumptions over agents’ rationality and knowledge (e.g., completeinformation vs. incomplete-information3 ). The most known solution concept is the Nash equilibrium: it is appropriate for strategic-form games when agents have complete information. I recall that a Nash equilibrium is a strategy profile σ = (σb , σs ), specifying a strategy for each agent, such that no agent i can improve her utility by deviating unilaterally from her strategy σi . Game theoretic works on bargaining are based on the assumption of common-information; in this section I discuss only the case with completeinformation, remanding the incomplete-information case to Section 2.4. Alternating-offers bargaining is an extensive-form game. Under the assumption of complete-information the appropriate solution concept is the subgame perfect equilibrium [19]. This concept is a refinement of Nash.4 Call subgame any subtree of the game tree. In the case of bargaining, a subgame is a reduced bargaining game starting from a time point t where agent ι(t) has tackled a given action. A subgame perfect equilibrium is a strategy profile such that it is a Nash equilibrium in every subgame. 3

Complete-information means that agents know with certainty all the parameters of the game (e.g., the utility parameters of the opponents), while incomplete-information means some agents have no information over some parameters of the game. 4 In extensive-form games, some Nash equilibria are non-reasonable given the sequential structure of the game. The subgame perfect equilibrium concept allows one to remove all the non-reasonable Nash equilibria.

8

2.3.2

N. Gatti

Equilibrium Strategies

Rubinstein’s Protocol A subgame perfect equilibrium can be found by backward induction when the game is finite. Bargaining is with infinite horizon and then backward induction is not applicable. However, the solution can be easily found by exploiting symmetries. The basic idea behind Rubinstein’s argument is that for every time point t each agent has an optimal offer (independent of t), say x∗b and x∗s for buyer and seller respectively, such that each agent is indifferent between accepting the optimal offer of the opponent and making her optimal offer (that will be accepted at the next time time). The optimal offers can be computed in closed form as: x∗b =

δs (1 − δs ) , 1 − δb δs

x∗s =

1 − δs 1 − δb δs

The subgame equilibrium strategies are simple: the buyer accepts any seller’s offer smaller than x∗s otherwise she offers x∗b , while the seller accepts any buyer’s offer larger than x∗b otherwise she offers x∗b . Formally, the strategies are defined as:

σb∗ (t) =

8 > :t > 0

∗) offer(x b ( accept offer(x∗b )

8 > :t > 0 otherwise

∗) offer(x s ( accept if b’s offer ≥ x∗b . offer(x∗s ) otherwise

This equilibrium is unique and prescribes that agents achieve immediately an agreement. Extensions When agents’ utility functions present deadlines, the game, although it is with infinite horizon, is essentially a finite game. This is because there exists a time point, say T , beyond which at least one agent strictly prefers to leave the negotiation rather then to negotiate and therefore the game terminates. Easily, time T is defined as T = min{Tb , Ts }. From T back, the equilibrium strategies can be found by backward induction. The solution of game is conceptually similar to the solution the Rubinstein’s model, differing exclusively for the definition of the agents’ optimal offers (see [20] for details). In this case, optimal offers depend on t. Call x∗ (t) the ι(t)’s best offer at t. Consider the subgame which starts at t = T − 1. This subgame is essentially an ultimatum game [21]. ι(T ) accepts any offer x such that Uι(T ) (x, T ) ≥ 0 (x ≤ RPb if ι(T ) = b and x ≥ RPs if ι(T ) = s), she leaves the game otherwise. The ι(T − 1)’s optimal offer x∗ (T − 1) maximizes ι(T − 1)’s utility (i.e., x∗ (T − 1) = RPb if ι(T − 1) = s


9

and x∗ (T − 1) = RPs if ι(T − 1) = b). The subgames which start at time t < T − 1 can be studied in a similar way. Suppose that x∗ (t + 1) have been found and x∗ (t) needs to be derived. Consider the subgame composed of time points t and t + 1 as an ultimatum game variation in which ι(t + 1) accepts any offer x such that Uι(t+1) (x, t + 1) ≥ Uι(t+1) (x∗ (t + 1), t + 2) and offers x∗ (t + 1) otherwise. The ι(t)’s best offer, among all the acceptable offers at time point t + 1, is the one which maximizes ι(t)’s utility. This offer can be computed as: ( ∗

x (t) =

RPs + (x∗ (t + 1) − RPs ) · δs RPb − (RPb − x∗ (t + 1)) · δb

if ι(t) = b . if ι(t) = s

The computation of the values x∗ (t)s is linear in t. The buyer’s optimal strategy is, before her deadline, to accept at t any seller’s offer that is larger than x∗ (t − 1), at the deadline, to accept any offer, and, after the deadline, to make exit (the seller’s ones are analogous). Formally, the buyer’s strategy is:

σb∗ (t) =

8 t=0 > > > > > > > > > > t=T > > > > > : t>T

∗ (0)) offer(x ( accept if s’s offer ≤ x∗ (t − 1) ∗ offer(x (t)) otherwise ( . accept if s’s offer ≤ x∗ (t − 1) exit otherwise exit

This equilibrium is unique. It can be observed that limT →+∞ x∗ (0) is equal to x∗b when ι(0) = b and equal to x∗s when ι(0) = s and therefore the solution with deadline T converges to the solution of the Rubinstein’s model as the deadline goes to the infinity. I report in Fig. 2.1 an example of sequence of agents’ optimal offers with and without deadlines. Consider the situation where there are multiple issues that negotiated inbundle. The solution is exactly the one with a single-issue except for two differences [22]. First, the optimal offers x∗ (t)s are tuples of values, one for each single issue. Second, with a single issue the offers to accept can be compactly expressed by specifying a threshold on the value of the received offer, e.g., s accepts at t any offer y such that y ≥ x∗ (t − 1), with multiple issues instead the threshold is on the utility of the received offer, e.g., s accepts at t any offer y such that Us (y, t) ≥ Us (x∗ (t), t). The sequence of the optimal offers x∗ (t)s can be found by backward induction. Essentially, the backward induction construction is the same: at each time point t the optimal offer x∗ (t) of agent ι(t) is the offer such that agent ι(t + 1) is indifferent at t + 1 between accepting it and making her optimal offer x∗ (t + 1). Formally, Uι(t+1) (x∗ (t), t + 1) = Uι(t+1) (x∗ (t + 1), t + 2). The difference between the

10

N. Gatti

1 0.9 0.8 x*s

0.7

value

0.6 0.5 x*(0)

x*(2)

0.4

x*

b

* x (4) *

x (6)

0.3

x*(1)

x*(3)

0.2

x*(8)

x*(5) *

x (7)

0.1 0

* x (9)

0

1

2

3

4

5 time

6

7

8

9

10

Fig. 2.1 Sequences of agents’ optimal offers with: RPb = 1, δb = 0.7, Tb = 10, and RPs = 0, δs = 0.8, Ts = 15, and ι(0) = s. The dotted lines denote the agents’ optimal offers without deadlines.

multiple issue situation and the single issue situation lays on how x∗ (t) can be computed given x∗ (t + 1). With a single issue, x∗ (t) can be computed in closed form given x∗ (t + 1). With multiple issues, x∗ (t) can be computed as the result of an optimization problem. When utility functions are linear, the problem is polynomial, being a linear optimization problem (an interested reader can find the non-linear case in [23]). The resolution of issue-by-issue procedure is more involved and therefore it is not discussed here. I point an interested reader to [9]. It is worth noting that the choice of the negotiation procedure is associated with the problem of finding an equilibrium whose outcome is Pareto efficient. It has been proved that the in-bundle procedure always grants to find Pareto efficient agreements, while issue-by-issue does not [9, 22]. This pushes for the employment of the in-bundle procedure. The resolution of bargaining in markets and multilateral-bargaining is very complicated and therefore it is not discussed here. Also in these two cases the solution is constituted by multiple sequences of optimal offers and a set of choice rules as above. The computation of the equilibrium strategies keeps to be polynomial if the utility functions are linear. 2.3.3

Experimental Evidences

The computation of the equilibrium strategies with complete information is easy (when utility functions are linear) and agents achieve an agreement immediately at time t = 0. Notice that the presence of the agents


11

deadlines affects the value x of the agreements, not the time point at which the agreement is achieved. The scientific community has compared these theoretical results with respect to the agreements achieved by human beings playing the alternating-offers game [24]. Surprisingly, the behavior of the human beings has been much different from the above equilibrium strategies. More precisely, the agreements were never achieved an the beginning of the game, but close to the deadline where the player with the longest deadline concedes almost all the surplus to the opponent.

2.4 2.4.1

Bargaining with Uncertainty Solution Concept

In the presence of information incompleteness (e.g., when an agent does not know the opponent’s discount factor), the game is cast to a game with uncertainty by introducing probability distributions over the unknown parameters. This game is said to be with uncertainty. A game with uncertainty (a game where agents cannot perfectly observe the opponents’ actions) cannot be solved as it is, but it must be cast to an imperfect information game according to the Harsanyi’s transformation. Exactly, each agent can be of different types (each differentiating for the values of the uncertain parameter) and each type is associated with a probability. The game is with imperfect information because at the root of the game tree the Nature selects the type of each agent that will be private information of the agent. The concept of subgame perfect equilibrium (appropriate with complete information) is not satisfactory when information is imperfect, i.e., when agents cannot perfectly observe their opponents’ actions. Specifically, it does not have the power to cut the so called incredible threats [8], i.e., Nash equilibria that are non-reasonable given the sequential structure of the game. The most common refinement of the subgame perfect equilibrium concept in presence of information imperfectness is the sequential equilibrium [25]. I review this concept. Rational agents try to maximize their expected utilities relying on their beliefs about the opponent’s private information [1] and such beliefs are updated during the game, depending on which actions have been actually accomplished [25]. The set of beliefs held by each agent over the other’s private information after every possible sequence of actions in the game is called a system of beliefs and is usually denoted by µ. These beliefs are probabilistic and their values at time point t = 0 are given data of the problem. How beliefs evolve during the game is instead part of the solution which should

12

N. Gatti

be found for the game. A solution of an uncertain information bargaining is therefore a suitable couple a = hµ, σi called assessment. An assessment a = hµ, σi must be such that the strategies in σ are mutual best responses given the probabilistic beliefs in µ (rationality); and the beliefs in µ must reasonably depend on the actions prescribed by σ (consistency). Different solution concepts differ on how they specify these two requirements. For a sequential equilibrium a∗ = hµ∗ , σ ∗ i, with σ ∗ = hσb∗ , σs∗ i, the rationality requirement is specified as sequential rationality. Informally, after every possible sequence of actions S, on or off the equilibrium path, the strategy σs∗ must maximize s’s expected utility given s’s beliefs prescribed by µ for S, and given that b will act according to σb∗ from there on and vice versa. The notion of consistency is defined as follows: assessment a is consistent in the sense of Kreps and Wilson (or simply consistent) if there exists a sequence an of assessments, each with fully mixed strategies and such that the beliefs are updated according to Bayes’ rule, that converges to a. By Kreps and Wilson’s theorem any extensive-form game in incomplete information admits at least one sequential equilibrium in mixed strategies [25]. Moreover, as is customary in economic studies, e.g. Rubinstein’s [26], I consider only stationary systems of beliefs, namely, if s believes a b’s type with zero probability at time point t, then she will continue to believe such a type with zero probability at any time point t > t. 2.4.2

Known Results

The study of bargaining with uncertain-information is an open challenging problem even in the bilateral case with a single issue without market competition. I recall that, differently from what happens with complete information, the backward induction method cannot be applied when information is uncertain. Although the bargaining problem has been studied for about 30 years, no work presented in the literature so far is applicable regardless of the kind (i.e., the uncertain parameters, e.g., RPb or RPs or both, or δb or δs or both, or both RPi and δi ) and the degree (i.e., the number of the parameters’ possible values, e.g., RPb can be of ten or hundred or thousand types) of uncertainty. Furthermore, no general computational complexity result is known. The algorithmic/closed form results presented in the literature can be split into two main class, differing for the adopted techniques and for the scope. The first one is composed of the works proposed in the microeconomics literature. The second one is composed of the works proposed in the computer science literature.


13

Microeconomics Works The microeconomic literature focuses on a simplified model where deadlines are not present. Although this simplification makes the problem much easier, only partial results are known. The main microeconomics results are in closed form. In [26], the author studies the situation in which uncertainty is over the discount factor of one agents with two possible values. In [27], the authors study the situation in which uncertainty is over over the reservation price of both agents with two possible values per agent. Other well known results are: in [28] the authors study onesided uncertainty in a simplified model where only the seller make offers and the buyer can only accept or reject them; in [29] the authors develop a general selection criterion, termed perfect sequential equilibrium, and apply it to the alternating-offer bargaining game without deadline, but this equilibrium does not generally exist (e.g., it fails when the discount factor is sufficiently high); in [30] the authors study the role of delay in one-sided uncertainty over the reservation price. No known result deals with uncertainty over multiple parameters.

Computer Science Works The computer science literature [31] provides general purpose algorithms to search for sequential equilibria [32], but they work only on games with a finite number of actions and they do not produce belief systems off the equilibrium path. This makes such algorithms not suitable for bargaining. Several efforts have been accomplished to extend the backward induction algorithm to solve games with uncertain-information [9, 33]. The basic idea behind these extensions is to break the circularity between strategies and belief system by computing at first the strategies with the initial beliefs and then deriving the beliefs that are consistent with the strategies. However, as showed in [34, 35], the solutions produced by these algorithms may not be equilibria, the strategies being not assured to be sequentially rational given the belief system. An hybrid approach [34] combines analytical results and searching algorithms to solve the setting in which uncertainty is over the deadline of one agent with an arbitrary number of possible values. However, due to the mathematical machinery it needs to solve a very specific setting of uncertainty, its extension to capture multiple uncertainty kinds appears to be impractical. No computationally complexity result is known, except the one provided in [34] where the authors show that bargaining with one-sided uncertain deadlines is polynomial in the length of the longest deadline. I recall that no approximate result (in terms of approximate equilibria, e.g., -Nash, see [31]) is known for bargaining. The lack of formal complete results

14

N. Gatti

pushes for the employment of tactic-based heuristics without any equilibrium guarantee, e.g., in [36] the agents use negotiation decision functions to decide the next offer to make. However, it is worthwhile remarking the lack of studies comparing the agreements obtained with the known exact algorithms with respect to those obtained with heuristic algorithms.

2.5 2.5.1

Critiques to the Game Theoretical Approach Critical Issues

Although bargaining is a microeconomic problem and game theory is the “natural” technique to address it, several assumptions required by game theory make its application hard in realistic bargaining situations. There are two main critical issues: the assumption of common knowledge and the computational hardness. Common Knowledge As discussed in the previous sections, game theory requires that the knowledge (in this case, on the values of the parameters of the agents’ utility functions) is common. With complete information, this means that every agent knows the values of the opponents, knows that every opponent knows her own values, and so on. Surprisingly, with uncertain information, the assumption of common knowledge is harder. Indeed, it requires that every agent has a Bayesian prior (expressed as a probability distribution) over the opponents’ parameter values and that these priors are common knowledge, e.g., if the seller agent believes that the buyer’s deadline belongs to a given probability distribution, then the buyer must know this probability distribution. Although the common knowledge assumption seems to be unrealistic in all the practical negotiation situations, it is common idea of the scientific community that it is reasonable in the electronic commerce scenario. The motivations follow. Negotiation in electronic commerce will be carried out by software agents on electronic marketplaces. Basically, the marketplace will have information on the agents (e.g., the traces of the previous negotiations) and can process it to produce Bayesian priors over the agents themselves. When a buyer (seller) will decide to negotiate with a seller (buyer), the electronic marketplace will provide the Bayesian priors to both agents. At the end of the negotiation, the electronic marketplace will update the priors. However, the Bayesian prior updating process combined to the play of the game can lead to stable states that are not Nash equilibria. These stable states, called self-confirming equilibria, are discussed below.


15

Computational Hardness While the problem of computing agents’ equilibrium strategies with complete information is polynomial, no general result is known when there is uncertainty. More precisely, finding a sequential equilibrium of a game is a PPAD-complete problem and, although we do not know whether or not PPAD⊆P, it is generally believed that PPAD6⊆P and therefore that in the worst case no polynomial time algorithm exists. The unique computational result on bargaining is discussed in [34], where the authors show that, when uncertainty is over the deadline of one agent, the problem is polynomial. Anyway, this result is based on the fact that the utility functions of all the types differ only at the deadlines, being the same before them. This property does not hold in general when uncertainty is over reservation prices and discount factors. The common belief in the scientific community is that there exists no polynomial time algorithm solving bargaining with uncertainty. This pushes researchers to resort to approximate equilibrium concepts.

2.5.2

Self-Confirming Equilibrium Concept

The concept of self-confirming equilibrium [37] can play a prominent role to address the two above critical issues. Although this concept has been developed about 20 years ago, it has been never taken into account in the bargaining literature. The basic idea is simple: it provides the stability conditions of the strategies of two (or more) agents that repeatedly play without having any information over the opponents’ utilities. Implicitly, this captures the situation in which agents, drawn from a given population, play a game and then update their beliefs on the basis of actions undertaken by the opponents. While in strategic-form games, every self-confirming equilibrium is a Nash equilibrium, in extensive-form games a self-confirming equilibrium may be not a Nash equilibrium. This is because, off the equilibrium path, agents can have wrong beliefs over the opponents strategies. Instead, every Nash equilibrium is a self-confirming equilibrium, the set of self-confirming equilibria containing the set of Nash equilibria. The resort to the self-confirming equilibrium concept allows one to address the critique to the common knowledge assumption. This is because it does not require the assumption of common knowledge. Furthermore, although the computational complexity class of finding a self-confirming equilibrium is the same of computing a Nash equilibrium [38], the computation of a selfconfirming equilibrium is much faster.

16

2.6

N. Gatti

Discussions and Future Research Directions

Bargaining is the most common negotiation situation where two or more agents try to divide an utility surplus. Microeconomics/game theory provides the formal tools to study the bargaining problem. In particular, two approaches can be recognized: the cooperative approach, that models the bargaining as a multiobjective problem without considering the procedure whereby the negotiation is carried out, and the non-cooperative (strategic) approach, that models the bargaining as a non-cooperative game and casts the problem of solving the bargaining as an equilibrium computation problem. In this chapter, I focused on this latter approach. I discuss the main game theoretic models for strategic bargaining. Furthermore, I discuss the solving the most known algorithms. The game theoretic study of bargaining presents two main critical assumptions: common knowledge and computational hardness. Along the paper I emphasized these two critical issues and I discussed how the concept of self-confirming equilibrium could address them. A recent preliminary work provides a promising approach to solve bargaining with any kind and degree of uncertainty [39]. The basic idea is to reduce a bargaining to a finite game, solve this game with algorithms known in the literature, and, finally, to remap the strategy to the original game. While with two types the solution is very easy (linear time), with more than two types the algorithm the algorithms takes exponential time. This algorithm allows one to solve in exact way small bargaining situations. An interesting research direction is the development of anytime algorithms (as, e.g., in [40]) to find approximate solutions when the number of types is large. This would lead to provable -approximate solutions.

References 1. J. von Neumann and O. Morgenstern, Theory of Games and Economic Behavior. Princeton, USA: Princeton University Press, 1947. 2. F. Edgeworth, Mathematical Psychics. London, UK: C. Kegan & Co., 1881. 3. J. Hicks, The Theory of Wages. London, UK: Macmillan and Co., 1932. 4. J. F. Nash, “The bargaining problem,” ECONOMETRICA, vol. 18, pp. 155–162, 1950. 5. ——, “Two-person cooperative games,” ECONOMETRICA, vol. 21, no. 1, pp. 128–140, 1953. 6. E. Kalai and M. Smorodinsky, “Other solutions to nash’s bargaining problem,” ECONOMETRICA, vol. 43, no. 1, pp. 513–518, 1975. 7. A. Rubinstein, “Perfect equilibrium in a bargaining model,” ECONOMETRICA, vol. 50, no. 1, pp. 97–109, 1982. 8. D. Fudenberg and J. Tirole, Game Theory. Cambridge, USA: The MIT Press, 1991.


17

9. S. S. Fatima, M. J. Wooldridge, and N. R. Jennings, “On efficient procedures for multi-issue negotiation,” in TADA-AMEC, Hakodate, Japan, May 9 2006, pp. 71–84. 10. K. Binmore, A. Shaked, and J. Sutton, “An outside option expirement,” Q J ECON, vol. 104, no. 4, pp. 753–770, 1989. 11. P. Jehiel and P. Moldovanu, “Cyclical delay in bargaining with externalities,” REV ECON STUD, vol. 62, no. 4, pp. 619–637, 1995. 12. A. Rubinstein and A. Wolinsky, “Decentralized trading, strategic behavior and the walrasian outcome,” REV ECON STUD, vol. 56, no. 1, pp. 63–78, 1990. 13. N. Gatti, “Extending the alternating-offers protocol in the presence of competition: Models and theoretical analysis,” ANN MATH ARTIF INTEL, vol. 55, no. 3-4, pp. 189–236, 2009. 14. M. Herrero, N-player bargaining and involuntary underemployment. London, UK: Ph.D. thesis, London School of Economics, 1985. 15. B. Jun, A structural consideration on 3-person bargaining. USA: Ph. D. Thesis, Department of Economics, University of Pennsylvania, 1987. 16. S. Chae and J. Yang, “An n-person pure bargaining game,” J ECON THEORY, vol. 62, pp. 86–102, 1994. 17. V. Krishna and R. Serrano, “Multilateral bargaining,” REV ECON STUD, vol. 63, pp. 61–80, 1996. 18. A. Shaked and J. Sutton, “Involuntary unemployment as a perfect equilibrium in a bargaining model,” ECONOMETRICA, vol. 52, pp. 1351–1364, 1984. 19. J. C. Harsanyi and R. Selten, “A generalized Nash solution for two-person bargaining games with incomplete information,” MANAGE SCI, vol. 18, pp. 80– 106, 1972. 20. I. Stahl, Bargaining Theory. Sweden: Stockolm School of Economics, 1972. 21. U. Gneezy, E. Haruvy, and A. E. Roth, “Bargaining under a deadline: Evidence from the reverse ultimatum game,” GAME ECON BEHAV, vol. 45, pp. 347–368, 2003. 22. F. Di Giunta and N. Gatti, “Bargaining over multiple issues in finite horizon alternating-offers protocol,” ANN MATH ARTIF INTEL, vol. 47, no. 3-4, pp. 251– 271, 2006. 23. S. F. M. Wooldridge and N. Jennings, “An analysis of feasible solutions for mulitissue negotiation involving nonlinear utility functions,” in AAMAS, 2008, pp. 1040–1041. 24. A. Roth, “Bargaining experiments,” Handbook of Experimantl Economics, pp. 253–348, 1995. 25. D. R. Kreps and R. Wilson, “Sequential equilibria,” ECONOMETRICA, vol. 50, no. 4, pp. 863–894, 1982. 26. A. Rubinstein, “A bargaining model with incomplete information about time preferences,” ECONOMETRICA, vol. 53, no. 5, pp. 1151–1172, 1985. 27. K. Chatterjee and L. Samuelson, “Bargaining under two-sided incomplete information: The unrestricted offers case,” OPER RES, vol. 36, no. 4, pp. 605– 618, 1988. 28. L. Ausubel and R. Deneckere, “Reputation in bargaining and durable goods monopoly,” ECONOMETRICA, vol. 57, no. 3, pp. 511–531, 1989. 29. S. Grossman and M. Perry, “Sequential bargaining under asymmetric information,” J ECON THEORY, vol. 39, no. 1, pp. 120–154, 1986. 30. F. Gul and H. Sonnenschein, “On delay in bargaining with one-sided uncertainty,” ECONOMETRICA, vol. 56, no. 3, pp. 601–611, 1988.

18

N. Gatti

31. Y. Shoham and K. Leyton-Brown, Multiagent Systems: Algorithmic, Game Theoretic and Logical Foundations. Cambridge, USA: Cambridge University Press, 2008. 32. P. B. Miltersen and T. B. Sorensen, “Computing sequential equilibria for twoplayer games,” in SODA, 2006, pp. 107–116. 33. S. S. Fatima, M. J. Wooldridge, and N. R. Jennings, “Multi-issue negotiation with deadlines,” J ARTIF INTELL RES, vol. 27, no. 1, pp. 381–417, 2006. 34. N. Gatti, F. Di Giunta, and S. Marino, “Alternating-offers bargaining with onesided uncertain deadlines: an efficient algorithm,” ARTIF INTELL, vol. 172, no. 8-9, pp. 1119–1157, 2008. 35. F. Di Giunta and N. Gatti, “Alternating-offers bargaining under one-sided uncertainty on deadlines,” in ECAI, Riva del Garda, Italy, 2006, pp. 225–229. 36. P. Faratin, C. Sierra, and N. R. Jennings, “Negotiation decision functions for autonomous agents,” ROBOT AUTON SYST, vol. 24, no. 3-4, pp. 159–182, 1998. 37. D. Fudenberg and D. Levine, “Self-confirming equilibrium,” ECONOMETRICA, vol. 61, no. 3, pp. 523–545, 1993. 38. N. Gatti, F. Panozzo, and S. Ceppi, “Mathematical programming formulations to compute steady states in two-player extensive-form games,” in IDTGT at AAAI, 2010, pp. 18–24. 39. S. Ceppi and N. Gatti, “An algorithmic game theory framework for bilateral bargaining with uncertainty,” in AAMAS, Toronto, Canada, 2010, pp. 1489–1490. 40. S. Ceppi, N. Gatti, G. Patrini, and M. Rocco, “Local search methods for finding a nash equilibrium in two-player games,” in IAT, Toronto, Canada, 2010, pp. 335– 342.