Feb 28, 2014 - maximize its profit. Multi-objective optimization : Pareto optima. Stability. Given a strategy, no agent should be able to increase its profit by itself, ...
Finding a Maximum Stable Flow in a Multi-agent Network with Controllable Capacities Nadia CHAABANE(1)
Cyril Briand (1)
(1) LAAS-CNRS,
Marie-Jose´ Huguet(1)
University of Toulouse
ROADEF’14 - 26-28 February 2014
1 / 37
Outline
1
Introduction
2
Multi-agent Network with Controllable Capacities Problem statement Example
3
Finding a maximum-flow stable strategy Characterization of Nash equilibria Problem complexity MILP formulation
4
Conclusion
2 / 37
Outline
1
Introduction
2
Multi-agent Network with Controllable Capacities Problem statement Example
3
Finding a maximum-flow stable strategy Characterization of Nash equilibria Problem complexity MILP formulation
4
Conclusion
3 / 37
Introduction Optimization in a multi-agent context A set of agents is involved in a common decision-making process. Every agent controls its own decision variables (strategy) and is interested in maximizing its profit. The agents’ profit can depend on the strategies of the other agents. ⇒ Which strategy to adopt in order to fulfill the social goal, provided agents’ objectives are satisfied ?
4 / 37
Introduction Optimization in a multi-agent context A set of agents is involved in a common decision-making process. Every agent controls its own decision variables (strategy) and is interested in maximizing its profit. The agents’ profit can depend on the strategies of the other agents. ⇒ Which strategy to adopt in order to fulfill the social goal, provided agents’ objectives are satisfied ?
Multi-objective Optimization opt F (x) = (Z1 (x), Z2 (x), . . . , Zm (x)) s.c. x ∈ ω A = {A1 , . . . , Am } Agents set x = (x1 , . . . , xm ) Vector of agents’ strategies. Zu (x) profit of agent Au 5 / 37
Strategy requirement Efficiency Only non-dominated strategies are of interest because every agent wants to maximize its profit. Multi-objective optimization : Pareto optima
Stability Given a strategy, no agent should be able to increase its profit by itself, while decreasing the profit of others. Game theory : Nash equilibria.
Efficiency Vs. Stability Price of cooperation I I
Price of anarchy : PA=(GOF value in the worst NE)/(OPT) a Price of stability : PS=(GOF value in the best NE)/(OPT)
a. Global Objective Function (GOF). 6 / 37
Outline
1
Introduction
2
Multi-agent Network with Controllable Capacities Problem statement Example
3
Finding a maximum-flow stable strategy Characterization of Nash equilibria Problem complexity MILP formulation
4
Conclusion
7 / 37
Multi-agent Network with Controllable Capacities Definition Multi-agent Network < G, A, Q, Q, C, π, W > : G = (V , E) is a Network : set of nodes V where s, t ∈ V are the source and the sink nodes ; set of arcs E each one having its controllable capacity and receiving a flow.
A = {A1 , . . . , Au , . . . , Am } a set of m agents ; Each arc (i, j) belongs to exactly one agent ; Eu set of arcs belonging to agent Au ;
Q = {q i,j }(i,j)∈E et Q = {q i,j }(i,j)∈E referred as the vectors of normal and maximum arc capacities. qi,j ∈ [q i,j , q i,j ] is the capacity of arc (i, j).
8 / 37
Multi-agent Network with Controllable Capacities Definition (2) C = {ci,j } is the vector of costs. where ci,j is the unitary costs incurred by agent Au for increasing qi,j by one unit ; The Ptotal cost incurred by agent Au for increasing its arc capacities is (i,j)∈Eu ci,j (qi,j − q i,j )
π the reward given by a final customer per unit of flow circulating in the Network. W = {wu } sharing policy of rewards among the agents. The total reward for agent Au is wu × π × F
F circulating flow in the Network. fi,j flow circulating on arc (i, j) holds fi,j ≤ qi,j where qi,j ∈ [q i,j , q i,j ]
9 / 37
Multi-agent Network with Controllable Capacities Multi-objective mathematical program Max s.c. (i) (ii) (iii)
(Z1 (S), Z2 (S), . . . , Zm (S)) fP i,j ≤ qi,j , ∀(i, j) P∈ E (i,j)∈E fi,j = (j,i)∈E fji , ∀j ∈ V {s, t} q i,j ≤ qi,j ≤ q i,j , ∀(i, j) ∈ E fi,j ≥ 0, ∀(i, j) ∈ E
Profit of agent Au is P Zu (S) = wu π (F (S) − F ) − (i,j)∈Eu ci,j (qi,j − q i,j ) S = (Q1 , . . . , Qm ) is the vector of individual strategies of all agents, where : Q ≤ Qu ≤ Q is the vector of the capacities chosen by agent Au for the arcs belonging to him. 10 / 37
Multi-agent Network with Controllable Capacities
Social goal The customer wants to maximize the flow in the Network and give a reward to agents.
Strategies of the agents The capacities chosen by the agent Au for its arcs corresponds to the individual strategy of Au .
Type of the game Non-cooperative game between the agents where each of them aims to maximize its profit.
11 / 37
Example Example of MA-MCF Network Network G(V , U) with controllable capacities Two agents : Blue (AB ) and Green (AG ) Reward and sharing policy : π = 120 and wB = wG =
1 2
F IGURE: Example of MA-MCF Network with two agents
12 / 37
Example (2) Increasing the flow Find an increasing path P in the residual graph such that costu (P) < wu × π for all Au , with : P P costu (P) = (i,j)∈P + ∩Eu ci,j − (i,j)∈P − ∩Eu ci,j
F IGURE: Example of MA-MCF Network with two agents
13 / 37
Example (3) Decreasing the flow Find a decreasing path P in the residual graph such that profitu (P) > wu × π for all Au , with : P P profitu (P) = (i,j)∈P − ∩E ci,j − (i,j)∈P + ∩E ci,j u
u
F IGURE: Example of MA-MCF Network with two agents 14 / 37
Example (4) Efficiency Vs. Stability
Strategy S0
Flow 0
Reward 0
costG 0
costB 0
ZG 0
ZB 0
F IGURE: Strategy S0
15 / 37
Example (5) Efficiency Vs. Stability
Strategy S0 S1
Flow 0 1
Reward 0 120
costG 0 30
costB 0 30
ZG 0 30
ZB 0 30
F IGURE: Strategy S1
16 / 37
Example (6) Efficiency Vs. Stability Strategy S0 S1 S2
Flow 0 1 2
Reward 0 120 240
costG 0 30 80
costB 0 30 80
ZG 0 30 40
ZB 0 30 40
F IGURE: Strategy S2 17 / 37
Example (7) Efficiency Vs. Stability
Strategy S0 S1 S2
Flow 0 1 2
Efficiency Vs. Stability
Reward 0 120 240
costG 0 30 80
costB 0 30 80
ZG 0 30 40
ZB 0 30 40
Strategy S2
S2 maximizes the flow. Is S2 Nash-stable ?
18 / 37
Example (8) Efficiency Vs. Stability
Strategy S0 S1 S2
Flow 0 1 2
Reward 0 120 240
costG 0 30 80
costB 0 30 80
ZG 0 30 40
ZB 0 30 40
F IGURE: Residual graph corresponding to the strategy S2 19 / 37
Example (9) Efficiency Vs. Stability
Strategy S0 S1 S2
Flow 0 1 2
Reward 0 120 240
costG 0 30 80
costB 0 30 80
ZG 0 30 40
ZB 0 30 40
F IGURE: Profitable decreasing path for agent AG 20 / 37
Example (10) Efficiency Vs. Stability Strategy S0 S1 S2
Flow 0 1 2
Reward 0 120 240
costG 0 30 80
costB 0 30 80
ZG 0 30 40
ZB 0 30 40
Agent AG can improve its own profit by modifying unilaterally its strategy. Strategy S20
Flow 1
Reward 120
costG 10
costB 80
ZG 50
ZB -20
21 / 37
Example (11)
Efficiency Vs. Stability
S2 is a Pareto Optimum but not a Nash equilibrium S1 is not Pareto Optimum but a Nash equilibrium
Objective Find a strategy that maximizes the flow, which is a Nash equilibrium.
22 / 37
Outline
1
Introduction
2
Multi-agent Network with Controllable Capacities Problem statement Example
3
Finding a maximum-flow stable strategy Characterization of Nash equilibria Problem complexity MILP formulation
4
Conclusion
23 / 37
Characterization of Nash equilibrium
Nash equilibrium A non-poor strategy S is a Nash equilibrium if and only if there is no profitable path P such that : I
I
∀Au ∈ A, ∀P ∈ P such that (i, j) ∈ Eu costu (P) > wu × π
(1)
profitu (P) < wu × π
(2)
∀Au ∈ A, ∀P ∈ P
24 / 37
Problem complexity Is there a Nash equilibrium strategy S such that F (S) > ϕ ? Strongly NP-complete.
Reduction from 3-partition problem Consider a set ζ = {a1 , . . . , aK } of K = 3 × k integer, such that a : each integer ai ∈]B/4, B/2], for all i = 1, . . . , K PK i=1 ai = k × B
Deciding whether ζ can be partitioned into k subsets so that the sum of integers in each subset is equal to B ? a. Garey, M.R. and Johnson, D.S. (1979). Computers and Intracability : A guide to the Theory of NP-Completeness. W.H. Freeman and Co., New York, USA.
25 / 37
Problem complexity
Proof : Reduction from a 3-partition problem Reduce an instance of the 3-partition problem to an instance of MA-MCF problem. Build up a network G from the 3-partition problem instance, such that K = 9, k = 3, B = 24 and ζ = {7, 8, 7, 7, 7, 8, 9, 10, 9}. Multi-agent Network flow with k = 3 agents each agent owns K = 9 arcs each integer ai is represented by 3 parallel arcs with capacity qi,j ∈ [0, 1] and cost ci,j = ai flow = 0 Reward of the agent Au : wu × π = B + where is small positive number.
26 / 37
Problem complexity Proof : Reduction from a 3-partition problem Build up a network G from the 3-partition problem instance, where K = 9, k = 3, B = 24 and ζ = {7, 8, 7, 7, 7, 8, 9, 10, 9}. Multi-agent Network flow with k = 3 agents each agent owns K = 9 arcs each integer ai is represented by 3 parallel arcs with capacity qi,j ∈ [0, 1] and cost ci,j = ai flow = 0 Reward of the agent Au : wu × π = B + where is small positive number.
0
(e9 , 7)
(e18 ,7)
(e2 , 7)
(e1 ,8)
(e0 , 7)
(e3 , 7)
(e5 ,8)
(e4 , 7)
(e7 ,10)
(e6 ,9)
(e8 ,9) (e17 ,9)
(e10 ,8) 1
2
(e19 ,8)
3
4
…
5
6
7
8
9
(e26 ,9)
A1 A2 A3
F IGURE: Reduction from a 3-partition instance problem with k = 3
27 / 37
Problem complexity Proof : Reduction from a 3-partition problem (2) Find whether it exists a Nash strategy such that the flow is strictly greater than 0? Find a profitable path such that the cost for each agent must not exceed the reward B + = 24 +
0
(e9 , 7)
(e18 ,7)
(e2 , 7)
(e1 ,8)
(e0 , 7)
(e3 , 7)
(e5 ,8)
(e4 , 7)
(e7 ,10)
(e6 ,9)
(e8 ,9) (e17 ,9)
(e10 ,8) 1
2
(e19 ,8)
3
4
…
5
6
7
8
9
(e26 ,9)
A1 A2 A3
F IGURE: Reduction from a 3-partition instance problem with k = 3
28 / 37
Problem complexity
Proof : Reduction from a 3-partition problem (2) Find, whether it exists, a Nash strategy such that the flow is strictly greater than 0? Find a profitable path such that the cost for each agent must not exceed the reward B + = 24 +
Problem complexity Finding a Nash Equilibrium that maximizes the flow in a multi-agent Network with controllable capacities is NP−hard.
29 / 37
Mathematical formulation
Finding a NE maximizing the flow Max s.t.
F
(1)
P
(iii)
0 ∀i 6= s, t F ,i =s , ∀i ∈ V (i,j)∈E fi,j − (j,i)∈Er fj,i = −F , i = t q i,j ≤ qi,j ≤ q i,j , ∀(i, j) ∈ E
(iv )
profitu (P) < wu × π, ∀P ∈ Gf
P
Stability constraints
fi,j ≥ 0, ∀(i, j) ∈ E
30 / 37
Mathematical formulation
How to formulate the stability constraints ? Linearize the Nash stability constraints as constraints of MILP. (iv ) profitu (P) < wu × π, ∀P ∈ Gf
Stability constraints
fi,j ≥ 0, ∀(i, j) ∈ E Formulating the constraints based on the identification of decreasing path P ∈ Gf having the maximum profit profitu (P).
31 / 37
MILP formulation
Constraint reformulation Constraint reformulation is based on identification of path P ∈ P(S) having maximal profitu (P). profitu (P) < wu × π, ∀P ∈ P(S) ⇒ max profitu (P) < wu × π ∀P∈P
All path have to be computed. Computing the longest path for a given strategy using primal/dual constraints.
32 / 37
MILP formulation Using primal/dual constraints : Primal : longest path Min s.t. (i) (ii)
u tn+1
tju − tiu ≥ δFi,j,u , ∀(i, j) ∈ E, ∀Au ∈ A tiu − tju ≥ δBj,i,u , ∀(i, j) ∈ E, ∀Au ∈ A tiu ≥ 0, ∀i ∈ V
Dual Max s.t.
P
(i)
P
(i,j)∈E
ϕui,j × δFi,j,u +
P
(i,j)∈Er
ϕui,j × δBj,i,u
0 ∀i 6= s, t u u −1 , i = s , ∀i ∈ V , ∀Au ∈ A (i,j)∈E∪Er ϕi,j − (i,j)∈E∪Er ϕj,i = 1,i =t φui,j ∈ {0, 1} ∀(i, j) ∈ E, ∀Au ∈ A P
33 / 37
Outline
1
Introduction
2
Multi-agent Network with Controllable Capacities Problem statement Example
3
Finding a maximum-flow stable strategy Characterization of Nash equilibria Problem complexity MILP formulation
4
Conclusion
34 / 37
Conclusion Multi-agent min cost flow Multi-agent min cost flow problem with controllable capacities. • Efficiency and stability notions. • Optimization problem = Find a Nash equilibrium that maximizes
the network flow. NP-hard in the strong sense.
35 / 37
Conclusion Multi-agent min cost flow Multi-agent min cost flow problem with controllable capacities. • Efficiency and stability notions. • Optimization problem = Find a Nash equilibrium that maximizes
the network flow. NP-hard in the strong sense.
Perspectives • Work in progress Mixed Integer Linear Programming (MILP) in order to find the best Nash equilibrium that maximizes the flow. • Future work Distributed approach to find efficient strategies that are Nash-stable. 36 / 37
Thank you for your attention !
37 / 37
