Evolutionary Bilevel Optimization - PPSN 2014

Evolutionary Bilevel Optimization A. Sinha, P. Malo and K. Deb Ankur Sinha, PhD Department of Information and Service Economy Aalto University School of Business Helsinki, Finland

14th September 2014, PPSN-2014, Ljubljana, Slovenia

Outline Ø 

Evolutionary Bilevel Optimization (EBO)

Ø 

Genesis

Ø 

Applications

Ø 

Solution Methodologies

Ø 

Results

Ø 

Multiobjective Evolutionary Bilevel Optimization (MEBO)

Ø 

MEBO with Lower Level Decision Uncertainty

Ø 

Robust EBO

Ø 

Conclusions


Single-level Optimization


Bilevel Optimization

A mathematical program that contains an optimization problem in the constraints. -Bracken and McGill (1973)


Bilevel Optimization (cont.)

Multiple Lower Level Optimal Solutions


Multi-level Optimization • 

Multi-level (L levels) optimization •  Two or more levels of optimization •  Nested structure Min F(x1,x2,…xL) s.t

• 

Bilevel: A more generic optimization concept than single-level optimization


Bilevel Optimization: A Genesis An Extension of Mathematical Programming •  All optimization problems are special cases of bilevel programming –Bracken and McGill (1973)

Stackelberg Games •  Bilevel programs commonly appear in game theory when there is a leader and follower –Stackelberg (1952)


Bilevel Problems in Practice Mathematical programming: Functional feasibility •  Stability or equilibrium satisfaction •  Lower level task implements the stability or equilibrium criteria •  Dual problem solving in theoretical optimization Stackelberg games: Leader-follower •  Leader is the first mover •  Leader needs to anticipate followers decisions •  Follower responds optimally to leaders decision


Properties and Challenges l 

Bilevel problems are typically non-convex, disconnected and strongly NP-hard

l 

Solving an optimization problem produces one or more feasible solutions

l 

Multiple global solutions at lower level can induce additional challenges

l 

Two levels can be cooperating or conflicting


Example How does a bilevel optimization problem look?

Min(x,y)

3y + x

Such that

y ∈ argmax(y)

2y

Such that

x + y ≤ 8, x + 4y ≥ 8,

x + 2y ≤ 13, 1 ≤ x ≤ 6

Bilevel Linear Optimization Problem


Example (Cont.) Solution to Bilevel Linear Optimization Problem

y F

f

Bold Line - Induced Set (2,6) - Bilevel Solution

x1

x2

x


Bilevel vs. Multi-objective Bilevel linear optimization problem is now modified as follows:

y

Min(x,y)

3y + x

Min(x,y)

-2y

Such that

Pareto-optimal Set (Decision Space)

Bilevel Optimum

x + y ≤ 8

x + 4y ≥ 8

x + 2y ≤ 13

1 ≤ x ≤ 6

x


Applications


Toll Setting Problem Authority's problem: l  Authority responsible for highway system wants to maximize its revenues earned from toll l  The authority has to solve the highway users optimization problem for all the possible tolls Highway users' problem: l  For any toll chosen by the authority, highway users try to minimize their own travel costs l  A high toll will deter users to take the highway, lowering the revenues

Brotcorne et al. (2001) 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Structural Optimization Upper level: Topology Lower level: Sizes and coordinates


Seller-Buyer Strategies l 

l 

l 

An owner of a company dictates the selling price and supply. He wants to maximize his profit. The buyers look at the product quality, pricing and various other options available to maximize their utility Mixed integer programs on similar lines have been formulated by Heliporn et al. (2010)

Max Profit (suppy, selling price, demand, other variables) Such that demand ∈ argmax { Utility | Lower Level Constraints} Upper Level Constraints

Heilporn et al. (2010) 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Stackelberg Competition Competition between a leader and a follower firm (Duopoly) Leader solves the following optimization problem to maximize its profit

If the leader and follower have similar functions, leader always makes a higher profit. - First mover’s advantage Can be extended to multiple leaders and multiple followers

Sinha, Malo, Frantsev and Deb (2013) 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Stackelberg Competition (Results)

Duopoly: Leader’s objective function in the presence of a follower, the function is well-behaved in the absence of the follower

Oligopoly: Change in leaders’ profits when number of players (leader or follower) change in a market

Sinha, Malo, Frantsev and Deb (2013) 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Forest Management Upper Level: Find an optimal harvesting strategy, i.e. when to harvest and when not! (Binary Variables) Lower Level: For a given strategy, determine the optimal harvesting level, i.e. how much to harvest! (Continuous Variables) Similar to Benders’ Decomposition! Sinha et al. (2014 working paper) 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Forest Management (Results) • 

Task: Optimal management of an uneven aged spruce forest

• 

Problem Class: MINLP

• 

Size: Typically 50 binary variables, 5000 continuous variables

Past Status • 

Computational Requirements: Satisfactory solution obtained in 1 week

Current Status • 

Upper level: A customized GA

• 

Lower Level: SQP with multiple starting points

• 

Computational Requirements: Better solutions obtained in less than 2 hours

Sinha et al. (2014 working paper) 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Parameter Tuning Upper Level: Find optimal parameters that maximize algorithm performance over a number of initial conditions Lower Level: Run the optimization algorithm to find optimized solution

Sinha, Malo, Xu and Deb (2014) 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Parameter Tuning (Results) 1

1

0.9

0.9

BAPT Results

0.8

0.7

0.7

0.6

0.6

CR

CR

BAPT Results 0.8

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0 0

0.2

0.4

0.6

0.8

1 F

1.2

1.4

1.6

1.8

2

Sphere Problem BAPT: Different parameters obtained using BAPT in 21 runs F≈0.4, CR≈0.9

0 0

0.2

0.4

0.6

0.8

1 F

1.2

1.4

1.6

1.8

Schwefel problem BAPT: Different parameters obtained using BAPT in 21 runs F≈0.4, CR≈0.95


2

Ø 

Ø 

A single evalua,on for each parameter vector was suﬃcient to solve the problem Mul,ple evalua,ons at each point increased computa,onal expense with negligible improvement in accuracy

Lower Level Optimization Calls (Normalized)

Parameter Tuning (Results) 1.0

100 90 80

0.8

70 60

0.6

50 40

0.4

30 20

0.2 Accuracy

10 0 0

5

10

15

20

q


0.0

Other Applications l 

l 

l 

l 

Maximizing production in chemical industries where reactions occur under equilibrium Defence applications like, strategic offensive and defensive force structure design, strategic bomber force structure, and allocation of tactical aircraft to missions Other applications include, optimal shape design, optimal operating configuration and optimal facility location Wide applications in economics like Principal-Agency problems, Taxation policies etc.


Solution Methodologies •  KKT conditions of the lower level problem are used as constraints (Herskovits et al. 2000) •  Lagrange multipliers increase the number of decision variables •  Constrained search space •  Applicable to differentiable problems only •  Another common approach: Nested optimization •  For every xu, lower level problem is solved completely •  Computationally very expensive •  Discretization of the lower level problem •  The best solution obtained from discrete set for a given xu is used as a feasible member at upper level


Solution Methodologies (cont.) •  Penalty based approaches •  Special forms of penalty functions have been used •  Lower level is usually required to be convex •  Penalty method requires differentiability •  Branch and Bound techniques (Bard et al. 1982) •  Used KKT conditions •  Handled linear problems •  Taking an approximation of the lower level optimization problem such that its optimum is readily available •  The optimal solutions from lower level might not be accurate


Solution Methodologies (cont.) •  Evolutionary algorithms have also been used for bilevel optimization •  Most of the methods are nested strategies •  Mathieu et al. (1994): LP for lower level and GA for upper level •  Yin (2000): Frank Wolfe Algorithm for lower level •  Oduguwa and Roy (2005): Proposed a co-evolutionary approach •  Wang et al. (2005): •  Solved bilevel problems using a constrained handling scheme in EA •  Method is computationally expensive, but successfully handles a number of test problems •  Li et al. (2006): Nested strategy using PSO •  EA researchers have also tried replacing the lower level problems using KKT (Wang et al. (2008), Li et al. (2007))


Why Evolutionary Approach? Ø 

No implementable mathematical optimality conditions exist (Dempe, Dutta, Mordokhovich, 2007) §  Lower level (LL) problem is replaced with KKT conditions and constraint qualification (CQ) conditions of LL §  Upper level (UL) problem requires KKT of LL, but handling LL-CQ conditions in UL-KKT becomes difficult §  Involves second-order derivatives in UL-KKT conditions

Ø 

Moreover, classical numerical optimization methods require various simplifying assumptions like continuity, differentiability and convexity

Ø 

Most real-world applications do not follow these assumptions

Ø 

EA’s flexible operators, direct use of objectives and population approach help solve BO problems better


Niche of Evolutionary Methods •  • 

Usually, lower level optimal solutions are multi-modal Computationally faster methods possible through metamodeling etc.

• 

Other complexities (robustness, parallel implementation, fixed budget) can be handled efficiently

• 

Many bilevel problems are multi-objective BO •  Both levels require to find and maintain multiple optimal solutions •  EAs are known to be good for these cases


Bilevel Test Problems •  Controlled difficulty in convergence at upper and lower levels •  Controlled difficulty caused by interaction of two levels •  Multiple global solutions at the lower level for any given set of upper level variables •  Clear identification of relationships between lower level optimal solutions and upper level variables •  Scalability to any number of decision variables at upper and lower levels •  Constraints (preferably scalable) at upper and lower levels •  Possibility to have conflict or cooperation at the two levels •  The optimal solution of the bilevel optimization is known


SMD Test Problem Framework (Sinha, Malo & Deb, 2014)

The objectives and variables on both levels are decomposed as follows:


Bilevel Test Probl

Roles of Variables Table 1: Overview of test-problem framework components

Panel A: Decomposition of decision variables Upper-level variables Lower-level variables Vector Purpose Vector Purpose xu1 Complexity on upper-level xl1 Complexity on lower-level xu2 Interaction with lower-level xl2 Interaction with upper-level Panel B: Decomposition of objective functions Upper-level objective function Lower-level objective function Component Purpose Component Purpose F1 (xu1 ) Difficulty in convergence f1 (xu1 , xu2 ) Functional dependence F2 (xl1 ) Conflict / co-operation f2 (xl1 ) Difficulty in convergence F3 (xu2 , xl2 ) Difficulty in interaction f3 (xu2 , xl2 ) Difficulty in interaction

Properties for inducing difficulties: Controlled difficulty in convergence at upper and lower levels. 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Controlling Difficulty for Convergence Ø  Convergence

difficulties can be induced via following

routes: Ø  Dedicated components: F1 (upper) and f2 (lower) Ø  Example:

Quadratic

Multi-modal


Controlling Difficulty in Interactions Ø  Interaction

as follows:

between variables xu2 and xl2 could be chosen

§  Dedicated components: F3 and f3 Ø  Example:


Difficulty due to Conflict/Co-operation Ø Dedicated

components: F2 and f2 or F3 and f3 may be used to induce conflict or cooperation Ø Examples: §  Cooperative interaction = Improvement in lower-level improves upper-level (e.g. F2 = f2) §  Conflicting interaction = Improvement in lower-level worsens upper-level (e.g. F2 = -f2) §  Mixed interaction = Both cooperation and conflict (e.g. F2 = f2 and F3 = Σi (xu2i)2 – f3


Controlled Multimodality Ø  Obtain

multiple lower-level optima for every upper level solution: §  Component used: f2

Ø  Example:

Multimodality at lower-level Induces multiple solutions: x1l1 = x2l1 Gives best UL solution: x1l1 = x2l1=0


Difficulty due to Constraints Constraints are included at both levels with one or more of the following properties: Ø  Ø  Ø 

Ø 

Ø  Ø 

Constraints exist, but are not active at the optimum A subset of constraints, or all the constraints are active at the optimum Upper level constraints are functions of only upper level variables, and lower level constraints are functions of only lower level variables Upper level constraints are functions of upper as well as lower level variables, and lower level constraints are also functions of upper as well as lower level variables Lower level constraints lead to multiple global solutions at the lower level Constraints are scalable at both levels


Test Problems: SMD1 – SMD3 SMD1: •  Interaction: Cooperative •  Lower level: Convex (w.r.t. lower-level variables) •  Upper level: Convex (induced space) SMD 2: •  Interaction: Conflict •  Lower level: Convex (w.r.t. lower-level variables) •  Upper level: Convex (induced space) SMD 3: •  Interaction: Cooperative •  Lower level: Multimodality using Rastrigin’s function •  Upper level: Convex (induced space)


Test Problems: SMD4 – SMD6 SMD 4: •  Interaction: Conflict •  Lower level: Multimodality using Rastrigin’s function •  Upper level: Convex (Induced Space) SMD 5: •  Interaction: Conflict •  Lower level: Complexity with Rosenbrock’s function •  Upper level: Convex (Induced Space) SMD 6: •  Interaction: Conflict •  Lower level: Infinitely many global solutions for any given xu •  Upper level: Convex (Induced Space)


Test Problems: SMD7 – SMD9 SMD 7: •  Interaction: Conflict •  Lower level: Convex (w.r.t. lower-level variables) •  Upper level: Multimodality SMD 8: •  Interaction: Conflict •  Lower level: Complexity with banana function •  Upper level: Multimodality SMD 9: •  Interaction: Conflict •  Lower level: Non-scalable constraints •  Upper level: Non-scalable constraints


Test Problems: SMD10 – SMD12 SMD 10: •  Interaction: Conflict •  Lower level: Scalable constraints •  Upper level: Scalable constraints SMD 11: •  Interaction: Conflict •  Lower level: Non-scalable constraints, multiple global solutions •  Upper level: Scalable constraints SMD 12: •  Interaction: Conflict •  Lower level: Scalable constraints, multiple global solutions •  Upper level: Scalable constraints


4

30

constrained.

at (x u1,x u2) = (2,2)

Interaction: −0.6 Cooperative −0.8 Lower level:−1Convex (w.r.t. lower-level variables) xl2 Upper level:−1.2Convex (induced space)

2

−0.6

5

5.5

8

−0.8 −1 −1.2

x u2

4

SMD1

5

0

4.5

1

20

4.3

−2

4.1

4.5

5.5 oblem where the lower level problem is a convex optimization task, and 11 −1.4 −1.4 −4 100 ex−1.6with respect to upper level variables 500 and optimal lower level variables. −1.6Upper and Lower 4 −4 −2 0 2 4 −0.4 −0.2 0 0.2 0.4 ate with each other. x u1 x l1

contours P: Upper level function contours ! p respecti to upper el variables with F1 = ! (xu1 )2 level variables i=1 at optimal lower q i 2level variables

Q: Lower level function contours with respect to lower level variables at (x u1,x u2) = (2,−2) 500

Function Contours 1.5

15

1 1

1.6

1.5

x l2

0.5 0

1.6

500

F2 = !i=1 (xl1 ) 1.4 0.5 1.4 !r r i 2 i i 2 1.2 1 F3 = ! i=1 (xu2 ) + i=1 (xu2 − tan xl2 ) x 1.2 x l2 (12) l2 1.5 p 1 werf level function for a four-variable SMD1 test problem. i contours 2 1 1.5 1 0.5 0 0.5 1 1.5 1 = !i=1 (xu1 ) x l1 0.8 0.8 q S: Lower level function contours f2 = !i=1 (xil1 )2 0.6 0.6 0.4 0.2 0 0.2 0.4 with respect to lower level variables r 0.4 0.2 0 0.2 0.4 i i 2 x l1 at (x u1,x u2) = (0,0) f3 = i=1 (xu2 − tan xl2 ) x l1 T: Lower level function contours R: Lower level function contours as follows, with respect to lower level variables with respect to lower level variables 100

0.1

100

0.5

11

11

4.3

4.3

15

4.1

4.3

5.5

5

5.5

5

4.1

4.3

4.5

4.5

5

5

5.5

5.5

4

30

xiu1 ∈ [−5, 10], xiu2 ∈ [−5, 10], xil1 ∈ [−5, 10], π xil2 ∈ ( −π 2 , 2 ),

Evolutionary Computation x, Number x at (x u1,x u2) = Volume ( 2,2)

∀ ∀ ∀ ∀

i ∈ {1, 2, . . . , p} i ∈ {1, 2, . . . , r} i ∈ {1, 2, . . . , q} i ∈ {1, 2, . . . , r}

2

0.6 5.5 5

0.8 4.5

x l2

4.3

4.1

4.5

5.5 11 100

500

0.4 0.2

0

x l1

0.2 0.4

1 1.2

x u2

(13)

5

0

4 1

2

20

xl2

0.8 4.5

4.3

4.1

4.5

5.5 11

4

100

500

4

2

0

x u1

2

4

P: Upper level function contours U: Lower function er level variables and lower level optimal variables is level given ascontours follows,

xil1 = 0, ∀ i ∈ {1, 2, . . . , p} xil2 = tan−1 xiu2 , ∀ i ∈ {1, 2, . . . , r}

0.6 5.5

8

1.4 1.6

at (x u1,x u2) = (2,2)

with respect to lower level variables with respect to upper level variables at (x u1,x u2) = ( 2, 2) at optimal lower level variables

0.4 0.2

0

x l1

0.2 0.4

1 1.2 1.4 1.6

Q: Lower level function contours with respect to lower level variables at (x u1,x u2) = (2, 2)

(14)

s at the optima are xu = 0 and xl is obtained by the relationship given 14th September 2014, PPSN-2014, Ljubljana, Slovenia d lower level functions are equal to zero at the optima.

vel optimization task. The lower level optimization problem is once ion task and the upper level optimization is convex with respect to optimal lower level Since, the two levels are conflicting, Interaction: Conflicting milar to the SDM1 test variables. problem, however there is a conflict between the optimum may lead to upper levellevel function value better than the true lower-level variables) level optimization task. The lower optimization problem is once Lower level: Convex (w.r.t. ization task and the upper level optimization is convex respect (induced to roblem. Upper level:with Convex space)

SMD2

and optimal lower level variables. Since, the two levels are conflicting, !p i lead 2 vel optimum may to upper level function value better than the true F1 = ! i=1 (xu1 ) q Upper and Lower el2problem. F =− (xi )2

l1 !r ! i !p (xu2i)2 2− ri=1 (xiu2 F3 = ! i=1 F1 = p ! 2) i=1 i(xu1 f1 = (x ) q i 2 u1 F2! = i=1 (x q− l1 ) ! i=1 i 2 ! r f2 = (xl1 )i )2 − r (xi F3! = i=1 (x i=1 r !p i iu2 2 ii=1 2 u2 f3 = (x − log x ) f1 = i=1 u2u1 ) l2 !i=1 (x i=1

as

f2 = !qi=1 (xil1 )2 r follows, f3 = i=1 (xiu2 −

is

−

log xil2 )2

18

− log xil2 )2

4.5

∈ (0, e], ∀ i ∈ {1, 2, . . . , r}

3.5 3 1.5 2.5

1

2 0.5

18 16

0.1

4.5

1

(15)

12 10

1.5

4.1

0.5

4.5

2 0

x l1

0.2

0.5

1

1.5

0.4

T: Lower level function contours with respect to lower level variables at (x u1,x u2) = ( 2,2) 0.5 0.4

x l2

4

30

x u2

(16)

5

0.2

0

x l1

0.2

0.4

R: Lower level function contours with respect to lower level variables at (x u1,x u2) = (2,2) 0.6 0.5

1

2

(16)

20

0.2

5.5

4

0.4

xl2

5

0

0.2

0.4

2

4

U: Lower level function contours Evolutionary Computation Volume x, NumberP: xUpper level function contours with respect to lower level variables at (x u1,x u2) = ( 2, 2)

with respect to upper level variables at optimal lower level variables

0.3

4.5

0.1

7.5

0.4

0.2

4.1

4.01 5

5.5

5.5

4 2x 0 Evolutionary Computation x Volume x, Number x u1 l1 0.4

2 0.2

0.4

4

5

0

4

0.1

7.5

6

4.5 5.5

2

0.3

4.1

4.01

8

8

5 4.5

10

4.1

0.6

5.5

12

4.01

S: Lower level function contours with respect to lower level variables at (x u1,x u2) = (0,0)

4

5

0

x l1

6

5.5 0.2

1

14

x l2

0.5

8

4.01

20

5

1.5

14

x l2

0.4

4

(15)

16

log xil2 )2

as follows, xiu1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , p} i xu2 i∈ [−5, 1], ∀ i ∈ {1, 2, . . . , r} u1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , p} xil1x∈ i [−5, 10], ∀ i ∈ {1, 2, . . . , q} xu2 ∈ [−5, 1], ∀ i ∈ {1, 2, . . . , r} xil2x∈ i (0, e], ∀ i ∈ {1, 2, . . . , r} ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , q} l1 i xl2

x l2

20

5

Function Contours

0.2

0

0.2

0.4

x l1 Q: Lower level function contours with respect to lower level variables at (x u1,x u2) = (2, 2)


l1

l1

u1

function contours lower level variables (−2,−2)

P: Upper level function contours with respect to upper level variables at optimal lower level variables

Q: Upper level function contours with respect to lower level variables at (x u1,x u2) = (2,−2)

level function contours for a four-variable SMD2 test problem.

Interaction: Cooperative LowerSMD2 level:testMultimodality using Rastrigin’s function pper level function contours for a four-variable problem. pect to upper level variables and optimal levelConvex variables.(induced space) Upperlower level:

SMD3

!p

(xiu1level )2 variables and optimal lower level variables. hFrespect upper 1 = !to i=1 q Upper F2 = !i=1 !p(xil1 )i2 2 ! r F1 =r!i=1i(xu12) i 2 i 2 F3 = q(xu2 i) 2+ i=1 i=1 ((xu2 ) − tan xl2 ) F2! =p!i=1i(xl12) !r r u1 )i 2 f1 =F =i=1 (x i 2 i 2 " (x ) + 3 u2 i=1 ((xu2 ) −%tan xl2 ) # $ !pi=1 ! 2 q 2i (xiu1 " )x f2 =f1q=+ i=1 − cos 2πxil1% l1 i=1 x l2 !r !iq 2 # i $2 i 2 i q +((xu2 cos f3 =f2 =i=1 ) −x tan−xl2 ) 2πxl1 !r i=1i 2 l1 f3 = i=1 ((xu2 ) − tan xil2 )2 25

and Lower Function Contours 1.5 10

x l2(18)

6

5

1 0.1

1.4

9

4.1

6

(18)

s as follows,

les is as follows,

xiu1 ∈ i [−5, 10], ∀ i ∈ {1, 2, . . . , p} x u1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , p} xiu2x∈ i [−5, 10], ∀ i ∈ {1, 2, . . . , r} u2 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , r} xil1 x∈i [−5, 10], q} ∈ [−5, 10], ∀∀ i i∈∈ {1, {1, 2, 2, ......,,q} l1 −ππ π ∀ i ∈ {1, 2, . . . , r} xil2 x∈il2 (∈−π 2( ,2 2, ), 2 ), ∀ i ∈ {1, 2, . . . , r}

1

0.5

0

0.5

x l1

1

0.5 0

1.1

= tan

(xu2 ) , ∀ i ∈ {1, 2, . . . , r}

x l1

1

1.5

1 1.5

1.4 4.1

6

6

6

5

6

1.3

4

1.2

1.5

1

1

2

20

0

x l1

0.5

1 1.5

1

1.5 25

15

1.4

9

0

4

4.1

xl2

6

6

6

5

1.3 1.2

9

1.1

0.5

0.5

R: Lower level function contours with respect to lower level variables at (x u1,x u2) = (2,2)

9

1

1.3 1.2

2

(19)

6

1.1

30

x u2 (19)

6

5

9

8

15 9

x l2

4.1

x l2

1.5

1.5 25

1.4

9


T: Lower level function contours with respect to lower level variables at (x u1,x u2) = ( 2,2)

1.5

x=il1 0, = 0,∀ ∀i ∈ i ∈{1, {1,2,2,. .....,,q} q} i −1 i 2 xl2 = tan −1 (x i u22) , ∀ i ∈ {1, 2, . . . , r}

0.5

15

0.5

10

1.5

0

0.5

1

1.1

4

1 1.5

4 2 en upper level variables andlower lowerlevel level optimal is given as follows, pper level variables and optimalvariables variables is xgiven as follows, l1

xil1 xil2

25

1

1.3 1.2

1

1.5

0

2

2

9

1.5

0.5

2

1

1

1.5

15

6

1

3

U: Lower level function contours with respect to lower level variables at (x u1,x u2) = ( 2, 2)

0

x u1

2

4


(20) (20)

1.5

1

0.5

0

0.5

1

1 1.5


ariables at the optima are xu = 0 and xl is obtained by the relationship given les atand thelower optima arefunctions xu = 0are andequal xl istoobtained byoptima. the relationship pper level zero at the Rastrigin’sgiven and lower level functions are equal to zero at the optima. thRastrigin’s 2 has multiple local optima around the global optimum, which introduces 14 September 2014, PPSN-2014, Ljubljana, Slovenia slties multiple locallevel. optima around the global optimum, which introduces at the lower

Interaction: Conflicting nd K. Deb Lower level: Multimodality using Rastrigin’s function upper level variables and optimal lower level variables. Upper level: Convex (induced Space)

SMD4

!p = ! (xiu1 )2 i=1 pect to upper level variables and optimal lower level variables. q i 2 =− (x ) Upper and Lower l1 i=1 !r ! i 2 !r i i 2 p ) i− 2 =! u2 (xu1 ) F1pi=1 = (x! i=1 (|xu2 | − log(1 + xl2 )) i=1 i q2 i 2 (21) = F2i=1 ) (xl1 ) = (x − u1i=1 " % ! ! x l2 r !q r (x#i i)2$− i i 2 2 (|x 3 =! u2 | i− log(1 + xl2 )) i=1 u2 i=1 = qF+ x − cos 2πx p l1 (21) f1r= i=1 (xiu1l1 )2 ! i=1 " % i! i 2 x # $ 2 l2 q = f i=1 l2 )) = q(|x + u2 | − log(1 xi +−xcos 2πxi

Function Contours 5

5

3

5

6

6

6

5

3

2

10 9

5

4

2

1 2

2

8

1

0.1

1 1

0

2

f3 =

!r i=1i i=1 (|xu2 |

as follows,

l1

− log(1 +

l1

xiu1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , p} i ∈ [−5, ∈ {1, , p} xiu2 ∈xiu1 [−1, 1], 10], ∀ i ∀∈ i{1, 2, .2,. ...,.r} i ∈ {1, 2, . . . , r} u2 ∈ [−1, xil1 ∈x[−5, 10], 1],∀ ∀i ∈ {1, 2, . . . , q} i xl1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , q} i xl2 ∈x[0, i e], ∀ i ∈ {1, 2, . . . , r} l2 ∈ [0, e], ∀ i ∈ {1, 2, . . . , r}

0.5

6

1

0.5

4

6

0

0.5

x l1

T: Lower level function contours with respect to lower level variables at (x u1,x u2) = ( 2,2) 5

6

6

6 5

x l2

6

7

5 4.1

1

0.5

2

20

4

4

6

1

0.5

0

0.5

1

3

U: Lower level function contours with respect to lower level variables (23) at (x u1,x u2) = ( 2, 2)

3

1

5

6

6

6

9

5

5

xl2

10

8 7 6

5

5

4.1 6

0.5

R: Lower level function contours with respect to lower level variables at (x u1,x u2) = (2,2)

0

1

0

x l1

8 4

4

6

5 6

6

6

2

(22)

8

5

6

4

(22) x u2 8

9 5

x l2

1.5

30

7

5

1

10 9

5

0.5


3

1

0

x l1

4.1

6

eenlevel uppervariables level variables lower leveloptimal optimal variables is given per andand lower level variables is given as follows,4 x l1as follows, i = 0, ∀ i ∈ {1, 2, . . . , q} xil1 = x0,l1 ∀ i −1 ∈ {1, 2, . . . , q} i i x = log |x | −1 u2 − 1, ∀ i ∈ {1, 2, . . . , r} l2 xil2 = log |xiu2 | − 1, ∀ i ∈ {1, 2, . . . , r}

1

5

xil2 ))2

ables is as follows,

1.5

6

10

6

6

5

7

5

5

6

2

0

x u1

2

4


(23)

4.1

6

1

0.5

4

6

6

0

0.5

1

3


variables at the optima are xu = 0 and xl is obtained by the relationship given pper level equal at the optima. es at and the lower optima arefunctions xu = 0are and xl to is zero obtained by the relationship given

d lowerthelevel areas equal to zero the optima.bilevel 14thprobSeptember 2014, PPSN-2014, Ljubljana, Slovenia resents samefunctions information in Figure 7 for at a four-variable

•  Recently proposed by Sinha, Malo and Deb (2013) for solving large instances of bilevel optimization problems •  Based on two aspects 1.  Assumption that close upper level members have close lower level optimal solutions 2.  Quadratic approximation of the inducible region •  Method is highly efficient when compared against nested approaches and other contemporary evolutionary techniques •  Tested on a recently proposed SMD test-suit and other standard test problems from the literature

BLEAQ

Bilevel Evolutionary Algorithm Based on Quadratic Approximations


BLEAQ

Aspect 1

Close upper level members are expected to have close lower level optimal solutions


BLEAQ

Aspect 2

Approximation with localization


Results •  • 

Mean (total function evaluations) results for ten bilevel test problems Comparison against the evolutionary algorithm of Wang et al. (2005,2011)

Number of Runs: 21 Savings: Ratio of FE required by nested approach against BLEAQ


Results (cont.) u 

u 

Following are the results for 10 variable instances of the SMD test problems (Sinha et al., 2014) using BLEAQ Comparison performed against nested evolutionary approach

Number of Runs: 21 Savings: Ratio of FE required by nested approach against BLEAQ


xl

SMD1 Quadratic Relationship Convergence 4

Results on SMD1 2 1.5 1 0.5 0 −0.5 True Relation Gen. 1 Gen 150 Gen. 450 Gen. 550 Gen. 610

−1 −1.5 −2 −2

−1.5

−1

−0.5

0 4 xu

0.5

1

1.5

2

Quadratic approximation at optima (0,0) improves with increasing generations

Figure 12: Quadratic relationship convergence. 14th September 2014, PPSN-2014, Ljubljana, Slovenia

SDM6

0.000000

0.000000

2

2

10

10

0

0

F

F0

10

0

10

10 Upper Level Func,on Value

−2

−2

10

−4

10

0

−4

50

100

150

200

250

300

350

400

450

500

10

550

2

0

50

2

10

10

0

0

f0

10

0

10

f

Progress path for the elite member

Convergence Plots on SMD1

−2

−2

10

10

Lower Level Func,on Value −4

−4

10

0

50

100

150

200 250 300 350 Upper Level Generations

400

450

500

550

10

0

50

Figure 10: Convergence plot for SMD1. Figure 11 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Advanced Topics on EBO Multi-objective EBO At least one level has multiple objectives

MEBO with decision-making One or both levels involve a decision maker

Robust EBO Uncertainty in at least one level


Bilevel Multi-objective Problems •  Bilevel problems may involve optimization of multiple objectives at one or both of the levels •  Dempe et al. (2006) developed KKT conditions (difficult to implement) •  Little work has been done in the direction of multi-objective bilevel algorithms (Eichfelder (2007), Deb and Sinha (2010)) •  A general multi-objective bilevel problem may be formulated as follows:


BLEMO (Deb and Sinha, 2009)

" " "

Population structure Lower and upper level NSGA-II Archiving


BLEMO Features: Different from a Pure Nested Procedure •  A population of xu is initialized •  Lower level NSGA-II attempts to move towards different Paretooptimal fronts

•  Based on intermediate information, xu population is updated to move towards feasible and Pareto-optimal region of upper level problem •  Again, lower level NSGA-II attempts to move towards focused areas on lower level Pareto-fronts •  And so on …


MEBO: Example 1 •  • 

Dashed lines represent lower level Pareto fronts Solid line represents upper level Pareto front

•  Lower level Pareto front depends on x •  Upper level Pareto-optimal front lies on constraint G1 •  Maximum two solutions from each x •  Not all x in upper Pareto-optimal front •  Solutions possible even below the upper level Pareto-optimal front

(Eichfelder, 2007) 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Results: Example 1 Nu=400, Tu=200 Nl=40, Tl=40 Near PO solutions found

x

y2

y1


Lower vs Upper Level: Example 1 • 

Identical function evaluations

• 

Large Tl means nested with less importance on upper level

• 

Small Tl means less care on lower level

• 

Tl=40 seems adequate

• 

Refer Sinha and Deb (2009), Deb and Sinha (2010)

Gen = 2

Gen = 3


MEBO: Example 2

•  •  • 

Only one point from a lower level PO front is on upper level PO front Scalable to higher dimensional problems 15 variable version is also solved

(Deb and Sinha, 2009) 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Results: Example 2 Close Pareto-optimal solutions are found by BLEMO

x

y1


Extensions to This Study • 

Difficult test problems suggested (Deb and Sinha, 2009, 2010) •  Allows an adequate evaluation for MEBO algorithms

• 

Self-adaptive and local search based BLEMO suggested (Deb and Sinha, 2010) •  Much faster approach (H-BLEMO) •  No additional parameter

• 

Interaction with the decision maker at upper level leads to substantial savings in function evaluations (Sinha, 2011)


•  Controllable difficulties

•  Scalable in terms of variables and objectives

DS-Test Problems


Applications


Taxation Strategy •  •  • 

Recently, there has been a controversy in Finland for gold mining in the Kuusamo region The region is a famous tourist resort endowed with immense natural beauty For any taxation strategy by the government, the mining company optimizes its own profits

Leader: Government Maximize revenue from taxes Minimize Pollution

Follower: Mining Company Maximize Profit

Sinha, Malo and Deb (2013) 14th September 2014, PPSN-2014, Ljubljana, Slovenia

Results Fig. 4.

Approximate Pareto-frontier for the extended model taking all technologies into account.

Fig. 5. Approximate Pareto-frontiers for the individual technologie considered one at a time.

Taxation Strategy (Results)

22

0

42

20

a=2

−200

2650.0