Lecture 4: Equality Constrained Optimization

Lecture 4: Equality Constrained Optimization  Tianxi Wang  [email protected]     

2.1 Lagrange Multiplier Technique (a) Classical Programming • max f (x1 , x2 , ..., xn )

→

objective function

where x1 , x2 , ..., xn are instruments/control variables • subject to a constraint g(x1 , x2 , ..., xn ) = b. • The most popular technique to solve this constrained optimization problem is to use the Lagrange multiplier technique. (b) Lagrangean Method • We introduce a new variable λ, called the Lagrange multiplier, and set up the Lagrangean. • For example if we want to maximize f (x1 , x2 ) subject to g(x1 , x2 ) = b, the Lagrange function is L(x1 , x2 , λ) = f (x1 , x2 ) + λ[b − g(x1 , x2 )] • Necessary conditions for a local maximum/minimum requires setting all first-order conditions equal to zero. ∂f ∂g ∂L = −λ =0 ∂x1 ∂x1 ∂x1 ∂L ∂f ∂g = −λ =0 ∂x2 ∂x2 ∂x2 ∂L = b − g(x1 , x2 ) = 0 ∂λ

2

(1) (2) (3)

• The third first-order condition (equation (3)) automatically guarantees the constraint is satisfied. • From the first-order condition we then solve for the critical values x∗1 , x∗2 and λ∗ . Divide equations (1) and (2) to eliminate λ ∂g/∂x1 ∂f /∂x1 = ∂f /∂x2 ∂g/∂x2 and re-arranging equation (3) b = g(x1 , x2 ) we now have two equations to find x∗1 , x∗2 . To compute λ∗ re-arrange equation (1) λ=

∂f /∂x1 ∂g/∂x1

and plug in x∗1 and x∗2 . 1

1

• Example: f = x 2 y 2 s.t. ax + cy = b. 1

1

L = x 2 y 2 + λ[b − ax − cy] F.O.Cs:

1 1 1 Lx = x− 2 y 2 − λa = 0 2 1 1 1 Ly = x 2 y − 2 − λc = 0 2 Lλ = b − ax − cy = 0

(i) (ii) (iii)

Dividing (i) by (ii) yields: y a ax = →y= x c c 3

(iv)

Using (iv) to substitute out y from (iii) yields x∗ x∗ =

b 2a

and substituting x∗ into (iv) yields y ∗ y∗ =

b 2c

To find λ∗ re-arrange (i) and substitute the values obtained for x∗ and y ∗ 1 1 1 1 λ = x∗− 2 y ∗ 2 = 2a 2a

∗



b 2a

− 21  λ∗ =

4

1 b 2 2c 1 1

1

2a 2 c 2

Section 2.2: Interpretation of the Lagrange Multiplier λ

• Since x∗1 , x∗2 and λ∗ are all functions of the constraint parameter b (where b is a fixed exogenous parameter), what happens to these critical values when we change b?

L∗ = f (x∗1 , x∗2 ) + λ∗ [b − g(x∗1 , x∗2 )] • Totally differentiating L∗ with respect to b we find dL∗ dλ∗ dx∗ dx∗ = fx1 1 + fx2 2 + [b − g(x∗1 , x∗2 )] db db db db  ∗ ∗ dx dx 1 ∗ − gx2 2 +λ 1 − gx1 db db Re-arranging: dx∗ dx∗ dL∗ = (fx1 − λ∗ gx1 ) 1 + (fx2 − λ∗ gx2 ) 2 db db db ∗ dλ +[b − g(x∗1 , x∗2 )] + λ∗ db where fx1 ,fx2 ,gx1 and gx2 are all evaluated at the optimum. • Now at the optimum we have fx1 − λ∗ gx1 = 0, fx2 − λ∗ gx2 = 0 and b − g(x∗1 , x∗2 ) = 0. Thus we obtain:

dL∗ = λ∗ db

• Therefore the Lagrange multiplier tells us the effect of a change in the constraint via parameter b on the optimal value of the objective function f. 5

• Note: for this interpretation of λ∗ the Lagrangean must be formulated where the constraint enters as λ[b − g(x1 , x2 )] and not λ[g(x1 , x2 ) − b].

Section 2.3: Second-Order Conditions for Constrained Optimization

(a) Sufficient conditions for a local max/min • Suppose we have an n variable function f (x1 , x2 , ..., xn ) and one constraint g(x1 , x2 , ..., xn ) = b. • Construct a Bordered Hessian matrix of the Lagrange function where the bordered elements are first-order partial derivatives of the constraint g and the remaining elements are second-order partial derivatives of the Lagrangean function L. 

0 g1 g2   g1 L11 L12  HB =   g2 L21 L22  .. .. .. . . . gn Ln1 Ln2

··· ··· ··· ... ···

 gn  L1n   L2n   ..  .  Lnn

• Note that the all partial derivatives in this matrix are evaluated at the critical values (x∗1 ,x∗2 ,...,x∗n ;λ∗ ) • Check the sign of the leading principal minors.

6

• Sufficient condition for a local max: the bordered Hessian is negative definite. | H1B |< 0, | H2B |> 0, | H3B |< 0, | H4B |> 0 ... where

and so on.

0 g 1 | H1B |= g1 L11

0 g1 g2 B | H2 |= g1 L11 L12 g2 L21 L22

• Sufficient condition for a local min: the bordered Hessian is positive definite. | H1B |< 0, | H2B |< 0, | H3B |< 0... • Example: z = xy s.t. x + y = 6 L = xy + λ[6 − x − y] nec:

Lx = y − λ = 0 Ly = x − λ = 0 Lλ = 6 − x − y = 0 ⇒ x∗ = y ∗ = λ∗ = 3 suff: Lxx = Lyy = 0 Lxy = Lyx = 1 gx = gy = 1 7

Construct the Bordered Hessian matrix of the Lagrange function 0 1 1 B | H |= 1 0 1 1 1 0

Check the sign of leading principal minors: | H1B |= 0 − 1 < 0. | H2B |= −1(0 − 1) + 1(1 − 0) = 2 > 0. Thus we have a local max at x∗ = y ∗ = 3.

(b) A Further Look at the Bordered Hessian • Border Hessian used in constrained optimization problems 0 g1 g2 g1 L11 L12 | H B |= g2 L21 L22 .. .. .. . . . gn Ln1 Ln2

··· ··· ··· ... ···

gn L1n L2n .. . Lnn

• Bordered Hessian used to test quasiconcavity/quasiconvexity 0 f1 | B |= f2 .. . fn

f1

f2 · · ·

f11 f12 f21 f22 .. .. . . fn1 fn2

8

··· ··· ... ···

fn f1n f2n .. . fnn

• Two key differences: (1) Bordered elements in | B | are first-order partial derivatives of the function f rather than g. (2) Remaining elements in | B | are second-order partial derivatives of f rather than the Lagrange function L. • However in the special case of a linear constraint g(x1 , x2 , ..., xn ) = a1 x1 + ..., an xn = m (1) The second-order partial derivatives of Lagrange function L reduces to the second-order partial derivatives of f : Lij = fij (2) The border in | B | is simply that of | H B | multiplied by a positive scalar λ. (3) Therefore: | B |= λ2 | H B | • Under this special case the leading principle minors | Bi |, | HiB | must share the same sign. i.e. if | B | satisfies the sufficient condition for strict quasiconcavity then | H B | must satisfy the secondorder sufficient condition for constrained maximization.

9

Section 2.4: Economic Applications

(a) Utility Maximization and Consumer Demand • U (x, y) s.t. Px x + Py y = M • Standard consumer problem. Consumer must maximize utility subject that she spends all her income M on purchasing two goods x, y, where the prices of both goods are market determined and hence exogenous and we will assume that the marginal-utility functions are continuous and positive i.e. Ux , Uy > 0. L = U (x, y) + λ[M − Px x − Py y] Lx = Ux − λPx = 0 Ly = Uy − λPy = 0 L λ = M − Px x − Py y = 0 • First-order equations imply that the consumer equalizes the ratio of marginal utility to the price for each good: Ux Uy = =λ Px Py

(i)

• Here the Lagrange multiplier can be interpreted as the marginal utility of money when utility is maximized: λ∗ =

∂U ∗ ∂M

• An alternative interpretation of (i) is: −

Px Ux =− Uy Py 10

i.e. Marginal rate of substitution (the slope of the indifference curve) = price ratio (the slope of the budget constraint). • Recalling that an indifference curve is the locus of combinations of x and y that yield a constant level of utility U : dU = Ux dx + Uy dy = 0 dy =0 Ux + Uy dx dy Ux ⇒ =− dx Uy Since Ux , Uy > 0 the slope must be negative. • Re-arranging the budget constraint y=

M Px − x Py Py

y 6

Indifference Curve )

y∗

dy

(slope= dx

E

= − UUxy )

Budget Constraint dy x (slope= dx = − P P ) y

 -

x∗

x

Figure 1: Utility Maximization

11

• Second-order conditions 0 P x Py B | H |= Px Uxx Uxy Py Uxy Uyy | H1B |= −Px2 < 0 | H2B |= −Px2 Uyy − Py2 Uxx + 2Px Py Uxy > 0 Recalling from F.O.C’s that Px =

Ux λ

and Py =

Uy λ .

Therefore:

−Ux2 Uyy − Uy2 Uxx + 2Ux Uy Uxy >0 |= | λ2 | H2B |= Ux2 Uyy + Uy2 Uxx − 2Ux Uy Uxy < 0 f or a local max H2B

• But this is just the condition that the utility function be strictly quasiconcave [For Homework check this!]. • Therefore the tangency point E is a local max when the indifference curve is strictly convex to the origin i.e. when the utility function is strictly quasiconcave. • Note that strict quasiconcavity of the utility function also ensures that the local maximum is unique and globally optimal. Theorem 1 In a constrained maximization problem max f (x) s.t. g(x) = 0 where f and g are increasing functions of x, if: (a) f is strictly quasiconcave and g is quasiconvex, or (b) f is quasiconcave and g is strictly quasiconvex, then a locally optimal solution is unique and also globally optimal.

12

(b) Cost Minimization under Cobb-Douglas Production Technology • min rk + wl s.t. y = k α lβ where α, β > 0. L = rk + wl + λ[y − k α lβ ] • First-order conditions:

Lk = r − λαk α−1 lβ = 0 Ll = w − λβk α lβ−1 = 0 Lλ = y − k α l β = 0 Solve for k ∗ and l∗ to obtain the demand functions for capital and labor. Here λ∗ = marginal cost. • Second-order conditions:

Lkk = −λα(α − 1)k α−2 lβ Lll = −λβ(β − 1)k α lβ−2 Lkl = Llk = −λαβk α−1 lβ−1 gk = αk α−1 lβ gl = βk α lβ−1 Homework: Derive the Bordered Hessian and show that the conditions for a local minimum are satisfied. i.e. | H1B |< 0 and | H2B |< 0, since α,β > 0. • Recall from problem set 1 Q3(b) where we showed that a CobbDouglas production function is strictly quasiconcave for any α, β > 0. Since the objective function is linear and hence quasiconvex, the standard cost minimization problem of the firm under Cobb-Douglas 13

technology generates a local minimum which is unique and is also a global minimum. Theorem 2 In a constrained minimization problem min f (x) s.t. g(x) = 0 (a) if f is strictly quasiconvex and g is quasiconcave, or (b) if f is quasiconvex and g is strictly quasiconcave, then a local minimum is unique and a global minimum.

14