LINEAR QUADRATIC OPTIMIZATION FOR MODELS WITH RATIONAL EXPECTATIONS AND LEARNING HANS M. AMMAN AND DAVID A. KENDRICK
Abstract. In this paper we present a method for using rational expectations in a linearquadratic optimization framework with learning. We present a method that allows a policy maker to derive an optimal policy in the presence of rational expectations and the possibility of parameter drift. In this fashion the Lucas critique can be mitigated.
1. Introduction Starting with the work of Kydland and Prescott (1976) and Lucas (1976), much critisism was placed on the use of control theory in economics. One of the major drawbacks of the use of control theory was that it cannot deal with rational expectations (RE). Another drawback is the Lucas critique in which a optimal policy is hampered by parameter changes due to policy anticipation. In this paper we try to lift both drawbacks by introducing a Linear-Quadratic policy framework that allows for RE and learns about parameter changes due to anticipated policy changes. Starting with the paper of Marcet and Sargent (1989), in the last couple of years learning has drawn attention in the macroeconomic literature under the title of Bounded Rationality, Sargent(1993). In this paper we apply similar procedures using a LinearQuadratic policy framework in the presence of rational expectations.
2. Problem statement and solution Following Kendrick (1981), the standard single-agent stochastic linear-quadratic optimization problem is written as: Find the set of admissible instruments X @ ix3 > x4 > ===> xW 4 j that minimizes the welfare loss function + MW @ H
W
OW +{W , .
W [4 w@3
, w
Ow +{w > xw ,
Date: April 11, 1998. Key words and phrases. Macroeconomics, rational expectations, learning, stochastic optimization, numerical experiments. JEL Classification: C63, E61. Corresponding author: Hans M. Amman, Department of Economics and Tinbergen Institute, University of Amsterdam, Roetersstraat 11, Room E1-913, 1018 WB Amsterdam, the Netherlands, Email
[email protected]. 1
2
H.M. AMMAN AND D.A. KENDRICK
with 4 W ,3 Z +{W { , +{W { 5 4 w ,3 Z +{w { w , . +{w { 5 4 w ,3 U+xw x w , . +{w { w ,3 I +xw xw , +xw x 5
OW @ Ow @
subject to the model {w.4 @ D+w ,{w . E+w ,xw . F+w ,}w . w
(1)
The vector {w 5 ?q is the state of the economy at time w and the vector xw 5 ?p contains the policy instruments. The initial state of the economy {3 is known, { w and x w are target values. Z , U and I are penalty matrices of conformable size; w is a white noise vector. The learning is introduced into the LQ framework by the unknown parameter vector 3 which has to be determined through a learning strategy, which we will discuss later on in this section. The above model is straightforward to solve, see Kendrick (1981). However, a serious drawback for economics is that equation (2) does not allow for RE. One way of allowing RE to enter the model is to augment equation (2) in the following fashion
(2a)
{w.4 @ D+w ,{w . E+w ,xw . F+w ,}w .
n [ m @4
Gm +w ,Hw {w.m . w
where the matrix Gm +w , is a parameter matrix, Hw {w.m is the expected state for time w . m as seen from time w, and n is the maximum lead in the expectations formation. In order to compute the admissible set of instruments we have to eliminate the rational expectations from the model. In earlier papers, Amman, Kendrick and Achath (1995) and Amman and Kendrick (1997), we described how the control model with RE can be solved using Sims (1996) approach. In this case we employ this analysis for the situations in which there is learning. For the sake of simplicity let us assume that we have an initial estimate of the parameter vector a3 , which is not the same as the true, but unknown, value of the parameter vector 3 . Also we treat this vector 3 as a constant for the time being. Following Sims (1996) we can rewrite the system equation (2a) in the following augmented form 3 {w.4 @ 4 { w . 5 xw . 6 }w . 7 w
(2)
where
LQ OPTIMIZATION WITH RE AND LEARNING
5 L G4 +a3 , G5 +a3 , 9 L 3 9 9 3 L 3 @ 9 9 .. 7 .
(3)
3 5
D+a3 , 9 3 9 9 4 @ 9 9 3 9 . 7 .. 3
3 L 3 .. . ===
6 Gn4 +a3 , Gn +a3 , : 3 3 : : 3 3 : : 8 3 3 L 3
=== === === .. .
===
=== === .. .
6 3 3: : : 3: : : L8
6 E+a3 , 9 3 : : 9 5 @ 9 . : 7 .. 8 5
6 F+a3 , 9 3 : : 9 6 @ 9 . : 7 .. 8 5
3
3
and the augemented state vector
5
3
{w H{w.4 H{w.5 .. .
3
5 6 L 93: 9 : 7 @ 9 . : 7 .. 8 3
6
9 : 9 : 9 : {w @ 9 : 9 : 7 8 H{w.n4
(4)
Taking the generalized eigenvalues of equation (3) allows us to decompose the system matrices 3 and 4 in the following manner, see Moler and Stewart (1973) and Coleman and Van Loan (1988), @ T3 ]
@ T4 ]
with ] 3 ] @ L and T3 T @ L. The matrices and are upper triangular matrices and the generalized eigen values are ;l $l>l @l>l . If we use the transformation zw @ ] 3 { w we can write equation (3) as zw.4 @ 4 zw . T5 xw . T6 } . T7 w
(5)
Given the triangular structure of and we can partition (4) as follows (6)
44 3
45 55
44 3
45
55
z4>w.4 z5>w.4 z4>w z5>w
@
T4 T4 T4 . x . } . T5 5 w T5 6 w T5 7 w
where the unstable eigenvalues are the in lower right corner, i.e in the matrices 55 and 55 . By forward propagation and taking expectations, it is possible to derive z5>w as a function of future instruments and exogenous variables, Sims (1996, page 5) 4 . Also see Amman and Kendrick (1997), Appendix B.
4
H.M. AMMAN AND D.A. KENDRICK
w @ z5>w @
(7)
4 [ m @3
4 P m 4 55 T5 +5 xw.m . 6 },
with
4 P @ 55 55 Reinserting equation (8) into equation (7) gives us w. 5 xw . 6 }w . 7 w . w w.4 @ z z
(8) with
44 45 44 45
@ @ 3 L 3 3 6 @ T4 6 7 @ T4 7 3 3
T4 5 @ 5 3 3 w @ w
Knowing that { w @ ] 3 zw we can write equation (7) as
{w . Ex w . F }w . w { w.4 @ D
(9) with (10)
3 4 ] D @ ]
@ ] 4 5 E
4 6 F @ ]
4 ]
and (11)
4 4 @ 44 3
4 44 45 L
}w @
}w w
4 7 w w @ ]
We have to make the assumption here that 44 is nonsingular. However, the diagonal elements will generally be nonzero, so it is very likely that the matrix is nonsingular. With equation (9) we have replaced the RE in the control model. E, F depend on the unknown parameter vecAs in equation (1), the matrices D, tor w and we have inserted an initial estimate a3 of this parameter vector in order to be w @ 3. able to solve the RE. The vector w also depends on a3 , so we have to assume that H However, the estimate of aw will change over time as new information becomes available or as a consequence of policy reactions in the economy. So, as soon as we have implemented the first control x3 , we will get a new realisation of the state vector {4 , which enables us to reestimate the parameter vector obtaining 4 . In the literature a number of procedures for such learning processes are described. For instance, ordinary least squares learning, filtering, or stochastic approximations. Here we will apply a stochastic approximation techniques. Following Sargent (1993), our parameter vector can be reestimated by using the following baysian updating scheme for the parameter estimates
LQ OPTIMIZATION WITH RE AND LEARNING
(12) (13)
aw.4 a w.4
@ @
5
4 3a a aw . w w Kw +{w.4 K w w , 3 a w . w +K Kw a w, w
a w is the estimate of the covariance matrix of parameter vector In the above equations w and w is an adjustment parameter; the matrix Kw contains the data that belong to corresponding unknown parameters. Starting with our initial estimates a3 and a3 we can update the parameter vector each time new information on the state of the economy becomes avialable.
If we combine the RE solution method with the learning strategy we get the following algorithm Step 0. Set w @ 3 and compute an estimate of aw and its corresponding covariance a w. matrix Step 1. Set the iteration counter @ 3 and set the instruments xw , w @ i3> 4> = = = > W .v4j and {3 (see below) to an initial value E and F Step 2. Compute w , w @ i3> 4> = = = > W . v 4j and compute D, Step 3. Apply the LQ framework in equations (1)-(1) to compute a new set of optimal instruments xw .4 using the equation below in place of equation (1).
.4 @ Dw { .4 . Ew x .4 . Fw } { w.4 w w w Step 4. Set @ . 4 and go to Step 1 until convergence is reached on the RE part Step 5. Set w @ w . 4 and go to Step 0 if w ?@ W Hence, Steps 0 to 4 outline the method for solving the Linear-Quadractic framework for the RE and a given initial parameter vector estimate a3 , which we have assume constant until now. In the real world this parameter vector is likely to change when we start implementing our optimal policy. Therefore, we have to extend our analysis with a learnining module that updates this parameter vector which we will do in our next section. There is still one remaining issue we have to take care of. In order to apply the algorithm for @ 3 we need to have an initial value of the augmented state vector, which is 5 (14)
{3 H{4 H{5 .. .
6
9 : 9 : 9 : {3 @ 9 : 9 : 7 8 H{n4
However, at the beginning of the first iteration step the expectational variables are not known, so we have to give the expectational part of { 33 an arbitrary value. Once we have
6
H.M. AMMAN AND D.A. KENDRICK
completed the first iterations we can the computed states to reinitialize { 43 . In the next section we will apply this algorithm using a simple macro model. 3. Learning the parameter vector 4. An example In this section we will present two example of the algorithm described in the previous secttion. The first example concerns the learning of an unknown but constant parameter vector. The second example is tries to address the Lucas critique in the sense we will try to learn an unknown parameter vector, which is influenced by value of the policy instruments. Consider a simple macro model with output, {w , consumption, fw , investment, lw , government expenditures, jw , and taxes w . The problem can then be stated as: Find for the for the model (15) (16)
{w.4 fw.4
@ @
fw.4 . lw.4 . jw.4 3=;+{w w , . 533
@ @
3=5Hw {w.5 . 433 . w xw
w.4
@
3=58{w.4
lw.4 jw.4
(17) (18) (19)
with {3 @ 4833, a set of admissible control X @ ix3 > x4 > = = = > x< j to minimize the welfare loss function MW @
(20)
44
4 4[ i+{w 4933,5 . jw5 j +{45 4933,5 . 5 5 w@3
If we reduce the above model to one equation for output we get {w.4 @ 3=9{w . xw . 3=5Hw {w.5 . 633 . w
(21)
which leads to the augmented system (22)
4 4
3=9 3=5 {w.4 @ Hw {w.5 3 3
3 {w 4 633 . xw . }w . w 4 {w.4 3 3 3
where }w @ 4; w. Lets assume that we do not know the value of the parameter values 3=9 and 3=5, which capture transition effect and the effect of the RE in the model. Hence, we assume that we have a bad initial estimate (23)
3=8 a3 @ 3=4
and lets assume further that it has a variance estimate (24)
3=6 a 3 @ 3
3 3=4
LQ OPTIMIZATION WITH RE AND LEARNING
7
which are both arbitarily chosen. As mentioned above we will assume that the true parameter vector is @
(25)
3=9 3=5
A case which is theoretical importance is the situation in where the unknown parameter vector is not constant, but influenced by the choice of the policy instruments. Lucas (1976), page 40, argues that discretionary policy is likely to be ineffective due to the fact that the policy instruments influence the the parameter vector w , so in general terms w @ J+xw > xw.4 > ==> xW 4 ,
(26)
Therefore, it is interesting to investigate to what extent our control model is able to learn this effect quickly enough to make discretionary policy effective. To investigate the learning capability of the algorithm we will also assume that there is a regime switch in period 8. For the periods 8 up to 45 we will assume that the true parameter vector is
3=:3 @ 3=58
(27)
CHA: We could also use a rule here. By apply the QZ factorization, Coleman and van Loan (1988), we can compute the the QZ decomposition (28)
+a3 , @
(29)
]+a3 , @
4=54;4 3=:53; 3 3=3;54
3=;;77 3=799; 3=799; 3=;;77
+a3 , @
T+a3 , @
3=9763 3=7;48 3 3=:::9
3=9;:: 3=:593 3=:593 3=9;::
so the eigenvalues are i3=9763@4=54;4> 3=:::9@3=3;54j @ i3=85:
