Expected Utility Maximization: the dual approach

Expected Utility Maximization: the dual approach Sara Biagini February 4, 2008

1

Introduction

Utility maximization has a long tradition in Mathematical Finance, dating back to the 1950s. It was introduced by Tobin [17] in portfolio choice to provide a theoretical foundation to the meanvariance selection of the target investment strategy by an investor or fund manager. The idea is to suppose that the agent is rational, acting in her own best interest and risk-averse. This attitude plus a specification of the risk aversion is then captured by a concave function, the utility U of the agent (see EQF03/007: Utility function). The utility function in Tobin’s work was quadratic, i.e. of the form U (x) = bx − ax2 , a > 0, tailor made for the mean-variance optimization. In the subsequent literature however, it is always assumed that U is increasing since the agent prefers more money to less. Given her U , the agent chooses the portfolio P ∗ that realizes the maximum utility over the trading horizon [0, T ]. As future positions are random, the selected P ∗ in fact maximizes the agent’s expected utility over [0, T ]. The first continuous time financial market models to be proposed were complete (see EQF04/006: Complete-incomplete markets) so that utility maximization in continuous time was initially applied to portfolio selection in this framework. Famous cases study are those considered in Merton [11], [12] where the agent is planning for retirement in a financial market where the short rate is constant and the risky assets are modelled as a diffusion with constant coefficients. In complete markets every conceivable risk can be hedged off by cleverly investing in the market. So, the price of any claim is uniquely assigned since by no-arbitrage it must coincide with the initial value of the hedging portfolio. In the more realistic situation of incomplete markets, the valuation and the hedging problems become highly non-trivial issues. Due to some factors, e.g. intrinsic sources of risk from non traded assets, there are claims that cannot be perfectly hedged. In general, there is a whole interval of prices for a claim that are consistent with no-arbitrage. In addition to the portfolio selection problem, expected utility maximization has turned out to perform very well also in the pricing and hedging problems in incomplete markets. In fact the optimal investment P ∗ determines a special pricing probability measure Q∗ via the derivative of U , i.e. the marginal utility U 0 . Thus an arbitrage- free price p can be assigned to any claim B in a linear way by taking expected values under the probability Q∗ : p(B) = EQ∗ [B]. So, this pricing technique turns out to be an application of the pricing by marginal utility principle, introduced in the option pricing context by Davis [5]. There is another utility based pricing method, which relies on a slightly different maximization problem where the claim to be priced shows up. The

1

resulting price is known as indifference price (see EQF04/011: Indifference pricing) and it is non linear in the number of units of the claim. The use of increasingly more complex probabilistic models of financial assets has continued to pose new challenges to the resolution of the utility maximization problem itself. The mathematical approaches may be quite different, depending on the probabilistic assumptions on the financial market model considered. If the setup is that of general (non-Markovian diffusion or semimartingale) models, one cannot use methods from stochastic optimal control as originally done by Merton and by many others after him. In the middle of the 1980s, with the works of Pliska [13], Karatzas et al. [7] and Cox and Huang [4] a new powerful methodology started to develop, namely the duality approach. This approach relies on convex duality and martingale methods, thus enabling the treatment of the most general cases. The price to pay for the achieved generality is that the results obtained have a mathematical existence-uniqueness-characterization form. As always, explicit calculations require the specification of a (very) tractable model. The presentation given here is based on the dual approach, in a general semimartingale model. For a treatment of the same problem with martingale methods in a diffusion context, see EQF14/008: Expected utility maximization or the book [8].

2

The various maximization problems

The agent is a price taker, i.e. her actions do not affect market prices, and her goal is to (dynamically) optimally invest up to the terminal time T , in order to achieve the maximum utility. A host of features can be taken into account, such as the initial endowment, the possibility of consumption and the presence of a random endowment at time T . Below there is a list of various situations. The mathematical details are left for the next section. 1. Utility maximization from terminal wealth. The preferences of the investor are represented by a von Neumann-Morgenstern utility function U : R → [−∞, +∞) which must be not identical to −∞, increasing and concave. 1 α αx

Typical examples are U (x) = ln x, U (x) =

with α < 1, α 6= 0, where it is intended that

U (x) = −∞ outside the domain, and U (x) = − γ1 e−γx with γ > 0. No consumption occurs before time T . The agent has the initial endowment x and can invest in the financial market. The resulting optimization problem is sup E[U (k)]

(1)

k∈K(x)

where K(x) is the set of random wealths that can be obtained at time T (terminal wealths) with initial wealth x. The formulation of the problem with random endowment, namely when the agent receives at T an additional cashflow B (say, an option), is the following sup E[U (k + B)] k∈K(x)

2

(2)

as her terminal possible wealths now are of the form k + B. 2. Utility maximization from consumption. Suppose that the agent is not particularly interested in consumption at the terminal time T , but rather she is willing to consume over the entire planning horizon. A consumption plan C for the agent is determined by its random rate of consumption c(t) at time t for all t ∈ [0, T ]. It is evident from the financial meaning that the rate c(t) must be nonnegative, so the consumption in the interval [t, t + dt] increases of the quantity c(t)dt. The goal of the agent is thus the selection of the best consumption plan over [0, T ], starting with an initial endowment x ≥ 0. The utility function will now measure the degree of satisfaction with the intertemporal consumption, or better with the rate of consumption. As this measure may change over the time, the utility depends also on the time parameter U : [0, T ] × R → [−∞, +∞) When t is fixed, then U (t, ·) is an utility function with the same properties as in item 1 above. As the rate of consumption cannot be negative, U (t, x) = −∞ when x < 0. The agent may clearly benefit from the opportunity of investing in the financial market, so in general her position can be expressed by a consumption plan C and a dynamically changing portfolio P . If X C,P (t) is the total wealth of the position (C, P ) at time t, then as there is no external infusion of money the variation of the wealth in [t, t + dt] must satisfy dX(t) = −c(t)dt + dV P (t) where dV P (t) is the variation of the value of the portfolio P at time t due to market fluctuations. Let A(x) indicate the set of all such consumption plans - portfolios (C, P ) when starting from the wealth level x. The maximization is then that of the expected integrated utility from the rate of consumption "Z sup

E

(C,P )∈A(x)

#

T

U (t, c(t))dt 0

3. Utility maximization from terminal wealth and consumption. Alternatively, the agent may wish to maximize expected utility from terminal wealth and from intertemporal consumption given her initial wealth x ≥ 0. Therefore, there are two utilities, U and U , from terminal wealth and from the rate of consumption respectively. Let A(x) be the set of the possible consumption plans - portfolios (C, P ), obtained with initial wealth x, and let X C,P (T ) be the terminal wealth from the choice (C, P ). Then the optimal consumptioninvestment is the couple (C ∗ , P ∗ ) that solves "Z T

sup (C,P )∈A(x)

E

#

£ ¡ ¢¤ U (t, c(t))dt + E U X C,P (T )

0

The case selected in the next section for the illustration of the duality technique and the main results is the first, i.e. utility maximization from terminal wealth. When intertemporal consumption is taken into account, similar results can be proved. Also, case 3 turns out to be a superposition of the cases 1 and 2, as shown in the book [8, Chapters 3, 6]. 3

3

Utility maximization from (discounted) terminal wealth

3.1

The setup

An analysis of any optimization problem relies on a precise definition of the domain of optimization and the objective function. Given the financial applications, one should also ensure that the optimization problem is well- defined, i.e. the optimal value is finite. Therefore the study of the maximization (1) requires a specification of 1. the financial market model and the admissible terminal wealths 2. the technical assumptions on U 3. some joint condition on the market model and the utility function 1. The financial market model considered is frictionless and consists of N risky assets and one risk free asset (money market account). Though it is not necessary, for convenience’s sake it is assumed that the risk free asset, S 0 , is constantly equal to 1 that is the prices are discounted. The N risky assets are globally indicated with S = (S 1 , . . . , S N ). The trading can occur continuously in [0, T ]. S = (St )t≤T is in fact an RN -valued, continuous time process, defined on a filtered probability space (Ω, (Ft )t≤T , P ). Since the wealth from an investment in this market is a (stochastic) integral, S is assumed to be a semimartingale so that the object ”integral with respect to S” is mathematically well- defined (see EQF02/13: Stochastic Integration for details). For expository reasons, S is a locally bounded semimartingale. This class of models is already very general, as all the diffusions are locally bounded semimartingales, as well as any jump- diffusion process with bounded jumps. The agent has an initial endowment x and there are no restrictions on the quantities she can buy, sell or sell short. Ht = (Ht1 , . . . HtN ) is the random vector with the number of shares of each risky asset that the agent holds in the infinitesimal interval [t, t + dt]. Bt represents the number of shares of the risk free asset held in the same interval. H = (Ht )t and B = (Bt )t are the corresponding processes and are referred to as the strategy of the agent. To be technically precise, H must be a predictable process and B a semimartingale. As there is no consumption and no infusion of money in the trading period [0, T ], the wealth from a strategy (H, B) is the process X that solves (

or Xt = x +

Rt 0

dXt = (Ht dSt + Bt dSt0 ) = Ht dSt X0 = x

Hs dSs . This can be equivalently stated by saying that the strategy (H, B) is self-

financed. Since dS 0 = 0, the self- financing condition enables a representation of the wealth X in terms of H only. This is the reason why one typically refers to H only as ”the strategy”. As usual in continuous time trading, (see EQF04/002 FTAP) to avoid phenomena like doubling strategies, not every self- financed H is allowed. A self-financed strategy H is said admissible only if during the trading the losses don’t exceed a finite credit line. I.e. H is admissible if there exists some constant c > 0 such

Z

t

for all t ∈ [0, T ],

Hs dSs ≥ −c P − a.s. 0

4

(3)

so that for any x the wealth process X = x +

R

Hs dSs is also bounded from below. Maximizing

expected utility from terminal wealth means in fact maximizing expected utility from the set K(x) RT of those random variables XT that can be represented as XT = x + 0 Ht dSt with H admissible in the sense of (3). Hereafter and in what follows, the notation E[·] indicates P-expectation and when considering expectation under another Q the notation is EQ [·]. Since the market model may be incomplete, there may be more than one pricing measure Q. As shown by [6] the relevant set of pricing measures Me is the set of the equivalent (local) martingale probabilities for S. But we need the less restrictive set M of the absolutely continuous (local) martingale probabilities Q for S as this is the set which will show up in the dual problem. The set M can be characterized in the following way

( M=

"Z

#

T

Q ¿ P | EQ

)

Ht dSt ≤ 0 ∀ adm. H

,

(4)

0

as the set of absolutely continuous probabilities which give nonpositive price to the terminal wealths from admissible self- financed strategies starting with zero wealth. Therefore, given any XT ∈ K(x) and any Q ∈ M,

"

Z

T

EQ [XT ] = EQ x +

# Ht dSt ≤ x

0

2. Hypothesis on U . As a case study, let us assume that U is finite valued on R, i.e. the wealth can become arbitrarily negative (the closest references are [15] and [2]). A typical example to have in mind is the exponential utility. The reason why we prefer the exponential utility (and all the other utilities with the properties listed below) to e.g. the logarithmic or the power utilities is that the dual problem is easier to interpret. References for the case when there are constraints on the wealth (and then U is finite only on a half-line), like U (x) = ln x or U (x) =

1 α αx ,

are [9] and the

notes [16] plus the bibliography there contained. A main difficulty the reader may encounter when comparing this literature is that papers are written in a somewhat different language and style. Very recently, Biagini and Frittelli [3] have proposed a unifying approach that works both for the case of U finite on all R and for the case of U finite only on a half-line. The result there is enabled by the choice of an innovative duality (an Orlicz spaces duality), naturally induced by the utility function U . Coming back to the technical conditions on U , it is required that: • U is strictly concave, strictly increasing and differentiable over (−∞, +∞) • limx↓−∞ U 0 (x) = −∞ and limx→+∞ U 0 (x) = 0 (these are known as Inada condition on the marginal utility U 0 ) Also, U satisfies the Reasonable Asymptotic Elasticity condition RAE(U ) introduced in [9], [15] lim inf

x→−∞

xU 0 (x) xU 0 (x) > 1, lim sup 0

An example is the archetypal couple (U, V ) U (x) = − γ1 e−γx    γ1 (y ln y − y) y > 0  V (y) =

  

0 y=0 +∞

y 0 U (XT ) ≤ XT y

dQ dQ + V (y ) dP dP

and taking P- expectations on both sides E[U (XT )] ≤ xy + E[V (y

dQ )] dP

because E[XT dQ dP ] = EQ [XT ] ≤ x. Therefore, taking the supremum over XT and the infimum over y, u(x) =

sup

E[U (XT )] ≤ inf xy + E[V (y y>0

XT ∈K(x)

dQ )] dP

(6)

As already noted by Merton, the above supremum is not necessarily reached over the restricted set of admissible terminal wealths K(x). Following a well- known procedure in the Calculus of Variations, a relaxation of the primal problem allows to catch the optimal terminal wealth. Here, this means enlarging K(x) slightly and considering the bigger domain of maximization KQ (x) := {k ∈ L1 (Q) | EQ [k] ≤ x} KQ (x) is simply the set of claims that have initial price smaller or equal to the initial endowment x. An application of the Separating Hyperplane Theorem gives that KQ (x) is the norm closure of K(x) − L1+ (Q) in L1 (Q). Then, an approximation argument shows that the optimal expected value u(x) and uQ (x) :=

sup E[U (k)] k∈KQ (x)

are in fact equal. The relaxed maximization problem over KQ (x) is much simpler than the original one over K(x). The replication-with-admissible-strategies issue has been removed and there is just an inequality constraint, given by the pricing measure Q. To find out the value uQ (x) = u(x), one can now apply the traditional Lagrange multiplier method to get uQ (x) =

sup E[U (k)] = k∈KQ (x)

sup

inf {E[U (k)] + y(x − EQ [k])}

k∈L1 (Q) y>0

The dual problem is defined by exchanging the inf and the sup in the above expression: inf

sup {E[U (k)] + y(x − EQ [k])}

y>0 k∈L1 (Q)

(D1)

From [14, Theorem 21] or from a direct computation, the inner sup is actually equal to xy + E[V (y

dQ )] dP

so that the dual problem takes the traditional form ½ ¾ dQ )] inf xy + E[V (y y>0 dP exactly the right hand side of (6).

7

(D)

Thanks to the condition (JC), the dual problem is always finite valued and so is u. A priori © ª however one has only the chain u(x) = uQ (x) ≤ inf y>0 xy + E[V (y dQ dP )] but under the current assumptions it can in fact be proved that there is no duality gap, that is the inequality is an equality ½ ¾ dQ u(x) = uQ (x) = inf xy + E[V (y )] (7) y>0 dP and the infimum is a minimum and the supremum over KQ (x) is reached. In fact, the RAE(U ) condition on the utility function U implies that E[V (y dQ dP )] < +∞ ∀y > 0, so the infimum in (D) can be obtained by differentiation under the expectation sign. The dual minimizer y ∗ (which depends on x) is then the unique solution of x + EQ [V 0 (y

dQ )] = 0 dP

or, equivalently, y ∗ is the unique solution of EQ [I(y

dQ )] = x dP

Therefore, the (unique) optimal claim is k ∗ = I(y ∗ dQ dP ) because it verifies: • the balance equation EQ [k ∗ ] = x, so k ∗ ∈ KQ (x) • the Fenchel equality

dQ dQ + V (y ∗ ) dP dP from which, by taking the P-expectations on both sides U (k ∗ ) = k ∗ y ∗

E[U (k ∗ )] = y ∗ E[k ∗

dQ dQ dQ ] + E[V (y ∗ )] = y ∗ x + E[V (y ∗ )] dP dP dP

which proves the main equality (7). By market completeness the Martingale Representation Theorem applies, so that k ∗ can be obtained via a self- financing strategy H ∗ : Z

T

k∗ = x + 0

Ht∗ dSt

∗

though H is not admissible in general, that is when optimally investing the agent can incur in arbitrarily large losses. Moreover, as a function of x, the optimal value u(x) is also a utility function finite on R, with the same properties of U . The duality equation (7) shows that u and   E[V (y dQ dP )] if y ≥ 0 v(y) =  +∞ otherwise are conjugate functions. Remark. The link between the primal and dual optima can also be expressed as dQ 1 = ∗ U 0 (k ∗ ) dP y So in the complete market pricing with the unique Q coincides with pricing by one’s marginal utility from the optimal investment. 8

3.3

Resolution via the duality method, the incomplete market case.

The same methodology applies to the incomplete market framework, only the technicalities require some more effort. The main results are (more or less intuitive) generalizations of what happens in the complete case, as summarized below (see [2] or [15] for the proofs).

1. The duality relation is the natural generalization of (7): u(x) =

sup

E[U (XT )] =

XT ∈K(x)

inf

y>0,Q∈M

{xy + E[V (y

dQ )]} dP

(8)

and there exists a unique couple of dual minimizers y ∗ , Q∗ . 2. As in the complete case, the supremum of the expected utility on K(x) may be not reached. Define the enlarged set, the generalization of KQ (x), KV (x) = {k | k ∈ L1 (Q), EQ [k] ≤ x∀Q ∈ M with finite entropy } Then, the supremum of the expected utility on KV (x) coincides with the value u(x) and it is a maximum. The claim k ∗ ∈ KV attaining the maximum is unique and the following link between primal and dual optima holds dQ∗ 1 = ∗ U 0 (k ∗ ) dP y 3. Q∗ may be not equivalent to P. But in case Q∗ ∼ P, k ∗ can be obtained through a selffinanced strategy H ∗ , albeit not admissible in general. 4. The optimal value u as a function of the initial endowment x is a utility function, with the same properties of U . In fact, it is finite on R, strictly concave, strictly increasing, it verifies the Inada conditions and RAE(u) holds. The duality relation (8), rewritten as u(x) = inf y>0 {xy + v(y)} with v(y) =

  inf Q∈M E[V (y dQ dP )] if y ≥ 0  +∞ otherwise

shows that u and v are conjugate functions. As Q∗ results from a minimax theorem, it is also known as the minimax measure. There are sufficient conditions that guarantee that Q∗ is equivalent to P, such as: i) U (+∞) = +∞ as noted in [1] or ii) in case U (x) = γ1 e−γx , the existence of a Q ∈ Me with finite relative entropy (see [16] for an extensive bibliography). When Q∗ is indeed equivalent, its selection in the class Me as the pricing measure is economically motivated by its proportionality to the marginal utility from the optimal investment. Utility maximization with random endowment. Under all the conditions above stated (on the market, on U and on both) suppose that the agent has a random endowment B at T , in addition

9

to the initial wealth x. For example B can be the payoff of an European option expiring at T . The agent’s goal is still maximizing expected utility from terminal wealth, which now becomes u(x, B) :=

sup

E[U (B + XT )]

XT ∈K(x)

The duality results in this case are similar to the ones just shown. In fact, ½ ¾ dQ xy + yEQ [B] + E[V ( u(x, B) = min )] y>0,Q∈M dP Note that the maximization without the claim can be seen as a particular case of this, with B = 0: u(x, 0) = u(x). The solution of an utility maximization problem with random endowment is the key step to the indifference pricing technique. The (buyer’s) indifference price of B is in fact the unique price pB that solves u(x − p, B) = u(x, 0) This means that the agent is indifferent, i.e. she has the same (optimal expected) utility, between i) paying pB at time t = 0 and receiving B at T ; ii) not buying the claim B. Related entries Utility function, Expected utility maximization, Indifference pricing, Complete-incomplete markets, Stochastic integration, Fundamental Theorem of asset pricing

References [1] Bellini F. and M. Frittelli: “On the existence of minimax martingale measures”, Math. Fin. 12/1, 1-21 (2002) [2] Biagini S. and M. Frittelli: “Utility maximization in incomplete markets for unbounded processes”. Fin. and Stoch. 9, 493-517, 2005. [3] Biagini S. and M. Frittelli: lems:

“A unified framework for utility maximization prob-

an Orlicz spaces approach”.

Ann. Appl. Prob. Forthcoming, downloadable at

http://www.imstat.org/aap/future papers.html [4] Cox J.C. and C.F. Huang: “Optimal consumption and portfolio policies when asset prices follow a diffusion process”. J. Econ. Th. 49, 33-83. [5] Davis, M.H.A.: “Option pricing in incomplete markets”, Mathematics of Derivative Securities, M. Dempster and S.R. Pliska editors, 216-27. Cambridge University Press (1997) [6] Delbaen F. and W. Schachermayer: “A General Version of The Fundamental Theorem of Asset Pricing ”, Math. Ann. 300, 463-520 (1994) [7] Karatzas I., S. Shreve, J. Lehoczky and G. Xu: “Martingale and duality methods for utility maximization in an incomplete market”, SIAM J. Contr. and Opt., 29, 702-730 (1991) [8] Karatzas I. and S. Shreve: “Methods of Mathematical Finance”, Springer (1998). 10

[9] Kramkov D. and W. Schachermayer: “The asymptotic elasticity of utility function and optimal investment in incomplete markets”, Ann. Appl. Prob. 9/3, 904-950 (1999) [10] Kramkov D. and W. Schachermayer: “Necessary and sufficient conditions in the problem of optimal investment in incomplete markets”, Ann. Appl. Prob. 13/4 1504-1516 (2003) [11] Merton, R.C.: “Lifetime portfolio selection under uncertainty: The continuous-time case”, Rev. Econ. Stat. 51, 247-57 (1969) [12] Merton, R.C.: “Optimum consumption and portfolio rules in a continuous-time model”, J. Econ. Th. 3, 373-413 (1971) [13] Pliska, S.R.: “A stochastic calculus model of continuous trading: optimal portfolios”, Math. Oper. Res. 11, 371-382 (1986) [14] R. T. Rockafellar. Conjugate Duality and Optimization. Conference Board of Math. Sciences Series, SIAM Publications, No. 16, 1974. [15] Schachermayer W.: “Optimal investment in incomplete markets when wealth may become negative”, Ann. Appl. Prob. 11/3, 694-734 (2001) [16] Schachermayer W.: “Portfolio Optimization in Incomplete Financial Markets”, Notes of the Scuola Normale Superiore di Pisa, Cattedra Galileiana (2004) [17] Tobin, J.: “Liquidity preference as behavior towards risk”, Rev. Econ. Stud. 25, 68-85 (1958)

11