A Measure of Trading Model Performance with a Risk ... - CiteSeerX

0 downloads 0 Views 97KB Size Report
Oct 1, 1991 - 3.2 and use integral 3.323 of the Gradshteyn and Ryzhik (1980) integral table, the. 1In the case of trading models, this variable is naturally ...
A Measure of Trading Model Performance with a Risk Component

M

ICHEL

¨ M. D ACOROGNA U LRICH A. M ULLER O LIVIERV. PICTET

MMD.1991-05-24 October 1, 1991

A discussion paper by the O&A Research Group

The purpose of this paper is to promote discussions and the exchange of ideas. Comments on the concepts presented are gratefully received.

1 Introduction

The purpose of this paper is to suggest a new measure of trading model performance which accounts for the following requirements: 1. a high total return, 2. a smooth behavior around a straight line, 3. a small clustering of losses, 4. no bias towards low-frequency trading models.

It is important to define a value which describes the performance well in order to minimize risk of over-fitting in the in-sample period and to be able to compare different trading m with each other.

In section 2 of this paper, we discuss the Sharpe index, a measure frequently used evaluate portfolio models. We show that it does not account for all of the above requirem In section 3, we propose a new measure based on a risk-averse trading profile and the util function formalism of Keeney and Raiffa (1976). This measure is numerically more stable the Sharpe index and exhibits fewer deficiencies. In section 4, this measure is extende multi-horizon measure in order to be able to account for the clustering of losses in the curve. In the same section, some numerical aspects of the computation of this variabl discussed.

2 Discussion of the Sharpe index The Sharpe index is a measure of the performance of a portfolio designed to account for risk. The definition of this index is

I



r p r2

(2.1)

2 where I is the index, r is the average returnand r is the variance of the return around the mean, which is computed as follows:

r2

=

N

N

(r 2

1

r 2)

(2.2)

r2 is the average of the squared where N is the number of returns entering the computation, 2 returns and r is the square of the average.

We have applied this measure to our trading models and found that it has several deficie cies. The most important are:

1. The definition of the Sharpe index puts  2 inthe the denominator, which makes the whole index numerically unstable under variations ofto zero. For a return behavior  2 close close to a straight line, the index will explode independent of the positive total achieved. O&A Discussion

1

2. The Sharpe index only measures the risks associated with realized losses in the bookk and neglects the risk due to unrealized losses while the system is in a position. Whe these unrealized losses are not so important for systems with a high dealing freque they can be particularly high for trading systems with a low dealing frequency. If neglect this risk component, we introduce an unfair bias in favor of systems with a dealing frequency.

3. The Sharpe index is unable to consider the clustering of profit trades and loss trade even mixture of profits and losses is usually preferred to clusters of losses and cl of profits, provided the total set of profit and loss trades is the same in both cases Sharpe index ignores the sequence and the distribution of trades over time. 4. The Sharpe index depends on bookkeeping conventions, such as

  

with or without intermediate gearing steps;

with intermediate gearings, the returns can be accounted for with every reductio the absolute gearing or only when the position is fully closed or reversed; bookkeeping per time (for example every Friday) or per trade.

One way of making the Sharpe index more insensitive to point 2 above is to measure the returns at regular intervals such as every month. This is often done when evaluating port performance. Yet, this way of computing the Sharpe index does not eliminate the deficienc listed under 1 and 3 above.

All these details have a distinct, systematic influence on the numerical value of the S index, which always tends to favor low-frequency models, making it unsuitable for the g stated in the introduction to this paper.

3 A measure based on constant risk aversion

Following the Keeney and Raiffa formalism, we define a trading profile with a constant ri aversion independent of the trading model outcome. Such an assumption gives form to th following utility function (see Keeney and Raiffa 1976, example 4.9, page 162):

r(x)

=

u00(x) u0(x)

=

C with C > 0

) u(x) =

e

Cx

(3.1)

1 where x is the stochastic variable of meanx and variance  2 aboutx over the testing interval T . In our empirical cases, x and  2 will always be determined by standard statistics formulas (as in eq. 2.2). The positive C constant gives the level of risk aversion. According to eq. 4.7 of Keeney and Raiffa 1976, the expected utility of such a variable can be computed as



[ ( )] =

Eu x

Z1

u(x)P (x)dx 1

(3.2)

()

whereP x is the probability distribution of the x. If variable we insert a Gaussian distribution 2 N x;  in eq. 3.2 and use integral 3.323 of the Gradshteyn and Ryzhik (1980) integral tabl

(

)

1

In the case of trading models, this variable is naturally related to the returns (see eq. 3.5).

O&A Discussion

2

expected utility becomes

[ ( )] = u(x)e C 

2 2 2

Eu x

(3.3)

()

This form of the expected utility depends on the type of distribution chosen for P x in eq. 3.2. To explore this dependence a bit, we have also used a rectangular distribution around the m With such a distribution, the expected utility has the form

p

sinhC 3 p ux C 3

[ ( )] = ( )

Eu x

(3.4)

2 2 If the risk factor is expanded in both cases, the first two terms areCthe  =same 2 for1 and 4 both cases (Gaussian or rectangular distribution); they only in the o  differ . Yet the third term of the exponential expansion is a better approximation than 0 for eq. 3.4. In the rema of this paper we therefore use eq. 3.3 as expression for the expected utility, keeping in m it depends on the type of distribution considered forxthe . variable

( )

The relevant variable x of a trading model must, of course, be related to theRtotal , return, of the model. We choose to define x as

x(t)

= R~(t)

R~ (t

t);

~( ) = R(t) + ro(t)

where R t

(3.5)

whereR is the total return of the past trades up to t and the ro time is the non-realized return of the current trading model position. t is the time horizon on which this quantity is measured. For a test period T , the variable x takesN T= t values. The choice R of as opposed to ro is due to the continuity property of this quantityrocontrary , which abruptly to changes its value after a trade.





~

= 

A measure of trading model performance comparable to the yearly average return is eas to deduce from the above formalism. Let usxdefine e to be this quantity, which is in units of yearly return but includes a risk term. By inverting eq. 3.1, xe the can variable be formally written as

xe

( [ ( )])

ln E u x

=

(3.6)

C

xe as a for function of the mean return and with the help of eq. 3.3, we get a simple expression x, its variance  2, and the risk aversionClevel : xe

=

x

C 2

(3.7)

2

where  2 is computed with the same formula as given in eq. 2.2 with the exception that th variable r is now replaced xby .



For a particular time horizon t, this is our new measure of trading model performance including the risk. We see that it has the units of an average return and is diminished by a proportionalC to2. This measure, contrary to the Sharpe index, is numerically stable and c differentiate between two trading models with a straight line behavior (  2 0) by choosing the one with the better average return. Moreover, the definition given in eq. 3.1 allocates a s

=

O&A Discussion

3

role to the bookkeeping details of a trade than eq.x2.1 also because contains elements of the current return and is computed at regular time intervals t.

 still depends on the time intervals t. It is hard to compare xe values

The measurexe for different intervals. The usual way to enable comparison across different intervals i annualization, that is, multiplication with the annualization factor:

xe ;ann

1 year x = = 1 year t e t (x

C 2 2

)=

X

C  2 (1 year ) 2 t

(3.8)



whereX is the annualized return in the usual sense, no longer dependent t. In theon second 2 term of the last form of eq. 3.8, we find the  factor 1 year = t, which is exactly the annualized volatility. For Gaussian random walks, it has a constant expectation, as used, for exam the option pricing theory by Black and Scholes. The risk C isaversion thus multiplied by a factor with constant expectation and turns out to be a parameter independent of t. It is thus consistent to assume the same C for different intervals t, and annualized effective returns xe ;ann , computed for different intervals t, can be directly compared.

[ (

)] 







We summarize the main assumptions used for defining this new measure: 1. the variable x is stochastic and follows a Gaussian random walk, 2. the risk aversion is constant with xrespect to .

The first assumption is probably not a bad approximation for any distribution with fin variance, as illustrated by the case of a rectangular distribution. But the formal work it has yet to be done.

4 A multi-horizon measure

In the previous section, we introduced a new measure of trading model performance includi a risk component that relies on the assumption of a constant risk aversion. This measure, t annualized by eq. 3.8, still depends on the timet when horizon x is evaluated (see eq. 3.5) and is insensitive to changes occurring with much longer or much shorter horizons. To reme this problem, we introduce a weighted average xe of computed withn different time ;annthe horizons ti :





Xe

=

Pn

i xe ;ann i=1 wP n w i=1 i

(ti)

(4.1)

wi can be chosen according to the relative importance of the time horizon where the weights ti and may differ for trading models with different trading frequencies. This equation advantage of the fact that annualized xe ;ann values have no systematic dependence on the horizon ti . Substituting xe ;ann by its expression (eq. Xe 3.8), becomes





Xe

=

X

O&A Discussion

1 2

Pn

w 1 year 2 Pn i ti i i=1 wi

i=1 Ci

(4.2)

4

where the variable X is still the average yearly return over the whole Ci issample, the risk aversion constant for each horizon, i2 isand the variance about the mean computed for the time horizonti . It is clearXthat  2 is never negative. e  X because



It is worth noting here that by adopting this new measure we depart from the formal util function theory defined by Keeney and Raiffa. One can easily see from eqs. 3.1 and 3.6 th 2 xe is not directly an additive quantity if the utility . Nevertheless, function is we chose this definition because we do not see each horizon utility function as a component of a meta-ut function but rather as representing a typical segment of the market. If one of these seg 3 endures a bad phase, its influence on the overall outcome need not be overproportional .

In the discussion of eq. 3.8, we showed that the risk aversion Ci has no systematic dependence on the horizonti . However, the traders using a trading model might perceive the risks of different horizons differently. We might calibrate the to their trading preferences, Ci according but having already introduced a weighting function for the different horizons, we do not to complicate the matter further. We introduce a common risk aversion C , parameter, and put a tilde to the weighting parameter wi as it now reflects not only the weight but also a certain risk aversion deviation of the horizon. Thus, the final version of the performance indic obtained:



~

Xe

=

C

X

Pn

~

(1 year) i2

i=1 wi Pn ti i=1 wi

2

~

(4.3)

By empirically balancing risk and return of some test trading models, we found values betw 0.08 and 0.15 to be reasonable C . for

~

The weightswi are determined with a weighting function which allows us to select the relative importance of the different horizons. We have chosen to center it around the t month horizon in order to give sufficient importance to the short horizons in comparison w the long ones. The weighting function used in the computation Xe is of

w~ (t)

=

2

+



1

2

(4.4)

t ln90days

=

with a maximum at t 90days. This weighting function is designed to be applied to horizon ti in a roughly geometric sequence.



The construction of the different sets ofused to measure the performance has been x values designed with the goal of being as simple as possible. Once a testing period (full sa has been established, the longest horizon to be tested is a division by 4 of this entire period. If this division results in a time horizon longer than 2 years, the result is div This is repeated until a horizon strictly shorter than 2 years is reached. We limit this horizon because traders usually close their books after one year and are less sensitive t clustering on longer horizons. The next horizon is obtained by dividing the previous one This is done until a last horizon between 5 and 10 days is reached. This shortest horizon i forced to be 7 days. All horizons are truncated to full days. If there is no integer multi resulting horizon that exactly covers the full sample, the first interval analyzed at th the full sample is extended accordingly. In order to fully include the end of the testing 2

as in Keeney and Raiffa’s theory as would be the case if we kept the formalism of additive utility functions.

3

O&A Discussion

5

ti

594

w~i i X ti Xe ;ann ;ti

0.086 21.92% 36.83% 7.88%

297

0.139 11.80% 18.42% 14.09%

148

74

0.212 7.81% 9.21% 15.15%

37

18

7

0.234 0.171 0.104 0.056 5.37% 3.64% 2.41% 1.47% 4.60% 2.30% 1.12% 0.43% 15.56% 16.12% 16.75% 16.99%

Table 1: Typical results for the performance measure according to each



horizon. The horizons ti are given in days, the weights are normalized to one.

the algorithm computes the first date when to compute thex variable by starting from the end date and going backwards. The resulting sequence of horizons ti is approximately geometric.







As the first analysis interval can be bigger than ti , the definition of the variable ti has to be clarified. For all the above equations, including eq. 4.3, we formally define

ti = nTi

(4.5)



x values used in the computation of horizon i and T is the size of where ni is the number of the full testing interval.

To illustrate these different values, Table 1 shows typical results for a trading mo the German Mark against the US Dollar tested on six and a half years of data (March 1986 t September 1992). The analyzed horizons ti , the weights wi , the standard deviations i , the average returns X ti , and the annualized effectiveX returns e ;ann ;ti (see eq. 3.8) are displayed. The average yearly return of this is Xrun = 22.65% and the effective yearly return (computed according to eq. 4.3 C with0:10) is Xe = 14.91%. The yearly return is reduced by a “risk premium” of about a third of the original value.



~

=

We have one last remark on the sampling x. of In a previous paper (MMD.1990-01-16), we suggested a sampling for the performance indicator with overlapping intervals for each ho so that the long ones would be measured more precisely. Such a solution for the sampling wou still be valid for the new indicator, but it is complicated to implement.

5 REFERENCES

Dacorogna Michel M. , 1990, Summary of the research on simple trading models, internal ref. MMD.1990-01-16.

Gradshteyn I. S., and I. M. Ryzhik , 1980, Table of integrals, series, and products, Acade London.

Keeney Ralph L., and Howard Raiffa , 1976, Decisions with Multiple Objectives: Preferen Value Tradeoffs, John Wiley & Sons New York. O&A Discussion

6

M ¨uller Ulrich A., M. M. Dacorogna, R. B. Olsen, O. V. Pictet, M. Schwarz, and C. Morgenegg, 1990, Statistical Study of Foreign Exchange Rates, Empirical Evidence of a Price Cha Law, and Intraday Analysis, Journal of Banking and Finance, 14, p. 1189-1208.

O&A Discussion

7