Encoding Trader 'Horse-Sense':

Stanford KSL

Encoding Trader ‘Horse-Sense’

Encoding Trader ‘Horse-Sense’: Experiments using Historical Foreign Exchange Data

Selene Makarios - Stanford Knowledge Systems Lab

Stanford KSL


Introduction • horse-sense colloq. (orig. U.S.)

“A coarse, robust, and conspicuous form of shrewdness.” • Early reference: 1870 Nation (N.Y.) 18 Aug. 105 “The new phrase – born in the West, we believe – of ‘horsesense’, which is applied to the intellectual ability of men who exceed others in practical wisdom.” (OED)

2

Stanford KSL


Introduction • A classical (pre-connectionist, pre-Bayesian network) artificial

intelligence paradigm: – Logical or algorithmic encoding of heuristic procedures. – Incorporation into computational processes.

3 3

Stanford KSL


Introduction • Trade with the trend. • Cut losses short. • Let gains run.

These three trading dictums, which we characterize as “horsesense” trading principles, are often found explicated in lay-audience treatments of trading.

4

Stanford KSL


Introduction Tools and Tactics for the Master Day Trader – Oliver L. Velez, Greg Capra Trader Vic–Methods of a Wall Street Master – Victor Sperandeo and T. Sullivan Brown The Four Cardinal Principles of Trading – Bruce Babcock Lessons from the Greatest Stock Traders of All Time – John Bok How to Trade Stocks – Jesse Livermore

5

Stanford KSL


Introduction • What do they mean? • Can they can be gainfully incorporated into computational trad-

ing? •

6 6

Stanford KSL


Spoiler • We find some evidence that suitable encoding and application

of the three dicta can result in a parameterized trading model in which historically-inferred parameter values yield returns exceeding those of the same strategy with randomized parameters; that is: – A strategy that chooses parameters by inspecting historical data behaves differently vs. – A strategy that ignores the historical data.

7

Stanford KSL


Formulating “Cut Losses Short” • What does this mean?

– Don’t allow an (unrealized) loss on a position to influence a decision to close. – A natural emotional reaction to an unrealized loss is to hope that it will reverse. – In fact, the greater the unrealized loss, the more difficult it is to close the position – converting an unrealized loss to a realized loss is painful. – Maintain a loss threshold, and if it is exceeded, cut the loss short and take the hit.

8

Stanford KSL


Formulating “Let Gains Run” • What does this mean?

– Don’t allow an (unrealized) gain on a position to influence a decision to close. – A natural emotional reaction to an unrealized gain is to fear that it will vanish. – Maintain a gain threshold, and until it is exceeded, hold the position. – If the gain evaporates, so be it.

9

Stanford KSL


Asymmetric Trading – the Trailing Stop

10

Stanford KSL


Asymmetric Trading – the Trailing Stop Sometimes very helpful:

(Note that vertical scale is log!) 11

Stanford KSL


Asymmetric Trading – the Trailing Stop • Use of trailing stop could be regarded as a single mechanism

that captures both sides of: – cut losses short – let gains run in a disciplined way.

12

Stanford KSL


Formulating “The Trend” • Definition of trend here based on heuristics loosely derived from

the writings of Jesse Livermore (the “World’s Greatest Stock Trader”), – Born July 26, 1877, began “bucket shop” trading at age 15, eventually reached sufficient notoriety due to his gains that the bucket shops collaboratively blacklisted him. – Gained $3M by shorting in the crash of 1907. – Lost most of that on one bad cotton future trade by repeatedly adding to a losing position. – Gained $100M shorting in the crash of 1929. – Managed to lose most of that by 1934. – Went bankrupt. – Died ignobly on November 28, 1940. 13

Stanford KSL


Formulating “The Trend” • Define an “upward trend”

– a “peak” is detected, – retreated from, – then surpassed, all within a given time window. • A “peak” is just a net downward change in price by a specified amount, within the time window. • So, “upward trend” = a peak, a dip, then a surpassing recovery. • “Downward trend” is the complement. 14

Stanford KSL


Formulating “The Trend”

15

Stanford KSL


“Agent”-based Approach • Agents (software entities) watch for formation of trends, and open

and close positions accordingly using trailing stops. • Dimensions of agent parameter space: – window size – peak size – trailing-stop size

16

Stanford KSL


Agent Parameter Space • 3-dimensional agent parameter space: – Sliding window length: 0.7 to 25 days, logarithmically spaced, 16 values. – Peak size (minimum dip): 7 to 350 pips, logarithmically spaced, 16 values. – Trailing stop spread: 6 to 300 pips, logarithmically spaced, 16 values.

• Each agent’s behavior is governed by one triple within the pa-

rameter space. • The entire population of agents numbers 16 x 16 x 16 = 4096.

17

Stanford KSL


Stanford KSL


• Experiment: Divide the historical data in half. Call the earlier

half “in sample”, and the later half “out of sample”. – Allow all 4096 agents to perform hypothetical trades on the insample data. – For each agent, record ∗ the highest net gain (gain relative to initial capital), ∗ the lowest net gain, and ∗ the final net gain.

• Each agent gets a score given by the sum of these three fig-

ures. (Why: this (very crudely) correlates with average net gain (from initial capital) of agent during trial.) • The ten highest-scoring agents participate in out-of-sample test.

19

Stanford KSL


The Arbitor • After in-sample run, choose the agents with the ten highest

scores. • Using the out-of-sample data, begin hypothetical trading with these ten agents. • For each agent, the arbitor maintains a sliding average of that agent’s net gain. • At regular intervals, the arbitor chooses the agent with the highest sliding-average net-gain currently (“rotating” strategy). • The arbitor (actually) executes the (hypothetical) trades indicated by the chosen agent, for the duration of the current interval. The hypothetical trades of the other agents are computed but not used by the arbitor. 20

Stanford KSL


The Arbitor • Selecting ten agents from the agent parameter-space amounts

to choosing 30 parameters of a trading model. • If in-sample price-change behavior contains no information relevant to out-of-sample price-change behavior, then a choice of the 30 parameters based on inspection of the in-sample price data should, on average, perform as well as (i.e. not better than) 30 parameters chosen randomly from the parameter-space. • Choose ten arbitrary agents and compare their performnce with the ten highest-scoring agents taken from the insample run.

21

Stanford KSL


Agent Performance on Out-of-sample Data Bse. AUD AUD CAD CHF EUR EUR EUR EUR EUR EUR GBP GBP GBP USD USD USD

Ctr. JPY USD JPY JPY AUD CAD CHF GBP JPY USD CHF JPY USD CAD CHF JPY

Best Rand. 10 Net 10 Net Gain Gain 568 -1822 -1591 -1463 66 -11 -1093 -1513 449 119 -14 -241 -689 -513 -910 -1371 -733 -806 -90 -404 -2437 -4534 398 -1619 501 -1571 -366 -2011 -1498 -3106 -1715 -1645

x x x x x x x x x x x x x

Bse. 2AU x 2AU 2CH 2DE 2DE 2EU x 2EU 2EU 2EU 2GB 2GB 2GB 2GB 2GB 2NZ x 2US

Ctr. JPY USD JPY CHF JPY CHF GBP JPY USD CHF DEM EUR JPY USD USD ATS

Best 10 Net Gain -2486 -2081 -1132 -577 -1100 -576 -1245 -3427 -1852 -6640 -6057 -2545 -711 317 -1007 -1451

Rand. 10 Net Gain -1936 -2367 -4266 -995 -4167 -2272 -900 -2839 -1885 -15984 -14504 -6010 -3134 -5982 -2022 -34846

x x x x x

x x x x x x x x

Bse. x 2US 2US 2US 2US 2US 2US x 2US x 2US 2US 2US 2US 2US 2US 2US 2US

Ctr. BEF CAD CHF DEM DKK ESP FIM FRF HUF ITL JPY NLG NOK SEK SGD

Best 10 Net Gain -16884 -618 -2636 -2705 -302 -3422 -9183 -12515 -9467 -4540 -961 -1757 -1290 -1790 -2341

Rand. 10 Net Gain -98093 -1679 -2606 -2834 -519 -35666 -14966 -7445 -32301 -2185 -5185 254 -56626 -28980 -872

x x x x x x x x x x x x x x x

Performance of best ten agents and of random ten agents on out-of-sample historical data.

22

Stanford KSL


Agent Performance on Out-of-sample Data • Sign Test (non-parametric): performance of best 10

vs. performance of random 10. • Null hypothesis: best-10 and random-10 are equally likely to produce higher gain. • Rejected at 99%+ confidence interval. • p-value is 0.000222

23

Stanford KSL


Price-Chart Discriminator • The computational machinery developed here can be used to

construct a discriminator of historical FOREX price-charts versus randomly generated price-charts. • Random price charts are generated by Wiener processes having drift and volatility the same as those of the historical data. • A total of 36 random price-charts were generated and tested.

24 24

Stanford KSL


Price-Chart Discriminator AUD AUD CAD CHF EUR EUR EUR EUR EUR EUR GBP GBP GBP NZD NZD USD USD USD 2AU 2AU 2CH 2EU 2EU 2EU 2EU 2GB 2GB

JPY USD JPY JPY AUD CAD CHF GBP JPY USD CHF JPY USD JPY USD CAD CHF JPY JPY USD JPY CHF GBP JPY USD CHF JPY

Mean -67.40 -0.3425 -2.3817 -50.105 -2.1876 -0.0421 -0.3288 -0.1705 -66.972 -0.6730 -2.0467 -176.29 -0.6957 -6.9914 -0.1280 -0.6774 -0.7992 -51.370 -425.22 -2.2583 -340.15 -15.713 -2.5231 -869.63 -2.2370 -44.095 -1563.87

Std 100.31 0.4733 2.4979 60.417 2.7625 0.0410 0.4784 0.2432 100.148 0.8392 2.6119 255.08 1.1111 13.8979 0.1791 0.6561 0.8876 74.065 766.72 4.5370 566.86 31.503 5.3610 1536.03 4.1729 57.827 2027.95

Mean/ Std -0.672 -0.724 -0.953 -0.829 -0.792 -1.027 -0.687 -0.701 -0.669 -0.802 -0.784 -0.691 -0.626 -0.503 -0.715 -1.032 -0.900 -0.694 -0.555 -0.498 -0.600 -0.499 -0.471 -0.566 -0.536 -0.763 -0.771

2GB 2NZ 2US 2US 2US 2DE 2DE 2GB 2GB 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2XG 2XU

USD USD CAD CHF JPY CHF JPY DEM EUR ATS BEF CZK DEM DKK ESP FIM FRF HUF ITL NLG NOK SEK SGD ZAR USD USD

Mean/ Mean Std Std -3.3557 6.0669 -0.553 -1.7765 3.4507 -0.515 -2.9311 4.8862 -0.600 -6.1424 10.5563 -0.582 -344.48 636.94 -0.541 -0.1670 0.2749 -0.608 -22.105 40.169 -0.550 -5.3227 6.8148 -0.781 -2.9077 3.9168 -0.742 -30.937 22.050 -1.403 -530.87 293.28 -1.810 -1470.78 735.90 -1.999 -1.7525 2.3737 -0.738 -133.35 107.40 -1.242 -996.67 495.34 -2.012 -10.2551 9.3788 -1.093 -12.6381 12.9553 -0.976 -12016.45 5694.70 -2.110 -21953.75 11412.82 -1.924 -2.5734 3.2189 -0.799 -204.41 165.04 -1.238 -420.31 280.00 -1.501 -3.5250 4.9206 -0.716 -520.28 255.83 -2.034 -357.67 181.08 -1.975 -14162.18 6202.24 -2.283

NOI NOI NOI NOI NOI NOI NOI NOI NOI NOI NOI NOI NOI NOI NOI NOI NOI NOI NNO NNO NNO NNO NNO NNO NNO NNO NNO

ZAJ ZAU ZCJ ZFJ ZEA ZEC ZEF ZEG ZEJ ZEU ZGF ZGJ ZGU ZNJ ZNU ZUC ZUF ZUJ IAJ IAU ICJ IFJ IEA IEC IEF IEG IEJ

Mean -356.26 -0.5010 -12.75 -221.48 -1.6304 -0.0620 -0.2893 -0.1572 -366.82 -0.5641 -1.8955 -565.20 -1.6875 -43.43 -0.2714 -1.1993 -2.3184 -430.31 -108.26 -0.8602 -12.75 -84.66 -2.2451 -0.0172 -0.3634 -0.1944 -157.89

Std 116.50 0.3777 3.9363 77.67 0.7673 0.0403 0.3294 0.1744 106.71 0.4172 0.7282 161.44 0.9969 14.388 0.2124 0.9314 1.0971 113.75 55.120 0.5187 3.9363 49.53 0.8603 0.0229 0.2737 0.2085 81.309

Mean/ Std -3.058 -1.327 -3.238 -2.852 -2.125 -1.540 -0.878 -0.901 -3.437 -1.352 -2.603 -3.501 -1.693 -3.018 -1.278 -1.288 -2.113 -3.783 -1.964 -1.658 -3.238 -1.709 -2.609 -0.753 -1.328 -0.932 -1.942

NNO NNO NNO NNO NNO NNO NNO NNO NNO

IEU IGF IGJ IGU INJ INU IUC IUF IUJ

Mean -1.2006 -1.9282 -271.56 -2.3736 -19.2289 -0.2425 -1.3708 -1.2729 -154.09

Std 0.7808 0.8896 104.79 1.1193 11.423 0.1449 0.8780 0.9275 80.544

Mean/ Std -1.538 -2.168 -2.591 -2.121 -1.683 -1.674 -1.561 -1.372 -1.913

Average agent performance on historical price-series (left) and on random-walk price-series (right). 25

Stanford KSL


Price-Chart Discriminator • Two populations

– average agent gain/loss on historical price-series – average agent gain/loss on randomly generated price-series • Null hypothesis: common mean for the two populations. • Rejected at a 99%+ confidence interval. • Two-sided un-pooled t-test

– p-value is 0.000000006.

26

Stanford KSL


Price-Chart Discriminator

Green: average agent performance on random price-charts. Red: average agent performance on historical price-charts. 27

Stanford KSL


Price-Chart Discriminator • These results could be construed as evidence against the

Efficient Markets Hypothesis. • What the is physical/sociological/scientific justification for the Efficent Markets Hypothesis? • Why should the distribution of changes in a stock price, bond return, commodity price, or FOREX rate be normally distributed and have no hysteresis?

28

Stanford KSL


Price-Chart Discriminator • Brownian motion is derivable from properties of molecular ac-

tivity (Einstein). • How is Brownian motion derived for price-series? • Without derivation from first principles, strong claim (normal distribution and no hysteresis) requires strong evidence. • Lack of well-known means for exploiting price hysteresis does not constitute evidence against non-zero hysteresis.

29 29

Stanford KSL


Relating Gain/Loss to Algorithmic Entropy • Interesting (and serendipitous) observed relation between: – average agent performance on a price-series, and – the ‘amount of information’ (roughly, the algorithmic entropy) in that series.

• Heuristically measuring information content: compression al-

gorithms. – Perfectly compressed information is indistinguishable from noise. – Noise does not compress; likewise, less compressibility corresponds to more information. – Idea: use compressibility as measure of “information content”.

30

Stanford KSL


Relating Gain/Loss to Algorithmic Entropy Bse. AUD AUD CAD CHF EUR EUR EUR EUR EUR EUR GBP GBP GBP NZD NZD USD USD USD 2AU 2AU 2CH 2DE 2DE 2EU 2EU 2EU 2EU

Ctr. JPY USD JPY JPY AUD CAD CHF GBP JPY USD CHF JPY USD JPY USD CAD CHF JPY JPY USD JPY CHF JPY CHF GBP JPY USD

uncomp. (KB) 8431.4 7293.7 251.4 8500.5 8669.8 249.2 6975.2 7473.8 9220.3 9653.0 8629.9 8507.4 8676.4 918.1 1769.1 6263.1 9167.7 9209.3 10260.6 11207.0 10210.6 4875.6 7109.3 11893.2 11932.8 12156.7 12548.1

bz2 (KB) 704.5 606.3 27.5 681.8 934.3 30.3 518.3 479.6 855.9 909.6 949.2 962.1 939.1 98.8 178.0 644.6 975.0 868.6 1294.7 1276.6 1237.2 482.5 787.6 1561.6 1246.3 1625.1 1511.0

gz (KB) 1248.0 1044.7 50.8 1202.7 1606.5 51.9 865.5 807.6 1545.4 1586.9 1623.3 1742.1 1661.6 187.7 315.4 1115.1 1735.3 1580.1 2391.4 2224.2 2291.1 815.8 1490.8 2500.9 2115.2 3054.2 2692.7

deflate (KB) 2884.0 2468.0 112.0 2832.0 3464.0 116.0 2168.0 2132.0 3332.0 3572.0 3496.0 3460.0 3584.0 412.0 728.0 2464.0 3748.0 3444.0 4972.0 5008.0 4820.0 1992.0 3176.0 5664.0 5036.0 6136.0 5940.0

log unc/ bz2 1.078 1.080 0.961 1.096 0.968 0.915 1.129 1.193 1.032 1.026 0.959 0.947 0.966 0.968 0.997 0.988 0.973 1.025 0.899 0.943 0.917 1.005 0.956 0.882 0.981 0.874 0.919

log unc/ gz 0.830 0.844 0.694 0.849 0.732 0.681 0.906 0.966 0.776 0.784 0.726 0.689 0.718 0.689 0.749 0.749 0.723 0.766 0.633 0.702 0.649 0.776 0.678 0.677 0.751 0.600 0.668

log unc/ deflate 0.466 0.471 0.351 0.477 0.398 0.332 0.507 0.545 0.442 0.432 0.392 0.391 0.384 0.348 0.386 0.405 0.388 0.427 0.315 0.350 0.326 0.389 0.350 0.322 0.375 0.297 0.325

Mean/ Std Gain -0.672 -0.724 -0.953 -0.829 -0.792 -1.027 -0.687 -0.701 -0.669 -0.802 -0.784 -0.691 -0.626 -0.503 -0.715 -1.032 -0.900 -0.694 -0.555 -0.498 -0.600 -0.608 -0.550 -0.499 -0.471 -0.566 -0.536

Bse. 2GB 2GB 2GB 2GB 2GB 2NZ 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2US 2XG 2XU

Ctr. CHF DEM EUR JPY USD USD ATS BEF CAD CHF CZK DEM DKK ESP FIM FRF HUF ITL JPY NLG NOK SEK SGD ZAR USD USD

uncomp. (KB) 12590.5 11051.3 10794.8 12443.5 12345.2 9959.6 10210.7 7896.3 10648.7 12365.8 7826.6 10904.7 11568.5 10307.0 10626.6 10576.3 7964.4 8732.5 12801.4 10425.8 11685.0 11710.7 6104.1 5724.5 3499.8 5757.2

bz2 (KB) 2111.9 1824.5 1397.5 1915.7 1640.0 1182.2 2167.3 2057.5 1321.3 1707.9 2637.6 1583.1 2599.6 4742.6 2118.5 2156.5 4229.3 5486.6 1576.4 1576.9 2855.2 2967.5 831.6 1693.6 987.7 3706.3

gz (KB) 3633.2 3222.4 2488.6 3685.7 2950.5 2061.6 3126.0 2061.2 2316.2 3057.4 3769.6 2820.7 4587.8 5144.7 3482.8 3493.8 4699.6 5283.9 2940.4 2777.4 4916.2 5065.8 1399.8 2597.6 1361.9 4054.7

deflate (KB) 7084.0 6224.0 5312.0 6816.0 6168.0 4568.0 5664.0 3876.0 5072.0 6332.0 5924.0 5640.0 7616.0 7744.0 6124.0 6184.0 6640.0 7332.0 6104.0 5480.0 8004.0 8104.0 2968.0 4084.0 2216.0 5460.0

log unc/ bz2 0.775 0.782 0.888 0.813 0.877 0.926 0.673 0.584 0.906 0.860 0.472 0.838 0.648 0.337 0.700 0.691 0.275 0.202 0.910 0.820 0.612 0.596 0.866 0.529 0.549 0.191

log unc/ gz 0.540 0.535 0.637 0.528 0.622 0.684 0.514 0.583 0.663 0.607 0.317 0.587 0.402 0.302 0.484 0.481 0.229 0.218 0.639 0.574 0.376 0.364 0.640 0.343 0.410 0.152

log unc/ deflate 0.250 0.249 0.308 0.261 0.301 0.339 0.256 0.309 0.322 0.291 0.121 0.286 0.182 0.124 0.239 0.233 0.079 0.076 0.322 0.279 0.164 0.160 0.313 0.147 0.198 0.023

Mean/ Std Gain -0.763 -0.781 -0.742 -0.771 -0.553 -0.515 -1.403 -1.810 -0.600 -0.582 -1.999 -0.738 -1.242 -2.012 -1.093 -0.976 -2.110 -1.924 -0.541 -0.799 -1.238 -1.501 -0.716 -2.034 -1.975 -2.283

Historical price-series: compressibility under various compression algorithms, and average agent gain/loss. 31

Stanford KSL


Relating Gain/Loss to Algorithmic Entropy

Log Compression vs. Gain/Loss, Burrows-Wheeler ρ = 0.875.

32

Stanford KSL



Log Compression vs. Gain/Loss, Lempel-Ziv77, ρ = 0.746.

33

Stanford KSL



Log Compression vs. Gain/Loss, DEFLATE, ρ = 0.797.

34

Stanford KSL


Relating Gain/Loss to Algorithmic Entropy • Why? Perhaps because:

– Compression algorithms operate by finding (various sorts of) recurring patterns in the data and exploiting them. – A price signal with no recurring patterns would probably not be profitable, and also would not compress well!

35

Encoding Trader 'Horse-Sense':

Encoding Trader 'Horse-Sense':

Suggest Documents