testing of one strategy on one market (onemarket system) produces insucient trades, the same strategy has to betested across many markets for relevance of performance results.
15.2.3 Definitions of market conditionsf Trends: One-directional price moves that can last for months, but includemoves of small magnitude as well. Statistically, trends are serial-correlatedmoves, in which a higher price has a higher probability to be followed byanother price rise again and vice versa. This leads to a series of correlatedclimbing or falling prices. Moving averages are a statistical tool to capturesuch serial correlation. If they (usually two averages) are systematicallypro®table over time a trending price chart is underlying.f Monetary de®nition of trend: The point of interest from a tradingperspective is how long a directional move has to be to qualify as a trend.As traders, we look for a de®nition in terms of dollars and cents. A trendhas the size of a move long enough to allow us to recognize it as directionaland to enter it. On the exit side, we again need the time to recognize theend of the move and to exit it. The trading pro®t from the trendmovements after costs has to be large enough to cover all unpro®tablesmall moves (false breakouts). In a random market, the unpro®table falsebreakouts will kill o all pro®ts of the longer moves. If strategies liketrend-following breakouts or moving-average systems are pro®table in thelong run, we have a certain degree of trending tendency in the market,which is also called black noise.f Black noise: Market behaviour which is partially random and partiallytrending. Black noise is what we recognize as trending movement, likemost of the interest-rate markets. Even the most favourable trendingmarkets do not trend all the time; they are a mix of randomness anddirectional correlated moves which result in black noise. Black noise can bepro®tably traded.
The need for performance evaluation
423
f White noise: This is pure randomness. In a purely random market, we willalways lose money, at least at the same rate of the occurring trading costs.We will experience ®nancial ruin with mathematical certainty in the longrun. Markets have changing and dierent degrees of randomness. Purewhite noise cannot be pro®tably traded. Modern Portfolio Theory taughtat most business schools assumes random markets with a natural distribution. This although most price charts display trends with according fat-taildistributions (similar to a Levy±Pareto distribution).f Pink noise: This is price behaviour in which the direction changes moreoften than randomly. The fast-reversing price direction is typical for rangedriven markets and short-term price action. This reversing-price behaviourcan be illustrated and measured with the parameter distribution resultingfrom an optimization of an inverted-trend-following strategy. Pink noisecan be pro®tably traded within an unknown and limited time frame. Thelag in recognizing the beginning and end of the process constitutes the riskand cost of range trading and limits its practical use virtually to zero. Ihave not yet come across a stable and pro®table range trading strategy.
15.3 PRACTICAL USE OF PERFORMANCE TOOLSAll tests in this study are performed with Trade Station and Excel. Theconcept and ideas do, however, apply to all technical analyses and are notlimited to these programs.15.3.1 Total net profit (NP)Since the performance of an indicator or a trading strategy cannot be reducedto one number, technical analysts look at several performance ®gures toassess risk and return. Of these numbers, total net pro®t is still the mostpopular single ®gure to be optimized. This is not necessarily wrong as long asit does not involve curve ®tting and is not done at the cost of unevenperformance distribution over time. In particular, tests that only show the®nal summary reports can obscure the fact that a strategy resulted in totallosses several times before it showed a pro®t. Even if we examine additionaldata like maximum drawdown, average annual return or the Sharpe ratio, wecannot see the entire picture. All performance numbers measured at the endof a trading or test period can hide the fact that we earned all pro®ts withinone trend, which could have been years ago. There is a need to visualize netpro®t in the form of a chart in order to see the performance data over time.This allows us to observe performance throughout its entire developmentand evaluate the probability of future pro®ts in changing markets. The
424
Advanced Trading Rules
display of single equity curves is available in programs like Trade Station.However, it is better for analysts to custom-build equity curves in aspreadsheet for the following reasons.f To compare the performance and correlations of dierent strategiesf To display many Pro®t and Loss (P & L) curves on the same printoutf To display performance as a percentage of invested capital for comparisonof marketsf To compare trading strategy results in dierent currency denominationsf To add equity curves to market or system baskets and to entire portfoliosf To produce all the necessary performance statistics of combined P & Lcurvesf To produce the raw material for equity-curve tradingf And ®nally, to produce great marketing material.
The ®rst step in producing equity curves involves sending performance datafor each bar of the chart to a ®le. We can do that in the form of an indicatorapplied to the chart. For Trade Station, I have written the following indicatorthat sends all requested performance data to an ASCII ®le, which can beopened and charted in Excel or Lotus. The input of the indicator allows us toenter the initial margin or starting capital for the market. As soon as theindicator is applied to a chart with a trading system, the accrued, dailypercentage return of the initial margin is exported. The resulting graphdisplays accrued percentage returns over time.Indicator to send Performance Date to ASCII ®leInput: InitialMargin(1600);vars:OpenEqu(0), TotalEquity(0), RPP(0),PP(0);{- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -}{This step is only for Trade Station users! It corrects the bug of a wrong open equity function}OpenEqu=(I_OpenEquity-I_ClosedEquity); TotalEquity=I_OpenEquity;{- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -}{Accrued P & L in Percent}{Daily P & L change in Percent}RPP= (TotalEquity/InitialMargin*100);PP=(TotalEquity/TotalEquity[1])/InitialMargin*100;{- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ± - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -}{Sends the accrued and the daily returns to ASCII ®le}Print(®le("c:\Performance\T-Note.txt"),FixDate(date),";",RPP:4:0,";", PP:3:2) ;{- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -}{Plots indicator to screen}Plot1(RPP,"RPP"); Plot2(PP,"PP");
The following function called FixDate has to be used with the indicatorabove to send correct dates after 1/1/2000 from Trade Station to Excel.
The need for performance evaluation
425
Correction ELDate functionInputs: DateSelect(Numeric);Variables: YearPortion(""), StringMonth(""), StringDay("");YearPortion = NumToStr(1900 + IntPortion(DateSelect * .0001), 0);If DateSelect >= 1000000 ThenStringMonth = MidStr(NumToStr(DateSelect, 0), 4, 2)elseStringMonth = MidStr(NumToStr(DateSelect, 0), 3, 2);StringDay = RightStr(NumToStr(DateSelect, 0), 2); FixDate=YearPortion + StringMonth + StringDay
(a)(c)(b)
(a)
(b)
(c)
Figure 15.1 Thirty-year T-bond return in % of margin
If we look at the equity curves of the two strategies in Figure 15.1, wesee the typical performance gap of the US 30-year Treasury bond from1986±1991. Most strategies on T-bonds had the same performance dicultiesduring these years. After 1991, we see good results coming from the mediumterm exponential moving average. But the short-term momentum strategynever picked up again, which suggests that it is the wrong strategy forT-bonds.On the US 10-year T-note (Figure 15.2), the two strategies perform inreverse order. The momentum strategy performs better than the exponential(a)(c)(b)
(a)
(b)
Figure 15.2 Ten-year T-note return in % of margin
(c)
426
Advanced Trading Rules
Figure 15.3 Combined T-bond and T-note equity curve return in % of margin
moving average from beginning to end. Interestingly, it also performs duringthe dicult 1986±1991 period. The performance comparison of dierentstrategies during varying market behaviour gives us useful feedback onboth trading strategies and markets. As expected, for a trend-followingstrategy, the performance of the exponential moving average is heavilydependent on a few, strong trends, and it is advisable that it not be usedalone. Although the shorter-term momentum clearly performs better on theT-note, we could choose a position that is split between the strategies toproduce a combined equity curve seeking lower volatility and a better Sharperatio (Figure 15.3).The good news emanating from the combined equity curve is that drawdowns are minimal. The bad news is that the period from 1986±1991 is still anon-performing ¯at-line period. Figure 15.3 demonstrates the followingpoints.f After long periods of non-trending price action, a market can come back totrendsf Most pro®table strategies produce similar equity curves in the same marketf Equity curves of trading systems are good indicators of a market'sunderlying price behaviour.
15.3.2 Drawdown and Max drawdown (DD and MaxDD)Most trading strategies are in a drawdown state from their last equity high upto 70 per cent of the time. It is important to accept this fact psychologically,and it should encourage eorts to diversify as eectively as possible. Drawdowns are the result of the size and frequency of losing trades. The size of
The need for performance evaluation
427
losing trades can be controlled with a stop-loss. But the frequency of losingtrades cannot be easily controlled since it results from the interplay of tradingstrategy logic and the underlying market behaviour, which falls often intorandomness. Max drawdown is the largest historical equity dip and hasbecome one of the most widely used measures of risk. In order to comparedrawdowns of strategies in dierent markets, they should be measured as apercentage of capital, referring to the last equity high as 100 per cent. Mostsoftware packages calculate DD only in reference to the starting capital,which is dangerous and produces overly optimistic risk expectations, resultingin over-trading and ruin with mathematical certainty as time progresses.Equity swings above the initial starting capital ± i.e. from 150 per cent backdown to 120 per cent ± constitute the same risk as a drop at the beginning oftrading. An analyst has to assume that trading can start at any given point intime, including the worst possible moment. This is in fact of someimportance, because we are forced to apply reinvestment, regearing or
Draw Down IndicatorInput: StartEqu(20000), -DDLimit(20);Vars: MyEquity(0),HighEquity(0), DD(0),MxDD(0),PrcDD(0), MxPrcDD(0), MxCount(0), TextNumber1(0),
TextNumber2(0);{- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -}{Calculates Draw Down and Max. Draw Down in percent}MyEquity=StartEqu +I_OpenEquity;if MyEquity>HighEquity then HighEquity=MyEquity;DD=HighEquity MyEquity;If DD>MxDD then MxDD=DD;If HighEquity<>0 then PrcDD=DD/(HighEquity/100);If PrcDD > MxPrcDD then MxPrcDD = PrcDD;{- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -}{Plots indicator and text to screen}Plot1(Round(Neg(PrcDD),1),"Current%DD");Plot2(Round(Neg(MxPrcDD),1),"Mx%DD");Plot3(Neg(DDLimit)," StopLimitDD");If Plot2 crosses under Plot3 then beginValue97=Text_New(Date, Time, High+(C/80), "Close of account");If GetBackGroundColor=1 thenValue98=Text_SetColor(Value97, Tool_Cyan)Else beginIf GetBackGroundColor<>1 then Value99=Text_SetColor(Value97, Tool_Blue);end;end;{- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -}{ Sends draw Down to ASCII ®le}Print(®le("c:\Performance\IMMCHF.txt"),FixDate(date),";",- PrcDD:3:0,";",- MxPrcDD:3:0,";", DDLimit:3:0) ;
428
Advanced Trading Rules
(a)
(c)(b)
Running drawdown in %
(a)
Max drawdown in %
Drawdown limit in %
(b)
(c)
Figure 15.4 Swiss franc IMM: DD with 1 unit traded
money management to our trading strategy. From a risk point of view, eachtime the position size is increased a new trading start is initiated.The above is the formula of an indicator I have written in Trade Station tocalculate the percentage MDD and the daily DD for every bar. It allows us toenter initial starting capital and set a DD limit. If this limit is crossed, an alertis shown.Figure 15.4 is the DD in per cent, of a strategy with one unit traded and noreinvestment or money management. Most tests are performed like this,which gives misleading risk assumptions. The daily DD ((a) in Figure 15.4)reaches its maximum of around 27 per cent in the second year, breakingbelow the chosen DD alert limit (c). Later, it never drops below 20 per centand stays above 15 per cent in the last 5 years. Does this mean that thestrategy improves over time? Of course not. It is the growing capital base thatmakes the drawdown of one unit appear smaller and smaller. In real trading,we are, however, forced to increase trading size with capital growth tomaintain the same percentage returns. Accordingly, we will also maintainthe same DD magnitude due to increased positions as seen in Figure 15.5.This is the same strategy's DD in per cent if money management orreinvestment is applied. The position size is increased according to capitalgrowth, with the result that the chosen DD limit of 20 per cent is consistentlybroken. This demonstrates that DDs remain at the same high level and bearthe risk of producing a marginally higher MaxDD at any time in the future.This demonstrates what many analysts agree on: that money management(position sizing) is one of the most important aspects of trading. Tosummarize, the size of drawdown risk is a function of the following underlying factors.f Market behaviour
The need for performance evaluation
429
(c)(a)
(b)
Running drawdown in %
Max. drawdown in %
(a)
(b)
Drawdown limit in %
(c)
Figure 15.5 Swiss franc IMM: DD with reinvestment
f Methodology of the trading strategyf Size of position.
A change in any of these three factors heightens the risk of higher MaxDDs.If the methodology of a trading strategy cannot be improved any further, andthe MaxDD in testing is still crossing limit levels, then the trading size has tobe decreased until the limit is no longer reached. This tool is not only helpfulin determining the DD risk for any strategy, it can also ®nd the optimalinvestment size and facilitate money management. If we optimize the initialcapital as a function of MaxDD, it tells us how much capital for a giventrading strategy is needed. Of course, there has to be a safety margin,assuming that marginally larger MaxDDs will occur at some point in thefuture.15.3.3 Average profit per trade (AT)In an increasingly competitive trading environment made up of day traders,scalpers and computer-supported traders, pro®t margins have decreased to apoint where trading costs have become a key factor. The average trade is anindicator of the amount of leeway available for commission, slippage ortesting errors, including some degree of unwanted curve ®tting resulting fromtesting. A special warning has to go to the optimization of net pro®t whileneglecting the average trade. The result can be strategies with high tradingfrequency and a very low pro®t per trade. As soon as we lose a little of ouredge in the market or more slippage occurs, these strategies systematicallyresult in losses.The following is the code I have written to send the average trade fromTrade Station to an ASCII ®le. In this example, I have added it to a breakout
430
Advanced Trading Rules
(a)(b)
(a)
(b)
Figure 15.6 Swiss francs (IMM 1980±2000)
system, but of course it can be added to any other strategy. In Figure 15.6, wecan monitor the development of the average trade over its entire history.Send Average Trade to ®leInput: Length(45); vars: TotalEquity(0),Trades(0), AveTr(0);{- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -}{regular Break-out system}IF Close >= Highest(c,Length)[1] Then Buy on Close; IF Close <= Lowest(c,Length)[1] Then Sell on Close;
{- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -}{ Calculates and sends "Average Trade" to ®le}TotalEquity=NetPro®t+OpenPositionPro®t; Trades=TotalTrades+1; AveTr=TotalEquity/Trades;Print(®le("c:\Averagetrade®le\sfr.txt"),FixDate(date),";",AveTr:5:0) ;
Looking at the average pro®t per trade of the Swiss franc from 1980±2000,we see that the trend-following breakout strategy did much better in thebeginning of the 1980s. The strong trends back then gave the trend followersuch a lead in average pro®t per trade that the resulting average is still bettertoday, which is misleading (as shown in Figure 15.7).From 1990±2000, we had a considerably less pronounced trend movement.Looking at the average trade size from 1990±2000, we see that theperformance of the two systems is very similar, and that short-termmomentum trading has become equally good as the trend follower withregard to average trade size. In fact, it has become the superior strategybecause it does not hold trades for as long and has smaller drawdowns. In thisexample, we can demonstrate how the evaluation of trading strategiesdocuments the long-term change in the markets from long, sustained trendsto shorter trends with higher volatility.The resulting AT chart can also be used for money or risk management,with the aim of decreasing the trading size for strategies that fall under a
The need for performance evaluation
431
(a)(b)
Average trade of breakout system
Average trade of momentum system
(a)
(b)
Figure 15.7 Swiss francs (IMM 1990±2000)
minimum ¯oor of average pro®t per trade. The size of this ¯oor must accountfor trading costs and a margin for white noise (randomness) volatility of amarket.15.4 ROBUSTNESS TESTS15.4.1 Parameter selection and distributionAt this point we have to talk about the robustness of performance tests. Alltests are built on the assumption that future performance will be sucientlysimilar to historical performance to allow some degree of generalization.However, in the unstable stationarity (see Glossary at the end of chapter) ofmarket price distributions, we look for additional tests to assess therobustness of our strategies in a changing environment.The ®rst and easiest test is to apply the same trading rule or indicator todierent markets, looking for trading strategies with stable performanceacross markets. Next we test for dierent parameter inputs of variables.Variables are all elements which allow for dierent inputs. For example, aten-day moving average is a variable with the parameter ten. The morevariables or trading rules we apply, the less general and less robust ourstrategy will be. We are talking about a loss in degrees of freedom. Robuststrategies use between one to four variables, which can be changed oroptimized. It is well known that optimization of historical data tunesparameters and rules to past and often singular price behaviour. The resultis curve ®tting with unreliable future performance. Nonetheless, optimizationis a very powerful tool if used correctly. Optimization should be referred to asvisualizing the distribution of parameter performance. We are not interestedin the single most pro®table setting of the past, but the distribution ofpro®table parameters across dierent markets.
432
Advanced Trading Rules
60 000
50 000
40 000
30 000
20 000
10 000
Figure 15.8 C1Day length
Figures 15.8±15.10 display the Net Pro®t (NetPrft) for every parameterlength of a system called C1 Day.Figure 15.8 is a parameter optimization of the number of look-back bars(from 20 to 200) for a breakout strategy on the Euro Bund. We see that allparameters are pro®table especially between the wide range from 50 to 150bars look-back. This re¯ects a very stable strategy on a tradable black noisemarket, which suggests that this trading system can be traded in the futurewith the same parameters between 50 and 150.Figure 15.9 is the same optimization on the gold price from 1990 to 2000which produces a much more unstable distribution with losing parametersfrom 20 up to 100. This result re¯ects a strategy which is unsuitable for thismarket, or a market with too much randomness (white noise). From othertests, I see that gold is in fact a dicult market for most strategies, but has inthe long run tradable trends and could be included in a trading portfolio asdiversi®cation.Figure 15.10 shows a simple breakout system on the British pound from1980±1993. Although the performance for dierent parameters is not verystable, all parameter lengths of the breakout system produced pro®tableresults. This re¯ects the strong trends (black noise) during that period. After
The need for performance evaluation15 000
10 000
5 000
0
5 000
10 000
15 000
Figure 15.9 C1Day length
1.5
1.0
NetPrft
0.5
0.020 0000
40 0000
30 0000
60 0000
50 0000
80 0000
70 0000
100 0000
90 0000
120 0000
110 0000
140 0000
150 0000
Figure 15.10 C1Day length
20 0000
433
434
Advanced Trading Rules
0.0
0:1
0:2
0:3
0:4
0:5
0:6
Figure 15.11 Breakout length
1993, the British pound changed its behaviour from strong trending tovolatile mean reverting.Figure 15.11 shows the same optimization on the British pound from1993±2000. All parameters (from 20 to 150 days) produced deeply negativetrading results. In this case it would even be possible to trade the breakoutsystem inverted, changing the buy signals to sell signals. The market haschanged from a strongly trending to a strongly reversing market. It changesprice direction as soon as we are able to measure the beginning of a trendmove. This frequent reversing within a range is what we describe as pinknoise. This example shows that performance evaluation cannot only producegeneralizations about the behaviour of trading strategies, but also about thebehaviour of the underlying market. The price behaviour of the British poundis, after the year 2000, again interesting. It has changed with the launch ofthe Euro, from the reverting range trading back to trend-following priceaction.
The need for performance evaluationEMA_Len150 60
100
30
40
50
20
4000
4000
2000
2000
NetPrft
0
0
2000
2000
4000
NetPrft
EMA_Len2150
435
4000
600060
600050
50
40
EM
A_
100
30
Le
n1
20
150
2
_Len
EMA
Figure 15.12 Three-dimensional evaluation
Figure 15.12 is an example of an optimization of two moving averages onsilver, producing a three-dimensional evaluation. We not only have a lot ofnegative results, but also a very unstable distribution. Silver has a lot ofrandomness and is accordingly dicult to trade. Most strategies lose moneyin silver, especially trend followers like moving averages. If the randomness insilver is, as I suspected, mostly white noise, then there is no strategy that canbeat this market in the long term.15.4.2 Parameter diversificationAs we see from the results above, there are changing optimal parameters foreach trading system. This is even more pronounced across dierent marketsand dierent years. As a result of this instability of parameter performance,we will never be in a position to continuously trade at an optimum. What weare looking for instead is a spread of robust parameters with a highprobability to produce continuously pro®table results. Since the optimalpeaks are unstable and move around, we are better o using severalparameters diversifying our trading signals.Figure 15.13 is an example of using a combination of moving averages. Thebuy (up) and sell (down) arrows show how the dierent parameters arespreading the signals and with it the risk, across the chart. The resultingperformance ®gures in the left corner of the chart are expectedly good andstable.
436
Advanced Trading Rules
Figure 15.13 A combination of moving averages
15.4.3 Optimization of diversified parametersTo observe the stabilizing eect of diversifying across several parameters, wehave optimized a system with multi-parameter inputs by moving all parameter inputs parallel at the same time in percentage moves up and down. Thefollowing is the formula of a simple breakout strategy with the option of threeinputs (50,100,150) moving together from 90 per cent to 90 per cent in anydesired increment (use odd numbers to avoid a system failure at zero input).
Multi-parameter optimizationInput: Perc(-91);{-99 bis 99}vars:perc2(0), Len1(0), Len2(0), Len3(0);If (50/100*perc)<>0 then begin If (100/100*perc)<>0 then begin If (150/100*perc)<>0 then beginLen1=50+(50/100*perc);end;Len2=100+(100/100*perc);end; Len3=150+(150/100*perc);end;IF CurrentBar > 1 and Close >= Highest(c,Len1)[1] Then Buy("Buy1") close;IF CurrentBar > 1 and Close <= Lowest(c,Len1)[1] Then Sell("Sell1") close;IF CurrentBar > 1 and Close >= Highest(c,Len2)[1] Then Buy("Buy2") close;IF CurrentBar > 1 and Close <= Lowest(c,Len2)[1] Then Sell("Sell2") close;IF CurrentBar > 1 and Close >= Highest(c,Len3)[1] Then Buy ("Buy3") close;IF CurrentBar > 1 and Close <= Lowest(c,Len3)[1] Then Sell("Sell3") close;
The need for performance evaluation
437
250 000
200 000
150 000
100 000
50 000
0
Figure 15.14 Breakout systems
Figure 15.14 is the optimization of the above formula on the Euro Bundover the last 10 years. The relative ¯at and even performance distribution,without performance gaps visualizes the diversi®cation eect across threeparameters very well.The optimization result shows not only a stable return distribution, but inaddition we are also exposed to less risk. Indeed, we are, due to the scaling,not continuously engaged with the full amount of contracts. This results inhigher returns per total initial margin requirement.Figure 15.15 is the equity curve resulting from the use of three parameterstrading three contracts. Despite a good growth rate, the trading speeddiversi®cation (several parameters), cannot avoid the performance stagnationof the last years. This is a sign of changing market behaviour and highlightsagain the fact that equity curves of trading strategies are the best indicator toanalyse market behaviour. This opens an entirely new chapter, which I cannotunfortunately address here. It involves equity curve trading. The use of equitycurves as an analytical tool can de®ne parameter and trading system selectionas well as money and risk management.
438
Advanced Trading RulesEuro Bund traded with 3 parameters
600000
500000
400000
300000
200000
100000
0
12
M
ai-
93
1
N2-
ov
-9
3
12
M
ai-
94
1
N2-
ov
-9
4
12
M
ai-
95
1
N2-
ov
-9
5
12
M
ai-
96
1
N2-
ov
-9
6
12
M
ai-
97
1
N2-
ov
-9
7
12
M
ai-
98
1
N2-
ov
-9
8
12
M
ai-
99
1
N2-
ov
-9
9
12
M
ai-
00
1
N2-
ov
-0
0
Net profit in Euro with 3 contracts traded
Figure 15.15 Euro Bund traded with three parameters.
15.5 CONCLUSIONSValuable analysis results from looking at trading results from as many anglesas possible. The exclusion of one single aspect can dramatically decreasereliability of performance test results. In