close of the past 20 days.■ Enter short if today’s close is the lowest close of the past 40 days.■ Exit short if today’s close is the highest close of the past 20 days.■
Similar parameters are used for the moving average crossover:Enter long if the 10-day simple moving average of closes crosses above the40-day simple moving average of closes.■ Enter short if the 10-day simple moving average of closes crosses below the40-day simple moving average of closes.■
The channel breakout and moving average crossover systems are our benchmarks for any new strategy performance. We will calculate the Sharpe ratio andK-ratio of the channel breakout and moving average crossover strategies over aportfolio of markets for comparison to the new ideas we will create in the secondhalf of this book.
PERFORMANCE EVALUATION TEMPLATESReaders need to become accustomed to the performance evaluation templates thatwill be featured frequently throughout this book. Strategy performance will bepresented in two sections using a combination of tables and graphs. The two sections are the Summary page and the Breakdown Statistics page, each of which wewill examine in detail.Summary PageThe name and description of the strategy tested, as well as the type of markets thestrategy has been tested on (futures, stocks, or relative value), is located at the topof the Summary page (Figure 4.15a).
CHAPTER 4 Evaluating Trading Strategy Performance
91
Trading Strategy Evaluation (Futures)
Dec-01
Jan-01
Jan-00
Jan-99
Jan-98
Jan-97
Jan-96
Jan-95
Jan-94
Jan-93
Jan-92
Jan-91
Jan-90
Random entriesNoneA dice is thrown every day. We enter long on 1, and short on 2.1/1/1990-12/31/2001#Avg.Avg. Avg.ofSharpe%Avg.ProfitAvg.Avg. Bars BarsMarket Net Profit K-ratio Ratio Max DD Trades Win Contracts Per Con. WinLoss Win LossFXAD101,940 –0.060.05 –681,180 3625325.571219,564 –21,209 89BP–396,413 –0.02 –0.20 –689,638 4094818.07–5718,419 –18,929 77CD1,460,640 0.22 0.66 –397,550 3725247.618223,971 –17,979 88JY238–0.09 0.00 –1,044,300 3634814.13523,259 –21,748 89SF–650,913 –0.08 –0.32 –696,800 3604915.91–110 19,220 –21,600 89RatesED1,199,750 0.26 0.54 –310,525 3365782.204323,909 –22,964 910TY542,7500.08 0.28 –536,751 3625129.675123,720 –21,419 88US343,8440.05 0.16 –770,094 3535320.794722,073 –22,772 98StockSP–34,7500.05 –0.02 –801,188 391529.04–1018,321 –19,768 78MetalsGC–803,800 –0.06 –0.37 –1,193,520 3814745.04–4719,463 –21,438 78HG–66,513 –0.09 –0.04 –1,086,363 3524932.16–919,291 –19,436 89PL–114,060 0.03 –0.06 –958,115 3544945.92–720,514 –20,272 98SL128,830 –0.02 0.07 –718,445 3995433.50817,671 –20,094 78EnergyCL–51,990 –0.06 –0.03 –704,830 3665129.29–621,256 –22,805 88HO–270,875 –0.02 –0.10 –774,068 3765323.75–2819,690 –23,295 79HU1,036,815 0.05 0.39 –676,507 3995422.3811822,156 –20,388 78GrainsC99,2630.04 0.05 –494,763 3445174.78421,647 –22,373 99S185,2750.08 0.08 –522,800 3675330.441920,690 –21,996 88W–244,613 –0.04 –0.12 –772,713 3865250.24–1319,828 –22,895 78MeatsFC837,1500.10 0.37 –492,015 3765440.565822,854 –21,733 88LC–887,080 –0.13 –0.39 –1,269,816 3585049.77–5018,940 –23,933 89LH240,740 –0.06 0.11 –697,888 3545031.831823,162 –22,276 98PB–444,432 –0.05 –0.19 –609,864 4034721.30–5721,297 –21,085 87SoftsCC–190,800 0.00 –0.10 –725,990 4014950.65–919,033 –19,422 87CT–707,720 –0.10 –0.38 –973,485 3764824.88–7619,238 –21,094 88JO137,0180.04 0.07 –512,348 3935036.011319,274 –18,247 87KC97,8040.08 0.04 –802,706 3764813.151322,419 –20,052 88LB–567,712 –0.11 –0.21 –1,028,712 3574734.22–4527,743 –27,544 89SB160,6750.03 0.08 –573,854 3705354.15618,583 –20,420 88Average 38,0350.00 0.01 –717,227 3604933.57–120,240 –20,639 882,000,0001,500,0001,000,000500,0000–500,000–1,000,000–1,500,000Portfolio StatisticsNet Profit:1,141,060Sharpe ratio:0.09Drawdown:–2,609,091Correlation breakout:–0.11K-ratio:0.02Correlation to 10-40 MA:–0.12Equity
Strategy Name:Parameters :Description :Run Dates :
© 2002 Lars Kestner – All Rights Reserved
FIGURE
4.15a
Sample performance evaluation. These sheets will be commonplace throughout the book.
4.15b
1991
K-ratio.027–0.30–0.030.260.140.43
1992
Sharpe Ratio1.07–1.86–0.500.560.231.10
1993
Year199619971998199920002001
1994
K-ratio–0.16–0.350.110.08–0.17–0.03
Year
1995
1996
Net Profit by Year
Net Profit10,522–581,40435,610204,140–212,371586,720
AverageWin20,88717,42618,32119,23521,03420,72221,56321,048
Average Avg. Bars Avg. BarsLossLossWin88–20,29366–16,789–19,76878–20,31088–22,16388–22,42188–22,25788–21,13088
1997
1998
2000
2001
© 2002 Lars Kestner – All Rights Reserved
1999
Num. ofNumber of Profitable PercentSharpe RatioLength Windows Windows Profitable144750.0152.08%1 Month14274–0.7652.11%3 Months720.0451.80%6 Months139620.1646.62%12 Months13312761–0.3548.03%18 Months121550.6145.45%24 Months
Breakdown by Market SectorAverageAverageAvg. Profit% WinNum TradesPer Contract37350–14263403539152–1037250–14380532836635237350–837949–16
Performance Breakdown by Year
AverageAverageAverageK-ratio Sharpe Ratio Max DD–0.010.04–701,8940.100.24–404,3420.05–0.02–801,188–0.04–0.10–989,111–0.010.09–718,4680.030.00–596,758–767,396–0.03–0.02–0.01–0.08–769,516
Breakdown Statistics (Futures)
Sample performance evaluation. These sheets will be commonplace throughout the book.
FIGURE
–2,000,000
–1,500,000
–1,000,000
–500,000
0
500,000
1,000,000
1990
Net Profit1,267,751–1,491,321–472,399586,055240,809896,329
Year199019911992199319941995
1,500,000
AverageNet Profit103,099521,586–34,750–213,886237,98313,308–63,406–178,456
0001/1/1990 – 12/31/2001
MarketSectorFXRatesStockMetalsEnergyGrainsMeatsSofts
System Name:Parameters:Description:Run Dates:
Net Profit
92
CHAPTER 4 Evaluating Trading Strategy Performance
93
Below the title and description are performance statistics of each individual market:Net Profit: Total profit over the life of the test, including trades still openK-ratio: K-ratio of that market’s equity curve over the life of the testSharpe Ratio: Sharpe ratio of that market’s returns over the life of the testMax DD: Maximum dollar drawdown over the life of the test for that specificmarket.# of Trades: Number of round-trip trades executed over the life of the test% Win: Percentage of trades closed with a profitAvg. Contracts: Average number of contracts or shares executed per tradeAvg. Profit per Contract: Net profit divided by the product of the number of tradesand the average number of contracts or shares per trade; measures the average profit per trade per contract or share tradedAvg. Win: Average dollar profit on winning tradesAvg. Loss: Average dollar profit on losing tradesAvg. Bars Win: Average number of periods (usually days) winning trades were heldAvg. Bars Loss: Average number of periods (usually days) losing trades were heldBelow the individual market statistics is a column labeled “Average,” whichaverages each statistic over all markets. The bottom of the summary page contains a graph of the portfolio equity curve over the life of the test. The portfolioequity curve sums the individual equity curves over all markets tested. Alongwith the portfolio equity curve, the following portfolio performance statistics arealso calculated:Net Profit: Total profit across all marketsDrawdown: Maximum dollar drawdown the portfolio experienced over the life ofthe testK-ratio: K-ratio of the portfolio’s monthly equity curveSharpe ratio: Sharpe ratio of the portfolio’s monthly returnsNote: The K-ratio and Sharpe ratio for the portfolio will typically behigher in absolute magnitude than the average of the markets individually.This occurs due to benefits of diversification, which we will discuss in thenext chapter.Correlation to breakout: Monthly returns from the system are compared withmonthly returns from the 40-day/20-day channel breakout; the correlationamong returns is calculatedCorrelation to MA: Correlation of monthly returns between the system tested andthe 10-day/40-day moving average crossover
94
PART 1 Structural Foundations for Improving Technical Trading
Breakdown Statistics PageThe Breakdown Statistics page groups performance based on market themes:Breakdown by Market Sector: An average of performance statistics across a sectorof markets grouped by commodity type or stock industry. This allows forquick and easy analysis by market sector.Performance Breakdown by Year: This table displays net profit, K-ratios, andSharpe ratios of the portfolio for each year individually. These results allowus to judge the consistency of a strategy over time.Profitability Windows: When performance is broken into time intervals, it’s usuallydone by looking at calendar year returns. Why, though, do we use January-toDecember returns instead of July-June returns? Or more specifically, why 12months instead of six? Profitability windows break performance into tightermeasurement intervals. For example, a three-month profitability window beginsby summing the returns of months one, two, and three. The window is slid forward one month, and the net profit of months two, three, and four are summed.This number is recorded, and again the window moves one month forward. Thisprocess continues until the end of the performance test. The percentage profitable statistic measures the percentage of windows that were profitable.Windows are calculated with lengths of one, three, six, 12, 18, and 24 months.Perhaps the most relevant information on the Breakdown Statistics page isthe graph of Net Profit by Year, where we can examine the performance of the system on a year by year basis graphically. This graph can reveal if the strategy’s edgeis diminishing over time.
THE “HALF LIFE” OF STRATEGY PERFORMANCESome traders have questioned whether a strategy could work indefinitely or if alltrading strategies are doomed to fail once enough traders begin to exploit the specific inefficiency. My best guess is that performance of any trading strategy willdegrade over time, similar to a nuclear particle that decays exponentially over time.“Half life” is a term borrowed from chemistry and physics. It refers to thelength of time it takes for half the amount of a radioactive element to change intoa nonradioactive substance through the nuclear decay process. I believe that theperformance of trading strategies, like radioactive elements, is constantly decaying over time. With all the resources allocated to research, and enough peoplelooking at trading strategies, sooner or later someone will find your Holy Grail.This diamond in the rough will be found by someone else, and eventually performance will suffer due to a crowding out effect.Trend-following has been a mainstay for traders across the world for manyyears. The phase “always trade with the trend” is a trading mantra. Although his-
CHAPTER 4 Evaluating Trading Strategy Performance
95
tory has shown trend-following to be profitable, can we expect it to continue? Inthe second edition of my Comparison of Popular Trading Systems, I predicted thattrend-following traders might be in for some trouble in coming years. That wasexactly the case. In fact, Barclays Systematic Traders Index had some of its worstreturns in 1999 and 2001 (Figure 4.16).How did I know subpar returns were ahead for trend followers? Look atthe net profit by year for the channel breakout and moving average crossoverstrategies in Figures 4.17 and 4.18. Although almost every year between 1990and 1998 was profitable, the size of the profits was decreasing. Actually, if weprojected profits forward, annual net profit was expected to reach zero by 1999or 2000. This below average profitability is exactly what occurred in 1998through 2000.
Barclays Systematic Traders Index40%
Return
30%
20%
10%
2001
2000
1999
1998
1997
1996
1995
1994
1993
1992
1991
1990
1989
–10%
1988
0%
YearFIGURE
4.16
Barclays Systematic Traders Index. Systematic traders had below average returns during 1999 and 2001.
Why did the performance of trend-following systems run into trouble? Itcould be due to too much money trading trend-following strategies. It could be thatin this information age, business cycles and price cycles have shortened, therebycausing trends to last weeks, not months. It could have been bad luck that causedthe bad performance. Whatever the reason, our performance studies successfullypredicted the rough time to come.
Annual Returns of Channel Breakout Strategy$3,000,000
$2,000,000
Profit
$1,000,000
$0
2001
2000
1999
1998
1997
1996
1995
1994
1993
1992
1991
–$2,000,000
1990
–$1,000,000
Year
FIGURE
4.17
Annual Returns of Channel Breakout Strategy. Annual returns of a 40-day entry/20-day exit channelbreakout declined throughout the 1990s.
Annual Returns of Moving Average Crossover Strategy$3,000,000
$2,000,000
Profit
$1,000,000
$0
–$1,000,000
–$2,000,000
FIGURE
1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001Date
4.18
Annual Returns of Moving Average Crossover Strategy. Annual returns of a 10-day/40-day moving average crossover strategy decline throughout the 1990s.96
CHAPTER 4 Evaluating Trading Strategy Performance
97
WHAT TO DO WHEN STRATEGIES DETERIORATEEven if a strategy’s performance is deteriorating, it does not make that strategyuseless. In many cases we will still want to trade the system, but closely monitorthe continuing degradation of performance.When do we stop using a system?One method I use to determine cutoff points is to calculate a linear regressionof the equity curve along with bands plotted two standard errors above and belowthe forecast’s fit (see Figure 4.19). I use the standard error of the regression—a statistic which measures the goodness of the model’s fit—to create these bands. Whenthe equity curve breaks the lower channel, trading of the strategy should be haltedand a decision must be made either to scrap the strategy or modify its workings toalleviate further performance deterioration.
GOLD MINES OF BAD PERFORMANCEOften during the testing process, I’ll find a strategy whose performance is sodreadful that I really get excited. Any strategy that performs poorly and producestrading losses can always be reversed to generate profits. To accomplish this, simply buy when the basic strategy signals a sale and sell when the system generates
Equity Curve and Regression Band Forecast4
2
30
28
26
24
22
20
18
16
14
12
10
8
6
4
2
0
0
–2Time
FIGURE
4.19
Equity Curve and Regression Band Forecast. We can forecast equity growth using past data. If equityfalls below the lower band, we must reevaluate the potential of the strategy.
PART 1 Structural Foundations for Improving Technical Trading
98
a buy. This is one reason I do not deduct charges for commissions and slippage inmy strategy testing. I prefer to look at the true performance of a strategy first, andthen scale down expectations due to trading costs in later performance tests.
OTHER TESTING METHODSThere are a few other methods for testing strategy performance I occasionally usein specific circumstances. Some trading strategies may produce profits due solelyto an overall rising or falling underlying market. For example, many methods oftrading the S&P 500 may profit because that market has a positive drift over time,not because the system is able to spot market inefficiencies.The past few years aside, the S&P 500 has returned, on average, 8 to 12 percent per year. For a spot check to see if a strategy is outperforming the inherentreturns of being long the S&P 500, I regress the strategy’s returns on both the S&P500’s returns and squared returns. This method, first suggested by Jack Treynorand Kay Mazuy in 1956, measures the ability of a strategy to produce profitsregardless of the market’s overall drift.returnstrategy = b0 + b1 returnmarket + b2 returnmarket2 + ⑀I look at the significance of t-statistic of the b2 coefficient to determine thevalidity of the system. If the t-statistic is less than +1, then chances are the strategy’s performance is due only to the market’s runup or decline. If the t-statistic isgreater than one, then the strategy can be considered valid and not biased by themarket’s movement in one direction or another.
THE FALLACY OF MAGICAL THINKINGPsychologist B. F. Skinner (1948) carried out a number of experiments in whichhe noted a phenomenon called “magical thinking.” Skinner fed pigeons on fixed15-second intervals and noted changes in their behavior over time. He found thateach bird would begin to repeat specific behaviors (head twitching, turning in circles), and theorized that it had to do with a learned response to the reception offood. Despite being fed at regular intervals, the birds associated the appearance offood with whatever random behavior they were performing at the time of the firstfeeding. As a result, they continued to perform this behavior. While the pigeonsmight believe that their “magical behavior” is the force creating the food, in reality the process is controlled by an entirely different power—the experimenter.I have no doubt that there is much “magical thinking” in the markets as well.Many traders utilize trading strategies without evaluating their historical performance. Traders rationalize that if a strategy worked once or twice in the past, then themethodology must be valid. Inevitably, these methods cause large losses, since theypossess no expected edge. The quantitative trader who carefully and rigorouslyevaluates his trading methodology should not suffer a similar fate.
CHAPTER
5
Performance of PortfoliosMaintaining Returns While Decreasing Risk
B
y diversifying our trading over multiple markets, multiple strategies, and multipleparameters within the same strategy, we can reduce the overall risk of our trading.While casinos have used the principles of diversification for hundreds of years, thefinance community has only utilized these powerful tools for the past 50 years.
LESSONS LEARNED FROM A CASINOAlthough I’m not a big gambler, I do make my way to Las Vegas or Atlantic Cityat least a couple times a year. Despite knowing that it’s unlikely that I’ll win, I stilllike to play blackjack. I do know that if I play correctly, I expect to lose 0.5 percentof the total money I bet on the tables in any given night; the fates and fortune areresponsible in deciding which side actually prevails once the money is on the table.Here’s something to consider: If winning or losing at the tables is due to luck,then should the pit bosses (those super serious ladies and gentlemen employed bythe casino to keep track of the amount of your bets) be worried that I walk away awinner? After all, if I have won, then their employer, the casino, has lost.In fact, the pit bosses are completely indifferent as to whether I win or lose.Because of the principles of diversification, the casino is indifferent as to whetherany single customer makes money. Each single bet is virtually irrelevant in thegrand scheme of the casino’s profit and loss. Because bets are uncorrelated withother bets, good luck and bad luck will tend to cancel over time and leave the housewith the goodies—the natural edge in its negative expectancy betting games.99
Copyright 2003 by Lars Kestner. Click Here for Terms of Use.
PART 1 Structural Foundations for Improving Technical Trading
100
THE BENEFITS OF DIVERSIFICATIONIn a simple game where we flip coins to determine winners and losers, I flip a coinin the air and record which side of the coin lands faceup. If the coin lands onheads, you pay me $1.00. If the coin lands on tails, I pay you $1.00.To turn the percentages in my favor I’d need a magic coin. For example’ssake, suppose I found a coin that would turn up as heads approximately 55 percentand tails 45 percent of the time.First, let’s calculate my expected edge of playing this game. In any gamewhere there are two outcomes, we measure the edge by multiplying the probability of winning by the dollar amount of the winning payoff, then adding the probability of losing multiplied by the dollar amount of the losing payout:Edge = Probability of winning ⭈ Amount if won + Probability of losing ⭈Amount if lossIn our game above, the probability of winning is 55 percent, the winning payout is $1.00, the probability of losing is 45 percent, and the losing payout is–$1.00. Plugging these values into our equation yields our edge on each coin flip:Edge = Probability of winning ⭈ Amount if won + Probability of losing ⭈Amount if lossEdge = 0.55 ⭈ 1 + 0.45 ⭈ –1 = 0.10The expected edge of playing this game is $0.10. That is, I should expect tomake $0.10 each time I play. However, on any given toss I have no idea if I’mgoing to win or lose. Now suppose we agree to play this coin game under threescenarios:1. We play the game 10 times, betting $1.00 on each flip2. We play the game 100 times, betting $0.10 on each flip3. We play the game 1000 times, betting $.01 on each flipAt the end of each scenario, we tally up the wins and losses and settle up thecash value. Using statistics, I calculate that I have the identical edge under eachscenario.In Game 1, my expected value for each flip is 55 percent ⭈ (+$1.00) + 45 percent ⭈ (–$1.00) = $0.10. Multiplied by the 10 times we agree to play, my totalexpected edge is $1.00.■ In Game 2, my expected value for each flip is 55 percent ⭈ (+$0.10) + 45 percent ⭈ (–$0.10) = $0.01. Multiplied by 100, my total expected edge is $1.00.■ In Game 3, my expected value for each flip is 55 percent ⭈ (+$0.01) + 45 percent ⭈ (–$0.01) = $0.001. Multiplied by 1000, my total expected edge is $1.00.■
CHAPTER 5 Performance of Portfolios
101
While my expected value in each strategy is equal, the risk of each strategyis not the same. After all the coin tosses are counted, I will win money 50 percentof the time playing Scenario 1, 82 percent of the time playing Scenario 2, and 99.9percent of the time playing Sc