net profitAnnualized % gain/lossTotal number of tradesWinning tradesLosing tradesPercent profitableGross profitGross lossAvg. profit/tradeRatio avg. win/avg. lossProfitability coefficientProfit factor
€93 09647 %110100 %€93 096€0€93 096
€58 946€94 759 €108 10036 %54 %49.34 %512611626147525124151.0 %53.8 %64.7 %€122 659 €124 582 €172 516−€63 713 −€29 823 −€64 416€1156€3645€9321.853.581.461.593.202.411.934.182.68
Max. drawdownMax. initial drawdownMax. consecutive lossesStandard deviation % of equityRisk adjusted annualized return
−€24 913−€13 28803.86 %12 %
−$14 618 −€19 254 −€15 507−€11 839 −€8092€06473.3 %3.3 %7.8 %11.0 %16.3 %6.3 %
Start date/testEnd dateTest period (years)Test period (days)Total trade length (days)Out of market (days)Average trade length – daysTotal trade length – barsAverage trade length – barsTrades/yearTime in marketTime exit – bars
1-19-0411-2-073.813831383013839489480.3100.0 %948
1-19-0411-2-073.813831156227238091613.583.6 %60
1-19-0411-2-073.81383109428942766296.979.1 %60
11-24-0311-2-073.91439143902910072029.4100.0 %50
Trading DAX Futures
211
Figure 13.2 The monthly % profit of the disparity system tested on 10 DAX componentstocks. The worst monthly return was in July 2007 during the summer 2007 correction.
average crossover, a popular and widely used trend-following method, to constructa system to trade DAX futures. The system was then enhanced using intermarketcriteria and their performance is compared in Table 13.2.
Test Procedure and System DesignA dual moving average crossover system generates a buy signal when the shorterof two moving averages exceeds the longer one; a sell signal is generated when theshorter moving average falls below the longer moving average.The short and long moving average period was optimized using data for the DAXindex for the period from 1993 to 2003 and the system was then tested out-of-samplefor the period from 19 January 2004 to 2 November 2007 on continuous DAX futuresdata obtained from Reuters Datalink.To avoid false signals associated with a congested market, the CI indicator wasagain used to detect the prevailing market trend. If the absolute value of the 40-day
212
Intermarket Trading Strategies
CI was above 30, it was assumed that a trend was developing and the moving averagecondition was used for long and short signals. If the absolute value of the CI wasunder 30 then the stochastic indicator was used to trigger buy and sell short orders totake advantage of oscillations associated with a range bound market.The best parameter set that managed to produce profits for long signals, in bothout-of-sample and in-sample testing, was 15 days for the fast moving average and 20days for the slow average. In the case of short sale signals, these values were 10 daysand 20 days for the fast and slow moving average periods respectively. The movingaverage optimization, however, apart from producing the best parameters, also createda problem. The slow moving average was in fact not slow at all, and therefore it didnot take into account the longer-term trends; as a result, some short sales that wereinitiated in bullish markets produced losses. To deal with short signals in a risingmarket, a second slower moving average condition was introduced that blocked anyshort sale signals when the 2-day was above the 150-day moving average. This extracondition helped improve the overall performance on the short side of the market.The best parameters for the stochastic (used only in a congested market) were to buywhen the 5-day stochastic crossed above its 4-day moving average and at the sametime the 40-day stochastic was below 30. Similarly, short sales were initiated whenthe 5-day stochastic crossed below its 4-day moving average and at the same timethe 40-day stochastic rose above 70.Long positions were liquidated when either of the following conditions were true:The CI dropped by 40 points or more during the last three days, or the 40-daystochastic rose above 85 and then dropped below 75 but only when the CI was below20 (in a congested market).Short positions were covered when: The CI rose by 40 points or more during thelast three days or the MACD crossed its 7-day exponential moving average and theprice also rose above the 7-day moving average.As you can see, the performance of the classic moving average system (summarized in the third column of Table 13.2) is nothing to write home about. It did notmanage to outperform the buy and hold strategy during the out-of-sample bullishtest period, but it outperformed it in both in-sample tests during mixed marketconditions.The disparity indicator was then used in order to filter out some signals that resultedin unprofitable trades. This time I used the Euro Stoxx 50 Index (STOXX50E) forin-sample testing and Euro Stoxx futures (FESX) for the out-of-sample test. Thus thedisparity indicator was used to filter out long trades when the disparity divergencebetween the DAX and Euro Stoxx was negative. The opposite was added to the sellconditions, in order to exclude some premature exit signals and prolong the holdingperiod in the case of positive disparity. Both short conditions were also enhanced byadding, this time not one but two, extra intermarket conditions. Thus a short signalwasn’t executed unless both the disparity was negative and the intermarket securitydeclined as well.
Trading DAX Futures
213
Evaluation of ResultsThe net effect of enhancing the system with the intermarket conditions is summarized in the fourth column of Table 13.2. The additional complexity of adding theintermarket signals reduced the number of trades by half to an average of one tradeevery two months. What was lost through fewer positions, however, was more thanmade up by tripling the stake per trade.Market conditions were very favorable for the buy and hold investor as the DAX wasin an almost continuous uptrend during the test period. Nevertheless, the intermarketenhancement did help to outperform the buy and hold method by 4 percentage pointson a risk-adjusted return basis but only marginally on a total profit basis.In conclusion I am convinced that although the enhanced moving average crossoversystem did not improve substantially on the buy-and-hold profitability, it does addstrength to the case for incorporating intermarket conditions that work in synergywith the conventional component parts of the system.
14A Comparison of a NeuralNetwork and a ConventionalSystem for Trading FTSE Futures
To conquer fear is the beginning of wisdom.–Bertrand Russell
The Financial Times Stock Exchange FTSE 100 index is the last of the trilogy of ourequity index trading systems.FTSE futures are traded on the Euronext LIFFE CONNECT electronic tradingsystem from 9:00 am to 10:00 pm Central European Time under the symbol Z (IBplatform). The Z futures market has a daily trading volume of approximately 100 000contracts and a daily price range of approximately 100 points. The contract valueis £10 per FTSE point and the overnight maintenance margin was £3000 at thetime of writing. Either each broker or the exchange can change margins periodicallyaccording to market volatility.Neural network-based systems have generated a great deal of interest lately but arethey as effective as the software vendors claim? I decided to investigate. In the firstpart of this chapter I will present a conventional rule-based system using the multipleregression model which produced the best results in trading gold (see Chapter 11).I used NeuroShell Trader Professional to construct a neural network model using,for comparison purposes, the same intermarket relationships with the conventionalmultiple regression system. The software’s built-in genetic algorithm was then usedto select the best inputs and optimize the parameters. The neural system is presentedin the second part of this chapter and the code and detailed step-by-step procedureare set out in Appendix C.
216
14.1
Intermarket Trading Strategies
CORRELATION WITH INTERNATIONAL INDICES
In order to select the best correlated indices to use in the multiple regression model, acorrelation matrix of 10-day (two week) percentage returns of major equity, currencyand interest rate indices was calculated (Table 14.1).The best and most obvious correlation of the FTSE, which stems from the UK’smembership of the European Union and its geographical position, was with thecorresponding European indices but because of the high multicollinearity betweenthem, only one can be included in the regression model. The best choices seemed tobe either the Euro Stoxx 50 (r = 0.85) or the French CAC 40 (r = 0.83) but it was aclose call. The dilemma was resolved by testing, and the CAC 40 was the finalist asit produced slightly better results on preliminary testing. The FTSE did not seem torespond, at least short term, to interest rates or interest rate changes as its correlationwith short sterling rates was virtually zero.With financials and oil and gas stocks making up almost 45 % of the index, thenext obvious candidates to include in the regression model were the bank and oilsector indices. I chose the American S&P Bank Index (BIX) and the Amex Oil Index(XOI) instead of the corresponding European sector indices, because of their lowercross correlation with the CAC 40 index.Thus the final multiple regression equation used in the model consisted of threeindependent variables and had the following form:FTSE = −0.11 + 0.551 ∗ CAC + 0.097 ∗ BIX + 0.116 ∗ XOI
(14.1)
Table 14.1 Pearson’s correlation of two-week (10 trading day) percentage returns betweenthe FTSE and other related markets. The ESTX is the Dow Jones Euro Stoxx 50 index, theBIX is the S&P Bank index, the XOI is the Amex oil index, the GBP is the USD per Britishpound sterling rate and the FSS is the three-month short sterling interest rate futures.Correlation of 10-day returns
FTSES&PESTXDAXCACBIXXOIGBPFSS
FTSE
S&P
ESTX
DAX
CAC
BIX
XOI
GBP
FSS
1.000.760.850.780.830.530.43−0.290.01
0.761.000.780.770.760.720.45−0.12−0.03
0.850.781.000.930.960.500.35−0.26−0.09
0.780.770.931.000.890.510.32−0.22−0.11
0.830.760.960.891.000.500.35−0.25−0.10
0.530.720.500.510.501.000.28−0.090.03
0.430.450.350.320.350.281.000.05−0.01
−0.29−0.12−0.26−0.22−0.25−0.090.051.00−0.12
0.01−0.03−0.09−0.11−0.100.03−0.01−0.121.00
Comparison of a Neural Network and a Conventional System for Trading FTSE Futures
217
where FTSE is the predicted 10-day percentage return of the FTSE index; CAC isthe actual 10-day percentage return of the CAC 40 index (FCHI); BIX is the actual10-day percentage return of the S&P bank index (BIX); XOI is the actual 10-daypercentage return of the Amex oil index.
14.2
SETTING UP THE TESTS
For a strategy to have value it must be adaptable to changing market conditionsso I divided the available data into two segments. I then used the first segment todevelop and optimize the regression model and the second segment of data to testit. There is no best way to divide the data into testing and out-of-sample segments.However, the out-of-sample segment must be long enough and different enough tobe able to make a judgment on whether the model is valid and can be used laterin actual trading without unpleasant surprises. In this case, the development periodwas from 1991 to 1999, and the out-of-sample period from 2 February 2000 to 19December 2007. The out-of-sample period, which included a complete market cycle,was different enough from the in-sample period and long enough in duration tojudge the model’s usefulness. You should keep in mind that you will need to load anextra 210 bars in order to take into account the time lag in calculating all indicatorsinvolved.A problem was again encountered with the historical data: The available data forFTSE and CAC 40 futures were not enough for both in- and out-of-sample testing.Another problem was that FTSE futures closed earlier than CAC 40 futures (theexchange only recently extended the trading time until 21:00 GMT). For the reasonsmentioned above I decided to use the actual index prices for developing and testingthe model.You cannot, of course, trade the index directly and therefore realtime trading resultsmay differ from the simulated ones presented below.Several assumptions underlie the trading model. The default trade size was onecontract per trade, and to avoid margin calls and the associated problems, the test wasconducted in points and converted to British pounds at £10 per FTSE point. Long andshort signals were executed the same day at the close and no interest was creditedto the account when out of the market as this was assumed to cancel out additionalslippage, skid and contract rollover costs.The commission charged by discount brokers of 0.17 points (£1.7) was deductedfrom each trade. Returns were calculated as a percent of the account size required totrade each system over the back-test period and compared to the buy and hold methodwhich involved buying and holding one FTSE contract for the duration of the test.Rollover costs involved in the buy and hold contract were ignored. The compoundformula was used to annualize percentage returns.The required account size was the greater of the initial margin required to trade theFTSE futures (£3750) or the maintenance margin (£3000) plus the initial drawdown.
218
Intermarket Trading Strategies
The percent standard deviation of change in equity was calculated by dividing themonthly change in equity in points by the value of the FTSE index also in pointsand the risk-adjusted return was calculated by dividing the annualized return by thepercent standard deviation of the change in equity.
14.3
SUMMARY OF CONDITIONS
Intermarket divergence systems usually generate few signals and this was the casewith the FTSE system as it produced only 22 signals on aggregate during the eightyear test duration. To increase turnover I added a second condition which took advantage of temporary pullbacks during strong trends but only on positive intermarketdivergence. This extra condition generated another seven long trades, thus increasingthe number of trades to a sufficient number for a valid statistical evaluation of thesystem.The system can be summarized as follows:Buy condition 1 (intermarket): The most powerful use of the intermarket momentum oscillator (IMO) is registered when the indicator reaches an extreme reading(usually above 50) and reverses direction. This signals an exhaustion of the downtrend and warns of an impending price reversal. Thus a buy signal was generated ifthe 200 bar IMO crossed under 50 from above.To eliminate premature signals, I used the 10-day stochastic and the 5-day exponential moving average to confirm the price reversal as follows: The system executedthe IMO buy signals only if the FTSE crossed over its 5-day moving average andthe stochastic indicator had crossed over 20 during the last three days prior to thedivergence signal day.Buy condition 2 (pullbacks): I used Bollinger Bands to identify temporary pullbacks as follows: A buy signal was generated if the FTSE declined below the lowerband during the last three days and the IMO was over 50.To ensure that this was a temporary correction and not a long-term bear market decline, buy signals were only taken if the FTSE was up by more than 2.4 %(0.03 % per day on average (specified by the linear regression slope) during the last80 days.Sell conditions: This was the exact opposite of the first buy condition. The positionwas liquidated when the IMO reached an extreme low value, reversed and crossedover 30. Once again the 10-day stochastic and the 10-day moving average were usedas confirming indicators and sell signals were only taken if the FTSE crossed belowits 10-day moving average and the stochastic crossed below 80.
Comparison of a Neural Network and a Conventional System for Trading FTSE Futures
219
Short conditions: These were very similar to the sell conditions described above,with the addition of an extra confirming requirement that the 10-day stochastic shouldbe below its 4-day moving average before executing the signal.Buy to cover: This was similar to the first buy condition described above.14.4
EVALUATION OF RESULTS
The results for both tests are summarized in Table 14.2. I have also marked in theaccompanying chart in Fig. 14.1 when each buy and sell signal was triggered but only(because of space limitations) from October 2005 to December 2007.In the period tested, the results were very impressive. During the course of the eightyear simulation, which included the devastating 2001–2002 bear market, the systemmade more than £50 000 with only 29 trades, which were roughly 83 % accurate.It managed to produce a profit factor as high as 9.9 while holding the maximumdrawdown to £6000 or 12 % of equity, which occurred on 24 April 2003, mainlydue to an unpropitious short trade during the sharp recovery from the preceding bearmarket downtrend.The risk and return statistics were also impressive. The monthly standard deviationof equity was only 3 % during the 8-year test duration which produced an 11 % riskadjusted return and a more than adequate Sharpe ratio of 9.4. The buy and holdinvestor, on the other hand, suffered a rather traumatic £35 600 drawdown during the2002–2003 bear market which, because of the margin calls, expanded the account sizerequired to hold one FTSE contract dramatically, from the initial minimum amountof £3000 to the ponderous sum of £38 600.The severity of the drawdowns should have wiped out any buy and hold investorslong before they could have enjoyed the profits from the ensuing 2003–2007 bullmarket. The intermarket system, however, managed to keep drawdown relatively lowby avoiding long trades during the intervening bear market.A perfect trading system is, however, impossible (is a perfect market possible?),and the current intermarket system is certainly no different. I identified the followingflaws after inspecting carefully the trade signals on the chart.The timing of the exits was not the best or most efficient. Some signals firedprematurely during extended up-trends but were late to materialize when needed most.In addition a couple of winning trades suffered large adverse price movements (adverse excursions) between 180 to 250 FTSE points before becoming profitable. Thiswas because of premature entry signals which are very common with countertrenddivergence indicators.This suggests that although the model was robust and maintained a more thanacceptable balance between risk and reward, the timing of the entry and exit signalscan be further improved by adding more conventional technical analysis rules andindicators.
220
Intermarket Trading Strategies
Table 14.2 Profit/loss report. Out-of-sample test results of the FTSE multiple regressionsystem based on its long-term relationship with the French CAC 40, the Amex oil index(XOI) and the S&P bank index (BIX).Test
Buy & hold
Regression
Security on test Intermarket security
FTSE
FTSE CAC, XOI, BIX
Account requiredTotal net profitProfit/loss %Annualized % gain/lossTotal number of tradesWinning tradesLosing tradesPercent profitableGross profitGross lossAvg. winning tradeAvg. losing tradeRatio avg. win/avg. lossProfitability coefficientProfit factor
£38 613−£67−0.2 %0.0 %1010%£0.00−£67
£6105£50 256823 %33 %2924582.8 %£55 905−£5649£2329−£11302.065.019.90
Max. drawdownDate of max drawdownMax. % drawdownMax. initial drawdownMax. consecutive lossesRisk/rewardStandard deviation % of equityRisk adjusted annualized returnSharpe ratio
−£35 6113-12-03−92.2 %−€35 6111−52 835 %4.22 %0.0 %−1.1
−£60154-24-03−12.4 %−£3104212.0 %3.00 %11 %9.4
Reference bars neededStart date/testEnd dateTest period (years)Total trade length (days)Out of market (days)Average trade length – daysTrades/yearTime in market
2-1-0012-19-077.92878028780.1100.0 %
2102-1-0012-19-077.91912966663.766.4 %
System ParametersRate of change – barsMomentum oscillator – barsMomentum oscillator – buyMomentum oscillator – sellTime exit – bars
−£67
102005030100
Comparison of a Neural Network and a Conventional System for Trading FTSE Futures
221
Figure 14.1 Chart of the FTSE from October 2005 to December 2007. Test signals generatedby the multiple regression method are superimposed on the chart. The 200-day divergencemomentum oscillator is plotted in the top window.
There is a limit to the performance that can be achieved using a single indicatorand many traders overlook the fact that using a diverse set of indicators, developedspecifically to work together, can add value and increase profitability.As I have mentioned in previous chapters, the underlying dynamics of the marketcan change over time. The regression coefficients used by the current system werethe best at the time of writing this chapter (December 2007). The markets, however,don’t stand still once a system is defined and by the time that this book is published,it is possible that some adjustments might be necessary.
14.5
A NEURAL NETWORK SYSTEM FOR TRADING THE FTSE
The real power of a neural net is in its ability to move a large amount of data and modelrelationships, if any, between the input and output. The output, of course, will dependon the input and a relationship between the security to be predicted and the input dataseries should exist in the first place. You should therefore include only those systemsthat you are sure have a relationship with the market to be predicted (in this case, theFTSE). In fact, one of the most powerful and easiest approaches for building a neuralnetwork system is using inputs from an existing profitable, rule-based system. Thisway, the neural system will try to improve results on known existing relationshipsinstead of searching on red herring relationships that may not exist. In developing
222
Intermarket Trading Strategies
Figure 14.2 Chart of the FTSE from February 2007 to February 2008. Test signals generatedby the ROC neural net system are superimposed on the chart. The equity line is plotted in thebottom window. The solid up arrows denote long entry, the solid down arrows short entry andthe hollow up arrows denote covering of short position. New long/short signals closed anyopen short/long positions.
a neural network to trade the FTSE, I tried the same related markets (the CAC 40,BIX and XOI) used by the conventional multiple regression system described above.I then combined the outputs from the neural system with conventional indicators todevelop a hybrid strategy.A problem encountered in developing the neural system was that, unlike conventional rule-based optimizers, the generic algorithm used by the neural networkto select and optimize the inputs is intolerant of even moderately correlated inputs.When I tried to input all three related markets (the CAC 40, the BIX and the XOI),
Comparison of a Neural Network and a Conventional System for Trading FTSE Futures
223
the algorithm used either the last two (the BIX and the XOI) or only the CAC,completely ignoring the others despite a moderate correlation (r = 0.5) betweenthe CAC and the BIX and a rather weak (r = 0.38) correlation between the CACand the XOI. I had to deal with this by developing another primary neural networkstrategy and feeding the outputs from the first tests back into the primary networkstrategy.Over-fitting is probably the most dangerous illusion a developer can encounterbecause it leads to false predictions that are based on isolated occurrences withinthe analyzed data set, and not genuine persisting relationships. An over-fitted modelcan be produced by including too many inputs or hidden neurons. This increasesthe capacity of the network which often leads to memorization instead of a genuinelearning progress. To mitigate the effects of over-fitting I reduced the maximumnumber of hidden neurons from the program’s default 10 to 7.Another common problem with neural network systems is that they tend to tradetoo often or too sporadically. In NeuroShell this can be adjusted by increasing thedesired shortest or longest average trade span (on the optimization tab). You can alsoadjust the number of trades indirectly by changing the training objective, for examplefrom “maximize net profit” to “maximize return on trades”.A technical support representative suggested that yet another way to deal withthe problem of overtrading is to increase commission costs. This will penalizebad trades more and cause the system to avoid them. The profit calculations,however, will be incorrect but these can be adjusted by adding back the excesscommissions.
Test ProcedureThe appropriate data sets for the FTSE, the CAC 40, the XOI and the BIX wereloaded on a NeuroShell chart page.I chose to select manually the data set sizes used for training, testing and outof-sample testing, although it is now possible to optimize these as well (using theAdaptive Turboprop 2 add-on).As a general rule, the data reserved for training the model should include enoughvarying market conditions to allow the generation of rules that can cope with likelyscenarios, rather than just what has happened most recently. But if the time period istoo long or too old then the rules will be affected by outdated relationships and thetrading results will suffer. Perhaps a good rule of thumb is to allocate approximately80 % of the available data for the training and testing subsets and to use the rest forout-of-sample confidence testing.The available data was divided into three datasets. The neural net was trained ondata spanning from 27 April 1993 to 31 December 2002. The second dataset, from1 January 2003 to 31 December 2004 was used for testing during training or “papertrading” and the last one for out-of-sample testing.
224
Intermarket Trading Strategies
The genetic algorithm optimized the model’s parameters on the training sample,but the testing sample was also used during the optimization process in order to tryeach new optimal solution. The testing sample is not therefore a true out-of-sampledata set and an additional three years, spanning from 1 January 2005 to 27 February2008, were reserved to see how the final model fared on data that it had never seenbefore.Having established the testing samples, we can now proceed in specifying thefollowing: Input data preprocessing, the prediction output, the size and structure ofthe neural system, and the trading details (commission, margin, etc.).
InputsOf course, the key to a successful neural network model is in choosing the bestinputs. Having established the most appropriate intermarket securities in the first partof this chapter, the next and most arduous task was deciding on the best preprocessingmethod.An initial consideration of the inputs to the neural network concerns whether to useraw data (e.g. direct prices of the target and intermarket securities), or preprocesseddata (e.g. price differences). This step is extremely important, since neural systemstrain much better on relative numbers such as differences, oscillators, momentumand ratios, which provide the relevant relationships explicitly rather than forcing thesystem to discover them. The more pertinent the data that you provide to the network,the better it will train and it is therefore important to select the training data carefullyand preprocess them before training.But which is the best transformation to use?The hardest task was in deciding whether to use raw data, preprocessed data of theintermarket securities, or spreads between the FTSE and its related markets. As thereare no standard rules available for determining the appropriate preprocessing method,I decided to use a brute force trial and error method and train a large number of neuralnets with different inputs, discarding the preprocessing methods of the models whichperformed the worst.Simple preprocessing included taking percentage differences of the price data,simple and exponential moving averages and the stochastic and disparity indicators.More elaborate preprocessing consisted of taking spreads between the target market(in this case the FTSE) and its related markets. This involved taking differences ofthe percentage price differences, and the disparities. I used nine different preprocessing transformations on aggregate, including the raw price inputs, but because ofspace limitations I chose to present only the best three in Table 14.3. These werethe percentage changes of the intermarket securities (ROC), the relative percentagechanges or spread between the FTSE and the intermarket securities and finally therelative disparities (the distances from their moving average) between the FTSE andthe intermarket indices.
Comparison of a Neural Network and a Conventional System for Trading FTSE Futures
225
Table 14.3 Out-of-sample profit/loss report. Out-of-sample test results of three intermarketneural network systems on the FTSE based on its relationship with the French CAC 40, theAmex Oil Index (XOI) and the S&P Bank Index (BIX). The first test (ROC) was based on the% rate of change of the intermarket securities, the second on the relative % rate of changebetween the FTSE and the intermarket securities and the third on the relative