pping  forwarddoes  is  increase  the  responsiveness  of  our  rules  if  we  continuallymodify  them.  It  cannot  change  the  fact  that  we  are  testing  on  the
4  My  verdict  on  1987  is  that  too  many  people  were  trying  to  get  through  the  doorat the  same  time. The market’s physical incapacity to handle the volume andfinancial  incapacity  to  handle  the  direction  exacerbated  the  downward  movementon  virtually  no  shift 5  See in of  freedom.  For  simplicity  and  directnes$  see Bob  Pelletier.Trading  System  Design,”  CSI  Technical  Journal,  vol.  X,  numbers  9,  10,  and  Il.
literature  of  overfitting   and  statistical  degrees “‘An Official Guide to
fundamentals.trading 
the  many  discussions 
the 
in 
past for a very different future. Our only true defense is keeping therules simple.
This  is  contrary  to  the  instinct  of  the  novice  developer  whothinks  that  by  piling  on  more  rules   more  ifs,  and  more  filters,  thesystem  becomes  more  refined  and  specific.  As  it  turns  out,  we’dgenerally  like  to  get  by  with  one,  two,  or  at  most  three  decisionrules; our knowledge of market behavior is that vague. This generalguidance is one reason moving averages are useful: they are simple.Another  point:  When  testing,  you’d  like  to  get  enough  samplesto  accurately  portray  the  underlying  rules  of  the  entire  populationof  trades  (assuming  the  rules  aren’t  changing).  The  more  rules,  themore samples you’ll need to have confidence that your sample doesreplicate  things.  Because  the  rule  of  thumb  is  to  get  at  least  30  ofwhat  you’re  after,  the  typical  trading  system  that  has  1  winner  forevery 2 losers may need 90 trades (30 winners and 60 losers) to besafely  in  the  ballpark.  Thus  there  is  a  demand  for  lots  of  data  touse  in  testing.
DETAILS  OF  SPECIFIC  CONTRACTS
The  first  difficulty  in  testing  is  determining  what  to  test  becausedifferent  trading  vehicles  have  different  tradable  horizons.  A  stock,for  example,  is  obviously  traded  year-round.  Developing  rules  on1983 data and testing them on 1996 data works out easily.
In  contrast,  the  most  tradable  bond  changes  with  the  shape  ofthe  yield  curve,  pricing  differentials,  and  supply,  among  many  fac-tors.  It’s  been  impossible  for  me  to  find  a  consistent  time  series  ofbond  pricing  with  realistic  daily  ranges  for  individual  issues,  evengovernment%  making  testing  cash  bond  trading  systems  more  of  aproblem  than  I  could  handle.
Futures contracts are exchange-traded but are more complicatedthan  stocks  having  various  delivery  months  and  definite  deliveriesthat  are  “most  active.”  Not  only  does  developing  something  on  anactive month and testing it on an inactive month not work well (be-cause daily ranges are different), but there may not be enough datain  the  off  month  to  even  test.  Each  contract  is  also  peculiar  to  its
7 0   Testinz
Details  of  New  York  light  Crude
71
commercial  market  and  its  seasonality,  with  the  exception  of  thefinancials,   though  people  sporadically  publish  evidence  of  season-ality here, too.6
One  solution  to  facilitate  testing  has  been  continuous  contracts,artificial  time  series  that  are  cobbled  together  from  the  underlyingreal  contracts.  Another  solution  is  to  step  through  the  actualrollover process that you’ll follow in real life, thereby generating re-alistic  amounts  of  slippage’  and  commissions.  A  third  solution  isto  find  a  contract  or  possibly  two  that  are  adequately  active  year-round  and  roll  once  or  twice  a  year.  I  prefer  the  third  alternativebut  use  the  first  and  occasionally  fight  through  the  second  depend-ing on the item being traded.
Of  the  continuous  contracting  methods,  I  use  the  perpetual  con-tract created by  CSI.*  This is a rolling, 90-day, weighted average ofthe front two contracts’ prices. It generates a time series that has aconsistent,  realistic  daily  range,  a  factor  that  is  vital  in  analyzingadverse  price  movement,  at  least  initially.  It  is  also  statistically  sta-tionary.9   Once  I’ve  analyzed  the  contracts  using  perpetual  data,  Ijump to the contract that trades well year-round, if it exists, on fail-ing  that,  to  the  most  active  contracts  in  the  commodity  or  financialtradable. There, I repeat the analysis to see if the stop levels changematerially.‘O   My  adjustments,  if  any,  are  usually  on  the  order  of  atick  or  three.  Probably  the  most  tedious  example  of  this  processpossible is the contract for New York Light Crude.
DETAILS  OF  NEW  YORK  LIGHT  CRUDE
Crude  contracts  are  most  liquid  for  about  two  nearby  months,  butthe  December  and  June  contracts  are  liquid  several  years  into  thefuture.  That  is  to  say,  for  example,  they  have  solid  open  interest  inthe  December  1998  contract  during  June  of  1995  even  if  tradingvolume  is  much  lighter  in  the  out  months.  This  can  be  seen  in  Fig-ure 4.1, where the 1998 contracts show very little range while at thesame time the 1995 contracts are quite dynamic.
Another  difficulty  in  selecting  the  right  contract  to  analyze  isthat daily range typically widens as the contracts near expiration. Ifyou  simply  looked  at  correlations  between  perpetual  contract  val-ues  and  actual  contract  values,  you’d  find  them  running  in  the  .9range, close to the perfect 1  .O.  However, practical trading is more
transaction  price  and
the  estimated  of   ordered 
6 For examples, see Moore Research Center Report, various issues, Moore Re-search  Center,  Inc.,  321  West  13th  Avenue,  Eugene,  OR  97401.  Phone  (503)  484.7256,  FAX (503)  484-2202.’  Slippage  is  the  difference  between the  actual  price  received  on  the  order.s Commodity Systems, Inc., Boca  Raton.people  ask  for  CSI’s   Code  46  contract Code  46  does  best  on  contracts  that  trade  frequently,  like  Crude.9  A  stationary  time  series  is  one  in  which  the  underlying  rules  that  generate  thetime series do not change over time. See Clifford J. Sherry, The Mathematics ofTechnical  Analysis   (Chicago:  Probus,  19921,  p.  9.lo It’s a refinement beyond the scope of this book to analyze the change in the to expiration. The  cl&r to expiration, therange as the contract draws nearerwider  the  range,  and  a  sophisticated  analyst  may  adjust  MAE  stops  accordingly.
 FL. Phone (407) 392.8663. Today mostthe  original  Code  39  contract.
instead  of 
FIGURE 4.1DECEMBER CRUDE. Nearby and distant Crude contracts.Daily  range  drops  dramatically  in  the  distant  Crude  oil  contracts  despitea substantial open interest. The Crude contracts are prime hedgingvehicles,  but  the  nearby  two  or  three  contracts  are  best  for  daily  trading.Data  source:  Dial  Data,  Inc.
7 2   Testine
Details  of  New  York 
light  Crude
7 3
difficult.  As  the  previous  contract  expires,  volume  floods  into  thenext  active  contract,  resulting  in  more  excesses  of  pricing  andwider ranges. This can be seen in the daily range of the June 1995New York Light Crude contract shown in Figure 4.2.
In  contrast,  the  perpetual  contract  flattens  out  this  surge  in  the
last two months, as shown in Figure 4.3.
This  phenomenon  happens  to  some  extent  in  all  commodities,necessitating  some  consideration  when  testing  for  stop  levels  usingadverse price movements. (In stocks, there are certainly volume andrange surges but not, in my small experience, seasonally.) The con-venience  of  using  a  perpetual  contract  runs  into  the  practicality  oftrading  the  real  thing.  You  don’t  want  to  develop  a  test  during  thequiet  periods  and  test  it  or  trade  it  later  on  an  active  period-you’re going to find a lot of those closely set stops hit. There’s nosubstitute for examining the behavior of each trading vehicle, so in-stead  of  using  the  perpetual  for  testing  and  trading  signals,  we’llcheck some actual contracts, (Again, this digression deals only withfutures,  not  with  stocks,  so  your  technique  needs  to  be  specific  to
FIGURE  4.2 DAILY  RANGE.  Crude’s  daily  range,  june   1995.  In  atypical  phenomenon.  June  1994  New  York  Light  Crude  daily  rangeexpands as the contract nears expiration. The widening range has adirect  impact  on  adverse  price  movement  estimates.  ~~
FIGURE  4.3 PERPETUAL  RANGES.  Daily  ranges  in  perpetual  Crude.  Incontrast to the June 1995 Crude daily ranges, the perpetual contractdoes not show the shift upward in the last months of trading of the Junecontract. Traders testing for MAE stops need to adjust for thisphenomenon.
the  futures  market  you’re  trading.)  Generally,  finish  testing  withdata you’d actually trade.
Crude is a good example because it’s a highly liquid market withcertain  contracts  that  are  more  active  than  others  plus  a  definiteseasonality.  Moreover,  it  trades  in  a  complex  of  petroleum-relatedcontracts  that  are  constantly  arbitraged  among  each  other  andagainst  cash  or  physicals,‘t  thereby  influencing  each  other  by  theirindividual  supply  and  demand.  These  considerations-time  to  ex-piration,  seasonality,  popularity,  and  arbitrage-affect  the  volumein a contract at different times.
A  New  York  Light  Crude  contract  trading  on  the  New  YorkMercantile Exchange expires every month of the year, so you couldtrade  something  expiring  in  any  month.  However,  the  DecemberCrude  contract  is  the  most  popular,  the  bellwether,  showing  the
” Physicals is a term that refers  to the delivery of the actual commodity. in thiscase  crude oil.
7 4   Testine
Details  of  New  York  Light  Crude
7 5
highest open interest the furthest in the future. Next most popular isthe  June  contract,  following  the  same  considerations.  Correlationsbetween  June  and  December  are  in  the  .9  range,  as  are  correla-tions between these two contracts and the other monthly contracts.To  avoid  testing  and  trading  every  single  month  serially  duringtrend trading, you could trade December only or June and Decem-ber  and  find  your  overall  trend  signals  fitting  very  well  with  tradingthe  nearby  and  rolling  constantly.  Practically,  this  would  also  mini-mize  commissions  and  slippage  on  rollovers.
However, the techniques in Chapter 3 arc very sensitive to highsand  lows.  The  question  is  whether  the  appropriate  MAE  stopswould change over the year if the daily ranges expanded as you gotcloser  to  expiration.  To  answer  this,  you’d  look  first  at  the  dailyrange experience. Figure 4.4 shows examples of 4 of the 11 Decem-ber Crude contracts available for analysis in 1995.
Just  from  these  four  graphs,  you  can  see  that  daily  range  is  very  dy-namic. Would it be a good idea to adjust MAE stops by daily rangeas the contract neared expiration‘7 Perhaps, but the process is intri-cate so I leave it for a more technical examination elsewhere. For thepurposes of campaigning, we need only be aware of this refinement.For  testing  purposes,  I  first  tested  using  the  monthly  contracts(for  example,  trading  June  during  the  month  of  April,  then  tradingJuly in May, and so on). Then I tested using each year’s Decembercontract  from  November  of  the  previous  year  through  the  end  ofOctober.i2  To both the monthly continuous and the December con-tract,  I  applied  the  trading  rules  and  checked  the  MAE  charts.  Ichecked  to  see  whether  to  use  a  yearly  contract  or  the  month  tomonth  alternative.  Occasionally,  these  different  views  are  nearlyidentical, but in a contract as active as Crude. it’s not likely, therebyincreasing  your  workload  as  you  select  the  approach  most  usefulfor  actual  trading.
l2  When   rolling  from  one  month  to  another,  I  calculate  my  stops  in  the  new  con-tract  by  using  the  new  contract’s  price  point  on  the  day  the  trade  was  entered.  Forexample, on January 15 you go long at the close of $20 in the March contract.April is at $20.50 on the same day. Your  stop  is 45 points away at $20-.45  =$19.55.  The  equivalent  stop  in~Apri1  will be  S20.50-.45   = $20.05.
VOLATILITY 
IN  THE  DECEMBER  CONTRACT.  Daily
FIGURE  4.4ranges  in  the  December  New  York  Light  Crude  contract  fluctuate  withthe  season,  activity  in  the  market,  nearness  to  expiration,  wars-youname it. Because MAE stops are sensitive to daily range, you mustanalyze this carefully to select the best contracts to both test and trade.Stock traders should see less seasonality.
To  spare  you  the  agony  of  detail  this  entails,  I’ve  summarizedthe  results  graphically  for  each  year  in  Figure  4.5.  Note  in  eachpairing  that  the  chart  on  the  top  shows  the  number  of  winningtrades  and  the  number  of  losing  trades  by  the  maximum  adverseprice  movement  the  trades  experienced  en  route  to  closure.  Youshould note where the bulk of the winning trades lie, as well as thebulk  of  the  losing  trades.  There  should  be  a  distinct  difference,  orelse  our  trading  system  cannot  distinguish  good  from  bad.
Below  the  MAE  chart  is  the  chart  of  profitability  as  the  stop  iswidened  by  the  size  of  the  bins  used  for  classification,  In  nearlyevery case, you’ll see discernible peaks. Because it’s difficult for theeye  to  see  patterns  between  charts,  Figures  4.6  and  4.7  show  the
(text  continues  on  page  88)
76  Testing
Details  of  New  York  light  Crude 77
FIGURE  4.5 MONTHLY  CONTRACTS  WITH  ROLLOVER,  1984-1994.These  graphs  for  1984-I  994  were  created  from  monthly  contracts  withrollover  on  the  last  day  of  the  most  active  trading  month.  Commissionsand  slippage  are  not  included.  You  can  compare  these  to  the  graphsproduced  by  the  December  contracts  in  Figure  4.8.
FIGURE  4.5  (Continued)
1984. Winning and losing trades versus MAE. This graph show  thatthere  were  five  winning  trades  with  a  maximum  adverse  excursion  of.15  or  less  and  four  winning  trades  with  the  same  performance.
1985. Winning and losing trades versus MAE. This chart shows a cleardistinction  between  the  adverse  movement  of  winners  and  that  of  losers.
1984.  Profitability  versus  MAE  stop.  Look  at  this  graph  to  checkprofitability.  The  first  bar  shows  profits  with  no  stops.  The  second  barshows profits with the first MAE stop, in this case .16  or 1~6 trad~ing  points.
1985. Profitability versus  MAE stop. Here, as the winners are added inby  widening  stops,  they  manage  to  overcome  the  increasing  number  oflosers  that  aren’t  cut  short.  This  is  not  always  what  happens.
78  Testing
Details  of  New  York  light  Crude
79
FIGURE  4.5 MONTHLY CONTRACTS WITH ROLLOVER, 1984-1994.(Continued)
FIGURE  4.5  (Continued)
1986.  Profitability  versus  MAE  stop.
1987.  Profitability  vwsu   MAE  stop.
80  Testing
Details  of  New  York  Light   Crude
81
FIGURE  4.5 MONTHLY  CONTRACTS  WITH  ROLLOVER.  1984-1994.
FIGURE  4.5  (Continued)
1988.  Winning and losing trades versus  MAE
nax,nvm   AmerLe   txrvrrlon
  I”  Trading   PO,“,.
1989. Winning and losing trades verws  MAE.
989.  Profitability  versus  MAE  stop.
82  Testing
Details  of  New  York  light  Crude
83
FIGURE  4.5 MONTHLY CONTRACTS WITH ROLLOVER, 1984-1994.(Continued)
FIGURE  4.5  (Continued)
1991. Winning and losing trades versus MAE. This is a soberingtwosome.  You  can  have  all  your  winners  clustered  with  MAE  under  .16and  still  barely  come  out  ahead.  In  1991,  this  system’s  winners  weretiny  and  its  losers  huge.
-2500
1
1991. Profitability versus MAE stop
8 4   Testine
Details  of  New  York  light  Crude 85
FIGURE  4.5 MONTHLY  CONTRACTS  WITH  ROLLOVER,  1984-1994.(Continued)
FIGURE  4.5  (Continued)
1993.  Winning and losing trades versus MAE.
1993.  Profitability  verws MAE stop.
86 Testing
Details  of  New  York  light  Crude 87
FIGURE 4.5 MONTHLY  CONTRACTS  WITH  ROLLOVER,  1984-1994.(Confinued)
FIGURE  4.6 MONTHLY CRUDE MAE. Winners and losers versusMAE, 1984-l 994. Using each monthly contract as it conws  up showsthe usual distribution of winners with MAE less than .15  and losers withMAE  peaking  at  .3   then  tailing  off  to  the  right.  Compare  this  figure  toFigure 3.11 in Chapter 3 and to Figure 4.9.
~200 
1  3
1994.  Profitability versus MAE stop.~
n*Ew.pr.Tm”P.8.1*
FIGURE  4.7 MONTHLY  PROFITABILITY.  Profitability versus MAE,198&1994.   Showing  a  smooth  rise  in  profitability  as  stops  are  increasedto  .46,  trading  all  12  monthly  contracts  consecutively  comes  closest  toan 
intuitive 
result.
88  Testing
Details  of  New  York  light  Crude
89
results  summarized  for  all  ten  years.  The  summary  charts  are  corn-parable  to  the  charts  for  the  perpetual  contract  in  Chapter  3.13
Before going on to the graphs produced using just the December
contracts, a few observations:
1.  1 see relative stability about the stop point at  .46,  which is thefourth bin from the left. Setting a stop at  .46  for a trend tradedidn’t always produce the most profit, but it was always closeand at $460 per contract it’s pretty reasonable. Generally,  .46was in hindsight either the best choice for the year or the nextbest  choice.  For  slightly  lower  losses,  stops  at  .31   still  gener-ated a good profit.
2. You can “step forward” yourself with this graph series by, forexample, picking a stop level from the 1984 chart and seeinghow it would work in 1985. Repeat the process for succeedingyears.
3.  Generally,  setting  stops  closer  than  .46   was  tougher  onprofits. This confirms experience that stops set too closely aresimply  likely  to  be  picked  off,  probably  by  normally  noisymarket  action.  It  may  be  that  Crude’s  noise  level  in  thenearby contract is somewhere below 46 trading points.
4.  The  general  ratio  of  wins  to  losses  here  is  about  1:2:  a  de-pressing  but  realistic  number  in  the  trading  systems  develop-ment  world.  This  exemplary  averages  system  gives  maximumadverse excursion a lot of damage to limit.
In  conclusion:  Based  on  experience,  you  could  definitely  bothtrade  and  test  on  the  monthly  contracts.  You’d  find  it  tedious  butworkable.  The  next  question  is  whether  life  couldn’t  be  simpler  byusing, for example, the December contract.
I3 The year of trading shown  here is the January through December contracts  fooreach year. They were  tested from November of the previous year through Octoberof the nominal year.
Using  the  December  Contract
An alternative to trading month to month is to trade the most activecontract  of  the  year,  perhaps  adjusting  for  the  increasing  volatilityover  the  year.  I  find  this  works  well  for  the  large  active  markets.  Eachtrader will make a judgment on this. but for now all we need is a toolto  make  the  choice.  The  question  for  Crude  is  ‘Which  to  use.monthly or yearly?‘* Let’s see if there’s much difference. Figure 4.8shows MAE and profitability of New York Light Crude Decembercontracts for each year between 1984 and 1994. Figure 4.9 summa-rizes  MAE  for  1984-1994   December  contracts:  and  Figure  4.10summarizes profitability for  1984-1994  December contracts.
Picking  a  Trading  Series
Comparing Figures 3.7:  4.6, and 4.9, we see some remarkable con-sistencies (see Table 4.1): despite quite a bit of variation in individ-ual years’ results.
You  don’t  need  skew  statistics  to  see  that  the  distributions  inFigure  4.9  are  centered  differently  and  lean  differently-which  sim-ply  means  that  winning  trades  perform  differently  from  losingtrades as far as adverse price movement goes. This confirms tradingexperience that good trades hardly go against you, and here we seehow far they don’t go against you-maybe 45 to 46 ticks.
One  more  thing  before  making  a  choice  for  campaigning:  Lookalso  at  profitability,  which  is  a  more  complicated  picture.  Theprofitability  charts  show  the  level  of  gain  or  loss  from  the  tradingrules  given  only  one  thing  changing:  the  stops.
Say you decided to trade only the monthly contracts and with nostops.   Experience  from  1984  to  1994  shows  you’d  have  seen  ups  anddowns  leading  to  a  profit  of  2,000  ticks  over  ten  years.  But  whatups and downs you experienced. There’d have been 1987 with fourtrades  having  adverse  price  movement  of  up  to  120  ticks.  There’dhave been 1991 where you’d have stepped into a hole 2.000 ticksdeep in one year. You need to stop these killer events.
90 Testing
Details  of  New  York  light  Crude
91
FIGURE  4.8 MAE  AND  PROFITABILITY  DECEMBER  CONTRACTS.
FIGURE  4.8  (Continued)
December 1985. Winning and losing trades versus MAE
December  1984.  Winning  and as  bad  as  it  gets.  Here  there  are  actually  more  losers  clustered  in  the  firstbin  than  winners.  The  distinction  between  the  two  distributions clear.
losing  trades  versus  MAE.  This 
is  about
is  still
December  1984.  Profitability  versus  MAE  stop.  Even save  this  year,  but  see  below  for  the  exact  opposite:  stops  getting  in  theway.
tight  stops  couldn’t
December  1985.  Profitability  versus  MAE 
stop
92  Testing
Details  of  New  York  light  Crude
93
FIGURE  4.8 MAE  AND  PROFITABILITY  DECEMBER  CONTRACTS.(Continued)
FIGURE  4.8  (continued)
1987.  Here’s  a  depressing  pair  of  graphs.  You  get  excellent  definitionbetween  the  adverse  movement  of  winners  versus  that  of  losers  and  youstill can’t pick a stop point. If you pick .16, you barely make money.Any  greater  and  you  lose.  The  wins  just  aren’t  big  enough  to  cover  theplethora  of 
losers.
December  1986.  Winning  and 
losing 
trades  versus  MAE.
December  1987.  Winning  and 
losing 
trades  versus  MAE.
December  1987.  Profitability  versus  MAE 
stop.
96  Testing
Details  of  New  York  Light  Crude
97
FIGURE  4.8 MAE  AND  PROFITABILITY  DECEMBER  CONTRACTS.(Continued)
FIGURE  4.8  (Continued)
December  1991.  Profitability  versus   MAE  stop.
98  Testing
Details  of  New  York  light  Crude
99
FIGURE  4.8 MAE  AND  PROFITABILITY  DECEMBER  CONTRACTS.(Continued)
FIGURE  4.8  (Continued)
December  1993.  Winning  and 
losing  trades  versus   MAE.
December  1993.  Profitability  versus   MAE 
stop.
100  Testing
Details  of  New  York  light  Crude 1 0 1
FIGURE  4.8 MAE AND PROFITABILITY DECEMBER CONTRACTS.(Continued)
FIGURE  4.9 DECEMBER CRUDE MAE, 1984-1994. Winners andlosers  versus  MAE.  Trading  just  the  December  contract  from  Novemberthrough  October  gives  the  usual  peak  of  winners  with  very  low  MAEand a few winners with MAE as high as .9. Compare this figure to Figure3.11 and Figure 4.6.
PROFITABILITY VS. MAE USING DECEMBER
FIGURE  4.10CONTRACTS 1984-1994. In contrast to Figure 4.7, trading