Many people presume that MCP delivers on two promises: (1) Thecandidate with the highest observed performance is most likely to per-form the best in the future, and (2) the observed performance of the best-performing candidate is a reliable estimate of its future performance. Itdoes deliver on the first promise. However, in the domain of TA rule backtesting, it does not deliver on the second.
With regard to the first promise, that the candidate with the highestobserved performance is also the one most likely to do best in the future,this was proved to be true by White” as the number of observations ap-proaches infinity. White showed that, as the sample size approaches infin-ity, the probability that the candidate rule with the highest expectedreturn (i.e., the truly best rule) will make itself known by having the bestobserved performance approaches 1.0. This tells us that the basic logic ofdata mining is sound! The rule with the highest observed performance isthe rule that should be selected. The validity of this assumption will alsobe demonstrated by mathematical experiments presented later in thischapter: Experimental Investigation of the Data-Mining Bias. These re-sults show that when a sufficient number of observations are used to com-pute a rule’s performance statistic (e.g., its mean return), the rule with thehighest observed performance does have a higher expected return than arule picked at random from the universe of rules tested. Data miningmust, at the very least, pass this minimal test of efficacy for it to be con-sidered a worthwhile research method, and it does!282 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
With regard to the second promise, that the observed performance ofthe selected candidate is a reliable estimate of its future performance, thenews is not good. Jensen and Cohen” point out that, when MCP is appliedin situations where randomness plays a significant role in observed per-formance, the observed performance of the best-performing candidateoverestimates its expected (future) performance. In other words, whenobserved performance is significantly determined by luck, it is gives a pos-itively biased estimate. This is the case in TA rule data mining.
Observed performance is a reliable estimate of the best candidate’sfuture performance in situations where randomness is nonexistent or solow that it can never influence the choice of the wining candidate. Such isthe case in a music competition or a competition at doing mathematicalproofs, where it is merit, not luck, that carries the day.
The bottom line: the MCP selected candidate is the candidate mostlikely to do best in the future so long as a large number of observationsare used to compute the performance statistic. Nevertheless, in problemswhere randomness has a significant impact, observed performance ispositively biased, and the selected candidate's future performance will, inall probability, be worse than the performance that allowed it to win thecompetition.
MCP Efficacy in Low Randomness Situations
First, consider the application of MCP to problems with low randomness.At this end of the spectrum, a candidate’s observed performance is domi-nated by merit, and MCP is effective in identifying superior merit.
This problem is exemplified by the task of hiring a new first violinistfor a symphony orchestra.” The universe of candidates consists of the setof musicians applying for the job. Each is asked to perform a challengingcomposition, without prior rehearsal, for a panel of judges. The judges’evaluation is the figure of merit. This acid test of instrumental compe-tence, known as sightreading, is effective because great performances donot occur by luck. If luck is a factor, it is a minor one. For example, a greatmusician may have an off day because of a marital spat or a flat tire on theway to the audition, but even these random influences will have only asmall effect on a truly meritorious musician.
In this situation, observed performance is an accurate indicator oftrue merit and an excellent predictor of future performance. This isdepicted in Figure 6.14. Each candidate’s merit, which is equivalent tohis expected performance, is indicated by the arrow. Both musiciansare excellent, although one is slightly better. The distribution of possi-ble performance surrounding each candidate’s merit is narrow, indicat-ing the minor impact of randomness on observed performance. NoteData-Mining Bias: The Fool's Gold of Objective TA 283
 
 
 
 
 
 
 
 
 
 
ExpectedPerformanceHigher Meriti did.Probability CandidateExpectedPerformanceLesserCandidate“—Poor Excellent
Performance
FIGURE 6.14 Low randomness—small merit differential. True merit shinesthrough thin fog of randomness.
that there is no overlap in the distributions so that even if the bettercandidate has an unlucky performance (i.e., at the extreme low end ofthe distribution) and the lesser candidate has an extremely lucky one,the better candidate will still be selected. This is another way of sayingthat randomness is unlikely to inflate the inferior competitor's perfor-mance sufficiently such that it would be selected over the higher meritcandidate.
MCP Efficacy in High Randomness
Now consider a situation at the opposite end of the randomness spec-trum, TA rule data mining. Here, randomness has a major impact on ob-served performance, for even the most potent TA rules or models possessrelatively little predictive power. This is a consequence of the complexand highly random nature of financial markets. The probability distribu-tion of observed performance of a rule with an expected return of zero isillustrated in Figure 6.15. Although zero is the most likely return, farhigher or lower returns are possible due to good or bad luck. If the rule isunlucky in the back test, observed performance will be negative. Evenmore problematic for the objective technician is when good luck shineson the worthless rule and it earns a positive rate of return. This may foolthe objective technician into believing that TA gold has been found. Infact, the rule’s signals coincided favorably with the market’s fluctuationspurely by accident.
Although an extremely positive or negative average return is unlikelyfor an individual rule, it becomes increasingly likely as more rules areback tested. Just as the greater the number of people playing the lottery284 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
 
 
 
 
 
 
 
 
 
 
 
 
 
ProbabilityExpectedReturn = 0%Possible PossibleBut ButRare Rarec ~ c >Bad.Luck | Good Luck |INegative Returns 0 Positive Returns
Annualized Return
FIGURE 6.15 > Probability distribution of observed performance.
increases the chance that there will be someone lucky enough to wintwice, the greater the number of rules back tested increases the chancethat a rule will have an extraordinarily lucky observed performance. Thisis the rule that would be picked by the data miner. The bottom line: whenobserved performance is primarily due to randomness, it is likely that thebest observed performance, among a large set of observed performances,will be largely an effect of randomness.
To see how luck might impact a data miner, imagine that you are ob-serving a data miner at work. Furthermore, imagine that you are in thefortunate position of knowing what no data miner ever knows, the trueexpected return of each rule that is back tested. Suppose the data minerback tests twelve different rules, and you know that each of the rules hasan expected return of 0 percent. All the data miner knows are the ob-served returns produced by each rule’s back test. In Figure 6.16, eachrule’s observed performance is depicted by an arrow on the probabilitydistribution. Each distribution is centered at zero, reflecting the fact thateach rule has an expected rate of return of zero. Notice that one rule gotlucky, producing an observed return of +60 percent. This is the rule thatwould be selected by the data miner. In this case, the data-mining bias wasa positive 60 percent.
If the data miner were to test an even greater number of rules, thechance of an even more positive observed performance would be in-creased. As will be discussed, the number of rules tested during data min-ing is one of five factors that impact the size of the data-mining bias. InFigure 6.17, 30 rules are tested, and one managed to produce an observedperformance of 100 percent. This is the rule that would be selected by thedata miner. You know it is highly probable that its future performance willbe disappointing.Data-Mining Bias: The Fool's Gold of Objective TA 285
AN AlJN!
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
0 0
 
 
 
 
 
 
 
 
 
 
0yl0
 
 
 
0
ZAIN
 
 
 
 
AN
0
60%]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
~
ae
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
FIGURE 6.16 Twelve different rules (each has an expected return of 0 percent).0 {o} 0 0 0oO 0 0 oO 0o 0 oO oOAN _H NO Ugk Lak0 0 0 0 0ZN 7K AK NT o 0 0 0|tN LER LK UK A
 
 
FIGURE 6.17 Data mining more rules increases probability of more extreme luck.286 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
The Risk of Picking an Inferior Rule
The prior examples assumed that all rules were of equal merit (all had ex-pected returns equal to zero). Naturally, the data miner hopes that there isat least one superior rule amongst all those tested, and that the superiorrule’s observed performance will identify it as such. Unfortunately, this isnot always the case.
This is another negative consequence of high randomness. The superiorrule, the one with highest expected return, may not get picked, because aninferior rule's lucky performance wins the data mining competition. An intu-itive sense of the likelihood of this unfortunate result is portrayed in Figure6.18. It shows the distribution of observed performance for two rules withnearly identical expected performances, but one is indeed superior. How-ever, the considerable overlap of the distributions gives a sense there is asubstantial chance the rule of lesser merit will produce the highest ob-served performance, causing it to get picked.
However, when the difference in expected returns (merit) betweenthe truly best rule and the next best rule is large enough, the data minercan be more confident that best-observed performance will reliablypoint to the superior rule. This is illustrated in the Figure 6.19. In otherwords, when the difference in merit between the best rule and its near-est competitor is large, merit will be more likely to shine through thefog of randomness. This is the illumination that guides the data miner tothe gold.
Practically speaking, the data miner is never aware of the risk of pick-ing an inferior rule. That would require knowledge of true merit (expectedreturn), which is a population parameter and hence never known. The
 
Expected ReturnHigher Merit Candidate
 
 
 
Probability[Expected Return || Lesser Merit Candidate |
 
 
Low
 
Performance
FIGURE 6.18 High randomness and small merit difference.Data-Mining Bias: The Fool's Gold of Objective TA 287
 
 
 
 
 
 
 
   
 
 
 
 
 
 
 
Probability: Expected ExpectedReturn i Bad Returni Luck Superior Rule“ t vL Highow 0 420% "9
Performance
FIGURE 6.19 High randomness and large merit difference (superiority over-comes randomness).
data miner never gets to see the full consequences of the data-mining biasuntil, as the song says, the future is past.
Five Factors Determine the Magnitudeof the Data-Mining Bias
Let us recap briefly what has been established thus far:
e The data-mining bias is defined as the expected difference betweenthe observed performance of a rule that wins the data mining compe-tition and its true expected performance.
¢ Observed performance refers to the level of performance achieved bya rule in back testing. Expected performance refers to the rule’s theo-retical performance in the future.
¢ The observed performance of the highest-performing rule found bydata mining is positively biased. Therefore, its expected performanceout of sample will be less than the in-sample observed performancethat allowed it to beat other rules tested.
¢ Observed performance is a combination of randomness and predic-tive power. The greater the relative contribution of randomness, thelarger will be the magnitude the data-mining bias.
The relationship between the data-mining bias and the relative contri-bution of randomness is illustrated in Figure 6.20. As explained in the nextsection, the degree of randomness encountered in a given data miningventure depends on five factors that characterize the venture.288 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
LargeData
MiningBias
No Bias jw
 
 
Spectrum of Randomness
FIGURE 6.20 Relationship of data-mining bias to degree of randomness.
The Five Factors Defined. Five factors determine the degree ofdata-mining bias. They are:
1.
go
Number of rules back-tested: This refers to the total number ofrules back tested during the data-mining process en route to discover-ing the highest-performing rule. The larger the number of rules tested,the larger the data-mining bias.
. The number of observations used to compute the performance
statistic: The larger the number of observations, the smaller thedata-mining bias.
. Correlation among rule returns: This refers to the degree to which
the performance histories of the rules tested are correlated with eachother. The less correlated they are, the larger the data-mining bias.
. Presence of positive outlier returns: This refers to presence of
very large returns in a rule’s performance history, for example a verylarge positive return on a particular day. When these are present, thedata-mining bias tends to be larger, although this effect is reducedwhen the number of positive outliers is small relative to the total num-ber of observations that are used to compute the performance statis-tic. In other words, more observations dilute the biasing effect ofpositive outliers.
. Variation in expected returns among the rules: This refers to the
variation in true merit (expected return) among the rules back tested.The lower the variation, the greater the data-mining bias. In otherwords, when the set of rules tested have similar degrees of predictivepower, the data-mining bias will be larger.
How Each Factor Impacts the Data-Mining Bias. Figures 6.21through 6.25 depict the relationships between each of the five factors andData-Mining Bias: The Fool's Gold of Objective TA 289
Large
DM Bias
 
1 ManyLog of Number of Rules Back-Tested
FIGURE 6.21) Number of rules back tested (1).
Large |;
DM Bias
 
1 ManyNumber of Observations
FIGURE 6.22 Number of observations used to compute performance statistic (2)
Large
DM Bias
 
0.2 0.4 0.6 0.8 1.0
Correlation Coeffient
FIGURE 6.23 > Correlation among rule returns (3).290 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
Large
DM Bias
 
Low High
Number & Magnitude of Outliers Relative to Number of Observations Used toCompute Performance Statistic
FIGURE 6.24 Presence of positive outlier returns (4).
Large }-
DM Bias
 
Low Variance High Variance(All Rules * Merit) (One Very Superior Rule)
FIGURE 6.25 Variance in expected ROI—degree of difference in merit in ruleuniverse(5).
the magnitude of the data-mining bias. These relationships pertain specifi-cally to data mining ventures where the performance statistic being usedis the rule’s mean rate of return. Because the sampling distribution ofother performance statistics, such as the Sharpe ratio, will differ from thedistribution of the mean return, the relationships between the five factorsand the data mining bias may also be different.
The astute reader will wonder how such curves could have been de-veloped, given that a rule’s expected performance is never known (it is apopulation parameter) but must be known to measure the data-miningbias. The reader is correct; the data-mining bias is never known for actualrules. However, it can be known for artificial rules and can therefore beinvestigated. An artificial rule is a computer-simulated trading signalData-Mining Bias: The Fool’s Gold of Objective TA 291
whose accuracy (fraction of correct signals) can be set experimentally.This makes it possible to know the expected return of the artificial rule.The preceding curves were based on a set of tests done on artificial rules.The results of these tests, which are described in the next section, showhow the five factors impact the data-mining bias.
EXPERIMENTAL INVESTIGATIONOF THE DATA-MINING BIAS
 
 
Scientists make observations and draw inferences from them. Sound infer-ences depend on accurate observations. Thus, a key task is determining theaccuracy of the procedure used to make observations. The process of mea-suring the random and systematic error that may be present in an observa-tional procedure is called calibration. One way to calibrate the accuracy ofa procedure is to test it on a problem whose correct answer is known. Thismakes it possible to measure its random and/or systematic errors.
Objective technicians are the scientists of the TA world. Their primaryprocedure is rule back testing. The observable produced is a performancestatistic. On the basis of this statistic, an inference is made about a rule’spredictive power or expected performance. Therefore, objective techni-cians are properly concerned with the random and or systematic errorthat that may be present in performance statistics obtained by back test-ing. Chapter 4 showed that performance statistics are subject to randomerror due to sampling variability. This chapter is concerned with a form ofsystematic error that stems from data mining—the data-mining bias.
This section describes the results of experiments that investigate thedata-mining bias by examining how each of the five factors impacts thesize of the bias. This is done by data mining a universe of artificial tradingrules (ATRs). Unlike real TA rules, ATRs are ideal for this purpose be-cause their expected return can be known because it is under experimen-tal control. This allows us to measure the data-mining bias associatedwith the rule that had the best-observed performance. This, in turn, tellsus how accurately its observed performance, a statistic which is known tothe data miner, portrays its expected performance, the population para-meter the data miner wishes to know.
Artificial Trading Rules andSimulated Performance Histories
The ATR performance histories produced in these experiments are com-posed of monthly returns. The expected return of an ATR is controlled292 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
by manipulating the probability of a profitable monthly return. Thisprobability can be expected to manifest over a very large number ofmonths, a manifestation of the Law of Large Numbers. However, overany small sample of months, the actual fraction of profitable months willvary randomly about the specified probability level. This variation intro-duces the crucial element of randomness into the experiments. An ATRperformance history can be generated for any specified number ofmonths (e.g., 24).
When the probability of a profitable month is known, as it is with anATR, its expected return can be determined precisely. This, in turn, allowsthe data-mining bias associated with the rule to be measured. The ex-pected return of an ATR is given by the formula used to compute the ex-pected value of a random variable:
EV = p(profitable month) x average gain — p(losing month)x average loss
The average gain and average loss are also known because the ATRswere applied to the absolute monthly percentage changes for the S&P 500over the period August 1928 through April 2003. The term absolute meansthe sign of the S&P’s actual monthly change was ignored. In the ATR teststhe algebraic sign (+ or —) of the ATR’s monthly return was determined bya random process that is described below. Over this time period, compris-ing nearly 900 monthly observations, the average absolute monthly returnfor the S&P 500 was equal to 3.97 percent. Thus, the formula for calculat-ing an ATR’s expected return is:
ER = ppm x 3.97 - (1 — ppm) x 3.97
Where ppm is defined as the probability of a profitable month.
ATR performance histories were generated by Monte Carlo simula-tion. Specifically, the absolute monthly changes in the S&P 500 were sam-pled with replacement from the 900-month history. A monthly return,without its sign, was chosen at random, and a roulette wheel, simulatedby a computer, determined if that particular month was a gain or loss.This represented one month of the ATR’s performance history. The proba-bility of a profitable month was set by the experimenter. A probability of0.70 would be as if the roulette wheel had 100 slots with 70 designated asprofit and 30 designated as loss. This procedure for generating monthlyATR returns was repeated for a specified number of months, another vari-able under experimental control. The performance history is then summa-rized with a statistic—mean monthly return, annualized. This procedure isillustrated in Figure 6.26.Data-Mining Bias: The Fool’s Gold of Objective TA 293
n-month meanReturn =?
ae
Random Device
               
AbsoluteMonthlyReturns
S&P 500
With Performance1928 -2003 Probability of HistoryRandom Profitable month ofSampling Set Experimentally M months
WithReplacement
FIGURE 6.26 Monte Carlo generation of ATR performance histories.
Let’s consider what the expected return for an ATR would be under fourdifferent settings for probability of a monthly gain: 1.0, 0.63, 0.50, and 0.0. AnATR with p(gain) set at 1.0 (all months profitable) would have an expectedmonthly return equal to the S&P 500’s average absolute monthly retwmn of+3.97 percent per month or a 47.6 percent annualized noncompounded re-turn. This value can be obtained by plugging into the formula for the ex-pected value of a random variable. The same formula tells us that an ATRwith probability of a gain month set to 0.63 would have an expected return of+12.4 percent. An ATR with probability of gain set to 0.50 would have an ex-pected return of zero whereas a probability of gain equal to zero would earna return of -3.97 percent per month or -47.6 percent per year.
Bear in mind that these are expected values that would be earnedover a very large number of months. Over any small number of months,the ATR’s observed mean return can vary from this value. The smaller thenumber of months comprising the ATR’s performance history, the largerwill be the random variation from the expected mean return.
To simulate the effect of data mining, for example picking the best-performing ATR of 10, 10 ATR performance histories were generated. Theone with highest mean return was selected, and its observed return wasnoted. This process was repeated a total of 10,000 times. The 10,000 ob-servations were used to form the sampling distribution for the statistic—mean return of the highest performing ATR selected from 10.
Experiment Set 1: Data Mining ATRs of Equal Merit
In the first set of experiments, all ATRs were specified to have equal pre-dictive power. This was accomplished by setting p(gain) to 0.50 for allATRs, thus giving them all an expected return of zero.294 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
Factor 1: Number of Rules Tested. All else being equal, the morerules back tested to find the best rule, the larger the data-mining bias.More monkeys dancing on keyboards increases the probability that onewill get lucky enough to type something that appears literate. Similarly,back testing a larger number of rules increases the chance that one willenjoy extraordinary luck.
In the tests that follow, each ATR was simulated over a 24-month pe-riod. First let’s look at the no-data-mining case—only one rule is backtested. There is no data mining and no data-mining bias. Although an ATRwith p(gain) set at 0.50 has an expected return equal to zero, samplingvariability makes it possible for any given 24-month history to produce amean return that varies above and below zero.
To demonstrate this, 1,000 ATR performance histories, each of 24months’ duration, were produced by computer simulation. The samplingdistribution for the statistic mean annualized return is shown in Figure6.27. As expected, the distribution is centered at zero, which is indeed theexpected return of an ATR with p(gain) set 0.50. Also as expected, themean return displays a wide range of variation around zero because 24 ob-servations is a relatively small sample size. A few ATRs, those out in theright tail of the sampling distribution, had very good luck (more than 50months wound up profitable). In the left tail of the distribution, we findthe performance histories with bad luck. However, it is the center of thesampling distribution that tells the story. On average, under the conditionof no data mining, an ATR with no predictive power can be expected to
250;-
200/-
150;-
100/—
50;-
 
| I L I |-48 -38 -29 -19 96 0 96 19 29 38 48
 
FIGURE 6.27 Mean annualized percent return (single ATR, 24 month history,1,000 replications).Data-Mining Bias: The Fool’s Gold of Objective TA 295
earn a mean return of zero. As the next tests will show, when a best-performing ATR is picked from two or more ATRs, the best is likely to showareturn greater than zero, even though it has an expected return of zero.
Now we will be data mining by picking a best ATR from two or more.To examine the relationship between the size of the data mining bias andthe number of rules from which the best is selected we will vary the sizeof the ATR universe—the number from which the best is picked. Specifi-cally, the bias is measured for the best ATR of 2, 10, 50, and 400. For ex-ample, if the number of ATRs was set at 10, the best-performing one of 10was picked, and its observed mean return was noted. This procedure wasrepeated 10,000 times and the sampling distribution for the statistic (ob-served return of the best ATR) was plotted. All performance histories are24 months, all rules are set to p(gain) = 0.50 (expected return = 0), and allrules are set to have independent returns (no correlation). Because allATRs had expected returns equal to zero, the data-mining bias is simplyequal to the observed return of the best performer averaged over 10,000replications.
Figure 6.28 shows the sampling distribution of observed perfor-mance for the best ATR out of two. The sampling distribution is centeredat +8.5 percent. In other words, when only two ATRs are tested and thebest is selected, the bias is a positive 8.5 percent. The data miner wouldbe expecting +8.5 percent in the future, but we know the true expectedreturn is 0 percent.
If the number of ATRs is increased to 10, and the best is selected, thedata-mining bias increases to +22 percent. However, we know that theselected ATR has a p(gain) of 0.50 and an expected return of 0 percent.
 
 
 
 
 
 
 
 
    
 
ObservedExpected ReturnReturn BestBest ATRATR Of2 RulesRelativeFreq.Data MiningBias+8.5%
 
 
 
 
-70 -60 -50 -40 -30-20-10 Q 10 20 30 40 50 60 70
FIGURE 6.28 Sampling distribution —mean return best of two ATRs.296 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
Figure 6.29 is similar to 6.28, except it shows the sampling distribution forthe mean observed return for the best-of-10 ATRs. The observed perfor-mance of the best-of-10 ATRs is biased by +22 percent.
When the size of the data-mined universe is expanded to 50 ATRs thebias increases to +33 percent. Figure 6.30 shows the sampling distributionfor the mean return of the best-of-50 ATRs.
A similar plot, in Figure 6.31, shows that the data-mining bias in-creases to a +48 percent for the best-of-400 ATRs.
The relationship between number of ATRs tested and the data-miningbias can be seen in Figure 6.32. It summarizes the results obtained for
 
 
 
 
 
 
 
 
 
 
22% “\ Observed
Expected Return
Return 0% Best
Best ATR
ATR Of
10
RelativeFreq.Data MiningBias+22%
 
 
 
-70 -60 -50 -40 -30-20-10 Q 10 20 30 40 50 60 70
FIGURE 6.29 Sampling distribution—mean return best of 10 ATRs.
 
 
 
 
 
 
 
 
 
 
 
ObservedExpected ReturnReturn | Sf a0 %] BestBest ATRATR Of50RelativeFreq.Data MiningBias433% [oo
 
 
 
 
 
-70 -60 -50 -40 -30-20-10 Q 10 20 30 40 50 60 70
FIGURE 6.30 Sampling distribution—mean return best of 50 ATRs.Data-Mining Bias: The Fool’s Gold of Objective TA 297
 
 
 
 
 
 
 
 
 
 
 
ObservedExpected ReturnReturn 0% BestBest ATRATR Of400RelativeFreq.Data MiningBias+48% NK | 4
 
 
 
 
 
-70 -60 -50 -40 -30-20-10 Q 10 20 30 40 50 60 70
FIGURE 6.31 Sampling distribution—mean return best of 400 ATRs.
 
 
 
 
60 %50%
D
M 40%
B 30%
