rived at by multiplying V by 10 different coefficients; 1.5, 2.0, 2.5, 3.0,3.5, 4.0, 4.5, 5.0, 5.5, 6.0. This resulted in 10 zigzag filters with varying sen-sitivity. The validity of the set of coefficients chosen by C&O were con-firmed by visual inspection of price charts by TA practitioners whoagreed that the 10 zigzag filters did a reasonable job of identifying head-and-shoulders patterns.
Next C&O addressed the problem of defining rules that qualify a can-didate pattern as a valid head-and-shoulders. These rules were applied tothe instrument once its price had been zigzag filtered and the peaks andtroughs at a given scale had been identified.The Scientific Method and Technical Analysis 155
First, the head and shoulders identified by C&O’s algorithm had to sat-isfy the following basic rules:
1. The head of the pattern must be higher than both the left and rightshoulders.
2. The instrument must be in an uptrend prior to the formation of thehead-and-shoulders pattern. Thus, the pattern’s left shoulder has to beabove the prior peak (PP) and the pattern’s left trough has to be abovethe prior trough (PT).
Next C&O grappled with more subtle and complex issues to qualify acandidate pattern as a valid head-and-shoulders. They accomplished thiswith a set of innovative measurements that qualified the pattern in termsof its vertical and horizontal symmetry and the time it took for the patternto complete. These rules allowed them to definitively label a candidatepattern as either head-and-shoulders or a non-head-and-shoulders.
Vertical Symmetry Rules
The vertical symmetry rules exclude patterns with necklines that are toosteeply sloped. The pattern in Figure 3.9 has acceptable vertical symmetry.
The rules compare the price levels of the right and left shoulders (Aand E) and the price levels of the right and left troughs (B and D) with aprice level defined by the midpoint of segment AB, designated as pointX, and the midpoint of segment DE designated as point Y. To qualify as a
Price
  
           
-fercal. E>X&D<XRule” A>Y&B<Y Time
FIGURE 3.9 Good vertical symmetry.156 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS.
Price
 
Time
FIGURE 3.10 > Poor vertical symmetry—neckline slope too steep.
vertically symmetrical head-and-shoulders, the pattern must satisfy thefollowing rules.
1. The price level of the left shoulder peak, point A, must exceed theprice level of point Y.
2. The price level of the right shoulder peak, point E, must exceed theprice level of point X.
3. The price level of the left trough, point B, must be less than the pricelevel of point Y.
4. The price level of the right trough, point D, must be less than the pricelevel of point X.
Figures 3.10 and 3.11 show two head-and-shoulders patterns thatwould be excluded because the vertical symmetry criteria have not beensatisfied.
Horizontal Symmetry Rule
Another feature used by C&O to distinguish head-and-shoulders patternsfrom non-head-and-shoulders patterns was horizontal symmetry. A pat-tern with good horizontal symmetry is one for which the head, at point C,is roughly equidistant from the peaks representing the pattern’s two shoul-ders (points A and E). C&O’s rule was that the distance from the head toThe Scientific Method and Technical Analysis
Price
157
 
Time
FIGURE 3.01 Poor vertical symmetry—neckline slope too steep.
one shoulder should not be greater than 2.5 times the distance of the headto the other shoulder. There is nothing magical about the value 2.5 otherthan it seemed reasonable. See Figure 3.12.
Figure 3.13 is an example of a pattern that fails the test for horizontalsymmetry. Note the right shoulder is stretched too far to the right. A pat-tern with excessive leftward stretch would also be disqualified.
Price
 
   
HorizontalSymmetryRule
 
D,<2.5°D,D,< 2.5+D,
 
 
FIGURE 3.12 Good horizontal symmetry.
  
Time158 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
Price
 
Time
FIGURE 3.13 > Poor horizontal symmetry.
Pattern Completion Rule: Maximum Timeto Neckline Penetration
C&O also specified a rule that excludes patterns that take too long to pen-etrate the neckline once the right shoulder, point E, has been formed. Aswith other features, this criterion is defined in terms of the pattern’s inter-nal proportions rather than some fixed number of time units. This allowsthe rule to be applied to all patterns irrespective of their time frame orscale.
The maximum time allowed for the movement from the right shoul-der, point E, until the penetration of the neckline is the temporal distanceseparating the two shoulders, points A and E. The pattern in Figure 3.14meets the time to completion criterion because the temporal distancefrom the right shoulder until neckline penetration (D4) is less than thetemporal separation of the shoulders (D3). The pattern in Figure 3.15 doesnot qualify because D4 exceeds D3.
Future Information Leakage: Look-Ahead Bias
In their simulation of head-and-shoulders patterns, C&O took precautionsagainst the future information leakage or look-ahead bias problem. Thisproblem afflicts back tests that assume the possession of knowledge thatwas not truly available when a trading decision was made. In the contextof back testing, this can make results appear more profitable than wouldbe possible in actual trading. An extreme example would be assuming ac-cess to the Wall Street Journal the day before its publication.The Scientific Method and Technical Analysis 159
 
      
Price Completion D<D.
Cc Rule aoe
 
 
 
A. Neck Line----
Time
FIGURE 3.14 Pattern completion rule satisfied.
In the context of back testing the head-and-shoulders pattern, futureinformation leakage can occur if the zigzag threshold percentage is largerthan the percentage distance between the right shoulder peak and theneckline. It would not be legitimate to assume a short-sale signal due to aneckline penetration until the right shoulder (point E) has been detected.However, it is possible for prices to cross the neckline before the rightshoulder is identified by the zigzag filter. To clarify, suppose the rightshoulder, point E, lies only 4 percent above the neckline, but the threshold
Price
 
Time
FIGURE 3.15 Pattern completion rule not satisfied.160 > METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
for the zigzag filter is 10 percent. It would not be legitimate to assume thata trader knew about the neckline penetration because a 10 percent de-cline from the right shoulder would have been required to identify theright shoulder of the head-and-shoulders pattern. In this example, assum-ing an entry price only 4 percent below the right shoulder would be morefavorable than waiting for a price 10 percent below the right shoulder, theprice movement required to have a fully formed head-and-shoulders pat-tern. To avoid this problem, C&O assumed an entry after the right shoul-der had been objectively identified by the zigzag filter. Though this was aless favorable entry price in some instances, the back test was unencum-bered by look-ahead bias. I mention this because it shows C&O’s attentionto detail.
C&O’s pattern definition also deals with numerous other issues re-garding entry and exit of positions, stop-loss levels, and so forth, but thepoint has been made that it is possible to transform subjective chart pat-terns into objective testable patterns. Readers may take exception to arbi-trary aspects of C&O’s pattern. All well and good if the reader has a betterobjective definition to offer.
As a final sanity check, C&O showed patterns that had been identifiedby their automated head-and-shoulders algorithm to a number ofchartists. C&O claim that the chartists agreed that the patterns identifiedby the objective head-and-shoulders algorithm did indeed conform to sub-jective head-and-shoulders criteria.
Head-and-Shoulders Back Test Results
Does the head-and-shoulders pattern carry predictive information with re-spect to stocks or currencies? In a word, the pattern hailed as a corner-stone of charting is a bust. Tests by C&O show that it is worthless onstocks and only modestly profitable on currencies. The pattern was prof-itable in two out of six currencies tested, but the relatively complicatedhead-and-shoulders algorithm was far outperformed by a much simplerobjective signal based on zigzag filters. Moreover, when C&O tested theoccurrence of a head-and-shoulders in conjunction with the zigzag rule ei-ther as a confirming or disconfirming signal, the head-and-shoulders pat-tern added no value. In other words, zigzag signals did no better when thehead-and-shoulders signal was in the same direction as the zigzag signaland the zigzag did no worse when the head-and-shoulders signal was inthe opposite direction.The bottom line for currency traders: the value ofhead-and-shoulders is doubtful.”
The head-and-shoulders performed worse on stocks. C&O evaluatedthe head-and-shoulders on 100 randomly selected equities over the pe-The Scientific Method and Technical Analysis 161
riod from July 1962 until December 1993. On average, each stock gave onehead-and-shoulders signal per year, counting both long and short signals,giving a sample of over 3,100 signals. To test the pattern’s profitability onactual stock prices, C&O established a benchmark based on the pattern’sperformance on pseudo-price histories. These simulated price historieswere generated from actual historical price changes strung together in arandom fashion. By using actual price changes, the pseudo-price historieshad the same statistical characteristics as real stocks, but any predictabil-ity due to authentic temporal structure—the structure TA patterns are in-tended to exploit—was eliminated Despite the fact that the pseudo-pricehistories were randomly generated, head-and-shoulders patterns that fitC&O’s definition still emerged. This confirms the results of Harry Robertsreferred to earlier.
If the head-and-shoulders patterns appearing in real stock data areuseful, they should generate profits superior to those achieved by tradingthe patterns appearing in the fake stock price histories. C&O found thathead-and-shoulders patterns in actual stock prices lost slightly moremoney than the signals on pseudo-price histories. According to the study“the results uniformly suggest that head-and-shoulders trading are notprofitable.” The signals lose on average about 0.25 percent over a 10-dayholding period. This compares with an average loss of 0.03 percent forhead-and-shoulders patterns in the pseudo-stock data. C&O referred totraders using the pattern as “noise traders,” speculators who mistake arandom signal for an informative one.
Confirming C&O’s findings is the work of Lo et al."* Lo used an alter-native method of objectifying the H&S pattern based on kernel regression,a sophisticated local” smoothing technique. Their study was unable to un-seat the null hypothesis that the head-and-shoulders pattern is useless.*!
Bulkowski®™ found the head-and-shoulders was profitable, but his re-search falls short. He does not provide an objective pattern definition thatback-testable pattern or entry and exit rules. In other words, his study issubjective TA. In addition, his results fail to adjust for the trend of the gen-eral stock market over the time period in which he tested the patterns.
SUBSETS OF TA
 
Given the preceding discussion, TA can be seen as comprised of four sub-sets: (1) subjective TA, (2) objective TA with unknown statistical signifi-cance, (3) objective TA that is not statistically significant, and (4)objective TA that is statistically significant.162 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
The first subset, subjective TA, has already been defined as methodsthat cannot be reduced to a back-testable algorithm. Subsets two, three,and four refer to objective methods.
The second subset is comprised of objective methods of unknownvalue. Though these methods are objective and may have been backtested, their results have not been evaluated for statistical significance.Here, it is simply a matter of applying the methods discussed in Chapters4, 5, and 6. This is not meant to suggest that applying statistical methodsto back test results is simple, but rather that the decision to do so is.
The third subset, which I refer to as useless TA, consists of objectiveTA rules for which results have been comprehensively back tested andevaluated with statistical methods, but have been reveled to add no valueeither on a stand-alone basis or when used in combination with othermethods. In all likelihood, the majority of objective TA methods will fallinto this subset. This is to be expected because financial markets are ex-tremely difficult to predict due to their inherent complexity and random-ness. In fact, in all fields of science, most proposed ideas do not work.Important discoveries are rare. This is not obvious because the numerousfailures are typically not reported in the lay press or even in scientificjournals. What is most important is having a procedure in place for gettingrid of methods that do not work.
The fourth subset, useful TA, consists of objective methods that pro-duce statistically significant or, better yet, economically significant re-sults. Though some rules will be useful on a stand-alone basis, thecomplexity and randomness of financial markets make it likely that most
NO Subjective
      
  
YESObjectiveRIGOROUSNO—»> ValueEVALUATION UnknownYES
 
 
 
 
 
Objective ObjectiveValuable _[*— YES Kennet NO—> | No Value
FIGURE 3.16 Subsets of Technical Analysis.The Scientific Method and Technical Analysis 163
rules will add value when used in combination with other rules to formcomplex rules.
Evidence-based technical analysis (EBTA) refers to subsets (3) and(4)—objective TA that has been back tested and subjected to statisticalanalysis. Given the preceding discussion, the categorization of TA is illus-trated in Figure 3.16.
The next three chapters discuss the application of statistical analysisto back-test results.Statistical
Analysis
 
 
renowned British scientist and author H.G. Wells (1866-1946) said
that an intelligent citizen in a twentieth-century free society wouldneed to understand statistical methods. It can be said that an intelligenttwenty-first-century practitioner or consumer of TA has the same need.This chapter and the next two address aspects of statistics that are partic-ularly relevant to TA.
Statistical methods are not needed when a body of data conveys amessage loudly and clearly. If all people drinking from a certain well dieof cholera but all those drinking from a different well remain healthy,there is no uncertainty about which well is infected and no need for sta-tistical analysis. However, when the implications of data are uncertain,statistical analysis is the best, perhaps the only, way to draw reasonableconclusions.
Identifying which TA methods have genuine predictive power ishighly uncertain. Even the most potent rules display highly variable per-formance from one data set to the next. Therefore, statistical analysis isthe only practical way to distinguish methods that are useful from thosethat are not.
Whether or not its practitioners acknowledge it, the essence of TA isstatistical inference. It attempts to discover generalizations from histori-cal data in the form of patterns, rules, and so forth and then extrapolatethem to the future. Extrapolation is inherently uncertain. Uncertainty isuncomfortable.
The discomfort can be dealt with in two ways. One way is to pretend
Sion is the science of data.! In the late nineteenth century,
165166 > METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
it does not exist. The other is the way of statistics, which meets uncer-tainty head on by acknowledging it, quantifying it, and then making thebest decision possible in the face of it. Bertrand Russell, the renownedBritish mathematician and philosopher said, “Uncertainty, in the presenceof vivid hopes and fears, is painful, but must be endured if we wish to livewithout the support of comforting fairy tales.””
Many people are distrustful or disdainful of statistical analysis andstatisticians are often portrayed as nerdy number-crunching geeks di-vorced from reality. This shows up in jokes. We deride what we do not un-derstand. There is the story about the six-foot-tall man who drowns in apond with an average depth of only two feet. There’s the tale about threestatisticians who go duck hunting. They spot a bird flying overhead. Thefirst shoots a foot too far to the left. The second shoots a foot too far tothe right. The third jumps up and exclaims, “We got it!!” Even though theaverage error was zero, there was no duck for dinner.
Powerful tools can be put to bad purpose. Critics often charge thatstatistics are used to distort and deceive. Of course, similar ends can beachieved with words, although language is not held liable. A more rationalstance is needed. Rather than viewing all claims based on statistics withsuspicion or taking them all at face value, “a more mature response wouldbe to learn enough about statistics to distinguish honest, useful conclu-sions from skullduggery or foolishness.”’ “He who accepts statistics indis-criminately will often be duped unnecessarily. But, he who distrustsstatistics indiscriminately will often be ignorant unnecessarily. The middleground we seek between blind distrust and blind gullibility is an open-minded skepticism. That takes an ability to interpret data skillfully.”
A PREVIEW OF STATISTICAL REASONING
 
Statistical reasoning is new terrain for many practitioners and consumersof TA. Trips to strange places are easier when you know what to expect.The following is a preview of the next three chapters
For reasons discussed in Chapter 3, it is wise to start with the as-sumption that all TA rules are without predictive power and that a prof-itable back test was due to luck. This assumption is called the nullhypothesis. Luck, in this case, means a favorable but accidental corre-spondence between the rule’s signals and subsequent market trends in thehistorical data sample in which the rule was tested. Although this hypoth-esis is a reasonable starting point, it is open to refutation with empiricalevidence. In other words, if observations contradict predictions made bythe null hypothesis, it is abandoned and the alternative hypothesis, thatStatistical Analysis 167
the rule has predictive power, would be adopted. In the context of ruletesting, evidence that would refute the null hypothesis is a back-testedrate of return that is too high to be reasonably attributed to mere luck.
Ifa TA rule has no predictive power, its expected rate of return will bezero on detrended’ data. However, over any small sample of data, the prof-itability of a rule with no predictive power can deviate considerably fromzero. These deviations are manifestations of chance—good or bad luck.This phenomenon can be seen in a coin-toss experiment. Over a smallnumber of tosses, the proportion of heads can deviate considerably from0.50, which is the expected proportion of heads in a very large number oftosses.
Generally, the chance deviations of a useless rule from a zero returnare small. Sometimes, however, a useless rule will generate significantprofits by sheer luck. These rare instances can fool us into believing a use-less rule has predictive power.
The best protection against being fooled is to understand the degreeto which profits can result from luck. This is best accomplished with amathematical function that specifies the deviations from zero profits thatcan occur by chance. That is what statistics can do for us.
This function, called a probability density function, gives the probabil-ity of every possible positive or negative deviation from zero. In otherwords, it shows the degree to which chance can cause a useless rule togenerate profits. Figure 4.1 shows a probability density function.® The factthat the density curve is centered at a value of zero reflects the null hy-pothesis assertion that the rule has an expected return of zero.
ProbabilityDensity
 
 
Negative Returns 0 Positive ReturnsBack-Tested Rate of Return
FIGURE 4.1 Probability density of chance performance—range of possible per-formance for a useless TA rule.168 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
In Figure 4.2, the arrow indicates the positive rate of return earned bya rule when it was back tested. This raises the question: Is the observedrate of return sufficiently positive to warrant a rejection of the null hypoth-esis that the rule’s true rate of return is zero? If the observed performancefalls well within the range of the deviations that are probably attributableto chance, the evidence is considered to be insufficient to reject the nullhypothesis. In such a case, the null hypothesis has withstood the empiricalchallenge of the back-test evidence, and a conservative interpretation ofthe evidence would suggest that the rule has no predictive power.
The strength of the back-test evidence is quantified by the fractionalarea’ of the probability density function that lies at values equal to orgreater than the rule’s observed performance. This portion of the densityfunction is depicted by the darkened area to the right of the vertical arrowin Figure 4.2. The size of this area can be interpreted as the probabilitythat a rate of return this high or higher could have occurred by chance un-der the condition that the rule has no predictive power (expected return =0, or the null hypothesis is true). When this area occupies a relatively largefraction of the density curve, it means that there is an equivalently largeprobability that the positive performance was due to chance. When this isthe case, there is no justification for concluding that the null hypothesis isfalse. In other words, there is no justification for concluding that the ruledoes have predictive power.
However, if the observed performance is far above zero, the portionof the probability density function lying at even more extreme values is
 
 
 
 
 
Rate of returna this low falls wellProbability within the range ofDensity L\ chance performanceNegative Returns 0 Positive Returns
Back-Tested Rate of Return
FIGURE 4.2 Probability of chance performance for a useless rule.Statistical Analysis 169
small. Performance this positive would be inconsistent with the assertionthat the rule has no predictive power. In other words, the evidence wouldbe sufficient to refute the null hypothesis. Another way to think of this isas follows: If the null hypothesis were true, a level of performance thispositive would have a low probability of occurrence. This probability isquantified by the proportion of the density function that lies at valuesequal to or greater than the observed performance. This is illustrated inFigure 4.3. Note that the observed performance lies in the extreme righttail of the density curve that would pertain if the rule were devoid of pre-dictive power.
It is important to understand what this evidence does not tell us. Ittells us nothing about the probability that either the null hypothesis or thealternative hypothesis is true. It only speaks to the probability that the ev-idence could have occurred under the assumption that the null hypothesisis, in fact, true. Thus, the probability speaks to the likelihood of the evi-dence, not the likelihood of the truth of the hypothesis. Observed evi-dence that would be highly improbable, under the condition that the nullhypothesis is true, permits an inference that the null hypothesis is false.
Recall that, in Chapter 3, it was shown that evidence that a creaturehas four legs cannot conclusively establish the truth of the hypothesis:The creature is a dog. Although evidence of four legs would be consistentwith the hypothesis that the creature is a dog, it is not sufficient to prove,deductively, that the creature is a dog. Similarly, while the observation ofpositive performance would be consistent with the hypothesis that a rulehas predictive power, it is not sufficient to prove that it does. An argumentthat attempts to prove the truth of a hypothesis with observed evidence
 
 
 
 
 
ProbabilityDensity Rate of returnthis highunlikely tobe due to chanceNegative Returns 0 Positive Returns
Back-Tested Rate of Return
FIGURE 4.3. Probability of chance performance for a good rule.1°70 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
that is consistent with the hypothesis commits the logical fallacy of affirm-ing the consequent.
If the creature is a dog, then it has four legs.The creature has four legs.Invalid Conclusion: Therefore, the creature is a dog.
If a rule has predictive power, then it will have a profitable back test.Back test was profitable.Invalid Conclusion: Therefore, rule has predictive power.
However, the absence of four legs is sufficient to prove that the hy-pothesis, the creature is a dog, is false.* In other words, observed evidencecan be used to conclusively prove that a hypothesis is false. Such an argu-ment uses the valid deductive form, denial of the consequent. The generalform of an argument, in which the consequent is denied, is as shown:
If P is true, then Q is true.Q is not true.Valid Conclusion: Therefore, P is not true (1.e., P is false).
If the creature is a dog, then it has four legs.Creature does not have four legs.Valid Conclusion: Therefore, it is false that the creature is a dog.
The argument just given uses the evidence, the absence of four legs,to conclusively falsify the notion that the creature is a dog. However, thislevel of certitude is not possible in matters of science and statistics. Onecan never conclusively falsify a hypothesis. Nevertheless, a similar logiccan be used to show that certain evidence is highly unlikely if the hypoth-esis were true. In other words, the evidence gives us grounds to challengethe hypothesis. Thus, a highly profitable back test can be used to chal-lenge the hypothesis that the rule has no predictive power (i.e., that it hasan expected return of zero).
If a rule’s expected return is equal to zero or less, then a back testshould generate profits that are reasonably close to zero.
The back-tested performance was not reasonably close to zero; infact, it was significantly above zero.
Valid Conclusion: Therefore, the contention that the rule’s expectedreturn is equal to zero or less is likely to be false.Statistical Analysis 171
How unlikely or rare must the positive performance be to reject thenotion that the rule is devoid of predictive power? There is no hard andfast rule. By convention most scientists would not be willing to reject ahypothesis so unless the observed performance has a 0.05 probability orless of occurrence under the assumption the null is true. This value iscalled the statistical significance of the observation.
The discussion so far pertains to the case where only one rule is backtested. In practice, however, TA rule research is typically not restricted totesting a single rule. Economical computing power, versatile back-testingsoftware, and plentiful historical data make it easy, almost inviting, to testmany rules with the aim if selecting the one with the best performance.This practice is known as data mining.
Although data mining is an effective research method, testing manyrules increases the chance of a lucky performance. Therefore, the thresh-old of performance needed to reject the null hypothesis must be sethigher, perhaps much higher. This higher threshold compensates for thegreater likelihood of stumbling upon a useless rule that got lucky in aback test. This topic, the data mining bias, is discussed in Chapter 6.
Figure 4.4 compares two probability density functions. The top onewould be appropriate for evaluating the significance for a single rule backtest. The lower density curve would be appropriate for evaluating the sig-
 
Appropriate Distributionfor Single-Rule Test
 
 
 
 
 
 
 
 
 
     
erated Observed Performancey Looks Significant0Appropriate Distribution for Best of 1000 RulesObserved PerformanceProbability Does Not LookDensity Significant
 
 
 
0)
Back-Tested Rate of Return
FIGURE 4.4 Great performance in a single rule back test is only mediocre when1,000 rules are tested.172 METHODOLOGICAL, PSYCHOLOGICAL, PHILOSOPHICAL, STATISTICAL FOUNDATIONS
nificance of the best-performing rule out of 1,000 back-tested rules. Thisdensity curve takes into account the increased likelihood of luck that re-sults from data mining. Notice that if this best rule’s observed perfor-mance were to be evaluated with the density curve appropriate for asingle rule, it would appear significant because the performance is far outin the right tail of the distribution. However, when the best rule’s perfor-mance is evaluated with the appropriate probability density function itdoes not appear statistically significant. That is to say, the rule’s ratherhigh performance would not warrant the conclusion that it has predictivepower or an expected return that is greater than zero.
THE NEED FOR RIGOROUS STATISTICAL ANALYSIS
 
The tools and methods of a discipline limit what it can discover. Improve-ments in them pave the way to greater knowledge. Astronomy took a greatleap forward with the invention of the telescope. Though crude by today’sstandards, the earliest instruments had 10 times the resolving power of theunaided eye. Technical analysis has a similar opportunity, but it must re-place informal data analysis with rigorous statistical methods.
Informal data analysis is simply not up to the task of extracting validknowledge from financial markets. The data blossoms with illusory pat-terns whereas valid patterns are veiled by noise and complexity. Rigorousstatistical analysis is far better suited to this difficult task.
Statistical analysis is a set of well-defined procedures for the collec-tion, analysis, and interpretation of data. This chapter and the next twowill introduce the way statistical tools and reasoning can be used to iden-tify TA rules that work. This overview is necessarily condensed, and inmany instances I have sacrificed mathematical rigor for the sake of clarity.However, these departures do not dilute the essential message: If TA is todeliver on its claims, it must be grounded in a scientific approach thatuses formal statistical analysis.
AN EXAMPLE OF SAMPLING ANDSTATISTICAL INFERENCE
 
Statistical reasoning is abstract and often runs against the grain of com-mon sense. This is good and bad. Logic that runs counter to informal in-ference is good because it can help us where ordinary thinking lets usdown. However, this is exactly what makes it difficult to understand. Sowe should start with a concrete example.Statistical Analysis 173
The central concept of statistical inference is extrapolating from sam-ples. A sample of observations is studied, a pattern is discerned, and thispattern is expected to hold for (extrapolated to) cases outside the ob-served sample. For example, a rule observed to be profitable in a sampleof history is projected to be profitable in the future.
Let’s begin to think about the concept in the context of a problem thathas nothing to do with technical analysis. It comes from an excellentbook: Statistics, A New Approach by Wallis and Roberts.° The problemconcerns a box filled with a mixture of white and grey beads. The totalnumber of beads and the numbers of grey and white beads are unknown.The task is to determine the fraction of beads that are grey in the entirebox. For purposes of brevity, this value will be designated as F-G (frac-tion-grey in box).
To make this situation similar to statistical problems faced in the realworld, there is a wrinkle. We are not allowed to view the entire contentsof the box at one time, thus preventing a direct observation of F-G. Thisconstraint makes the problem realistic because, in actual problems, ob-serving all the items of interest, such as all beads in the box, is either im-possible or impractical. In fact, it is