CORRELATIONCorrelation is another important descriptive statistic. It measures the strength of arelationship between two series. Correlation can range from –1 to +1, where theextreme values indicate a perfect relationship, while a value of zero indicates norelationship at all. The correlation statistic is calculated by multiplying the difference of one series from its mean by the corresponding difference of another seriesfrom its mean, taking the average product, and then dividing by the product of thestandard deviation of both series. The equation looks like this:1ᎏᎏ ∑ (x ⫺ ␮x)(y ⫺ ␮y)NCorrelation = ␳ = ᎏᎏᎏ␴x␴yThe best way to think about correlation is by graphing a scatterplot of twoseries. A scatterplot graphs values of one series against values of another series, asyou can see in the figures below.
CHAPTER 2 An Introduction to Statistics
43
Uncorrelated Values2.502.001.50
Value of y
1.000.500.00–3.00
–2.00
–1.00
0.00–0.50.
1.00
2.00
3.00
–1.00–1.50–2.00Value of x
FIGURE
2.4
Uncorrelated Values. When the value of y is plotted against the value of x, we see that no relationshipexists.
These graphs present three general states of correlation. In Figure 2.4, the scatterplot suggests that the relationship between x and y is random, and thus no correlation exists. In Figure 2.5, y increases as x does. This indicates a positive correlationbetween x and y. In Figure 2.6, a different relationship exists. As x increases, ydecreases, indicating negative correlation.Correlation can also be misused. When measuring the correlation of marketdata, it is important to perform your analysis using returns rather than prices. In mostinstances, the correlation of prices can trick the trader into believing there is a meaningful relationship between two series, when in reality no such pattern exists.Consider, for example, the relationship between the S&P 500 and naturalgas. When we create a scatterplot of these two data series, a strong positive relationship is observed. The correlation in prices is 0.70, which would indicate astrong relationship between natural gas and the S&P 500 (see Figure 2.7). If twoseries drift in similar or opposite directions over the life of the data, however, suchas they do in this example, the correlation numbers will become “artifacts.” Usedin the statistical sense, the term artifact refers to a result that is misleading andprobably biased. In fact, if in our example we calculate the correlation of weeklyreturns instead of prices (Figure 2.8), we see that the correlation is nearly zero.This correlation number is a more accurate representation of the “true” relationship between natural gas and the S&P 500 than the 0.70 price correlation.
Positively Correlated Values4.003.00
Value of y
2.001.000.00–2.50
–2.00
–1.50
–1.00
–0.500.00–1.00
0.50
1.00
1.50
2.00
–2.00–3.00–4.00Value of x
FIGURE
2.5
Positively Correlated Values. When the value of y is plotted against the value of x, we see that a positiverelationship exists.
Negatively Correlated Values4.003.00
Value of y
2.001.000.00–2.50
–2.00
–1.50
–1.00
–0.50
0.00–1.00
0.50
1.00
1.50
2.00
–2.00–3.00Value of x
FIGURE
2.6
Negatively Correlated Values. When the value of y is plotted against the value of x, we see that a negativerelationship exists.
44
Scatterplot: S&P 500 versus Natural Gas2500
S&P 500
2000
1500
1000
500
00
2
4
6
8
10
12
Natural Gas
FIGURE
2.7
Scatterplot: S&P 500 versus Natural Gas Prices. While statistical analysis would show a significant relationship between the level of the S&P 500 and natural gas prices, chances are this relationship will nothold true in the future.
Scatterplot: S&P 500 Returns versus Natural Gas Returns0.1
Returns of S&P 500
0.05
0–0.4
–0.3
–0.2
–0.1
0
0.1
0.2
0.3
0.4
–0.05
–0.1
–0.15Returns of Natural Gas
FIGURE
2.8
Scatterplot: S&P 500 Returns versus Natural Gas Returns. When weekly percent price changes are examined, we see that virtually no link exists between the S&P 500 and natural gas prices.
PART 1 Structural Foundations for Improving Technical Trading
46
Stocks
Weekly correlation of returns
Ford/General Motors
0.73
Merrill Lynch/Morgan Stanley
0.80
Merck/Pfizer
0.65
FIGURE
2.9
Correlation of Selected Stock Pairs. Companies in the same industry often have large positivecorrelations.
S&P 500
Gold
S&P 500
1.00
0.10
0.17
0.03
Gold10-year Treasury yields
0.100.17
1.00–0.17
–0.171.00
0.07–0.20
Oil
0.03
0.07
–0.20
1.00
FIGURE
10-year yields
Oil
2.10
Correlation of Macroeconomic Variables. Correlation among equities, bonds, and commodities is not ashigh as correlation among stocks.
Some markets are highly correlated, while others seem to have no effect on oneanother. Stocks of the same industry typically have very high correlations. Thecorrelation of weekly returns for selected pairs of stocks from 2001 is listed inFigure 2.9.Interest rates, gold, and the stock market have smaller correlations witheach other. For example, as can be seen in Figure 2.10, correlations betweenthese four markets run from a low of –0.20 for oil and 10-year yields, to a highof 0.17 correlation between S&P 500 returns and changes in 10-year yields.
THE USEFULNESS OF THE NORMAL DISTRIBUTIONStandard deviation is a popular method of measuring dispersion, primarily dueto its properties under certain circumstances, specifically those associated witha normal distribution. When we say that a distribution is “normal,” we’re making a statement about the probabilities of values occurring. Normal distributions pile high toward the center of the distribution and fan out toward theedges.The normal distribution is sometimes referred to as a bell curve, due to itsshape (as seen in Figure 2.11). Probably the most useful and most studied distribution in statistics, it’s also referred to as “Gaussian,” in honor of the Germanmathematician Karl Freidrich Gauss.The equation for determining normal distribution looks like this:
CHAPTER 2 An Introduction to Statistics
47
Normal Distribution2.50%
Probability
2.00%
1.50%
1.00%
0.50%
0.00%–3.00 –2.50 –2.00 –1.50 –1.00 –0.50 0.00
0.50
1.00
1.50
2.00
2.50
3.00
Standard Deviations from Mean
FIGURE
2.11
Normal Distribution. The normal distribution looks like a bell, often leading to its “bell shaped” nickname.
– (x ⫺ ␮)2
ᎏ12p (x) = ᎏ e ␴␴2␲
Once we know that a series follows a normal distribution, we can infertremendous information about the range of values using only the mean and standard deviation of the series. For example, we know that roughly 68.26 percent ofthe values in a normal distribution fall between ±1 standard deviation of the mean,95.44 percent fall between ±2 standard deviations, and 99.74 percent between ±3standard deviations.A quick exercise in Microsoft Excel shows the value of the normal distribution. Generate 1000 normally distributed random values by entering =NORMSINV(RAND()) into empty cells. Next, calculate the mean and standard deviation.In my run, I find a mean of 0.01 and a standard deviation of 0.99. Using the properties of the normal distribution, I know the following:68.26 percent of the values should fall between ±1 standard deviation:–0.99 and +1.0095.44 percent should fall between ±2 standard deviations: –1.98 and +1.9999.74 percent should fall between ±3 standard deviations: –2.97 and +2.98
PART 1 Structural Foundations for Improving Technical Trading
48
When I sort my values in Excel, I find that my distribution closely matchesthe normal, as in Figure 2.12.Market returns and prices follow some very specific tendencies associated withthe normal distribution. These tendencies are constant regardless of product—stocks,futures, and currencies all have these properties. Over short time periods, percentagemarket returns (dividing today’s price by yesterday’s price and subtracting the valueof one) are roughly normally distributed, following the popular bell shape normalcurve. Figure 2.13, below, depicts the daily return distribution for the S&P 500 overthe last five years. The bars represent the actual return distributions, while the smoothline is the expected distribution based on a perfect normal distribution.Standard deviation range
Expected
Actual
⫺1 to +1⫺2 to +2⫺3 to +3
68.26%95.44%99.74%
68.80%95.70%100%
FIGURE
2.12
Excel Exercise. By pulling random numbers into Excel, we see that their range is almost exactly what wewould expect given from the normal distribution.
S&P 500 Return Distribution6.00%
Percent Occurrence
5.00%
S&P 500 Daily ReturnsNormal Distribution
4.00%3.00%2.00%1.00%
4.00
3.50
3.00
2.50
2.00
1.50
1.00
0.50
0.00
–0.50
–1.00
–1.50
–2.00
–2.50
–3.00
–3.50
–4.00
0.00%
Standard Deviations Away from the Mean
FIGURE
2.13
S&P 500 Return Distribution. Daily returns of the S&P 500 between 1997 and 2001 look similar to a normaldistribution, except for more than expected occurrences near the middle and tails of the return distribution.
CHAPTER 2 An Introduction to Statistics
49
THE IRREGULARITY OF MARKET VOLATILITYAs seen above, returns follow the normal curve very closely. The small departuresinclude high peaks near the middle of the distribution and fat tails toward eachedge. This distribution is referred to as “leptokurtic” from the Greek word leptos,meaning small or narrow. The fact that short-term market returns do not follow atypical normal distribution had plagued economists for some time. Recent evidence has shown that the leptokurtic distribution likely results from the fact thatthe standard deviation of returns varies over time.Traders may have a better understanding of nonconstant market volatilitythan economists. When the market is slow, quiet, and no new news is hitting thetape, the market tends to remain in a state of low volatility for days or weeks. Onthe other hand, when markets are very volatile, they tend to take days or weeks offurther above average volatility until news and price changes are digested.Economists have created a new class of models to explain how volatilityvaries over time. These models are called GARCH, for Generalized AutoRegressive Conditional Heteroskedasticity. A GARCH process exists whenvolatility itself changes over time, wandering back and forth around a long-termaverage. In the Figure 2.14, below, we see the standard deviation of daily returnsfor the S&P 500 over a one month period. We can measure the volatility over shorttime frames by calculating a rolling 20-day standard deviation of percent returns.
One Month Volatility: S&P 50050%
Volatility
40%
30%
20%
10%
Date
FIGURE
2.14
One Month Volatility: S&P 500. Volatility tends to bounce between 10% and 20%.
1/30/01
1/30/00
1/30/99
1/30/98
1/30/97
1/30/96
1/30/95
1/30/94
1/30/93
1/30/92
1/30/91
1/30/90
0%
PART 1 Structural Foundations for Improving Technical Trading
50
We start by calculating the standard deviation of the first 20 returns. On thenext day, we drop the first return from our calculation, add the 21st return, andrecalculate the standard deviation of returns. The following day we drop the second return and add the 22d return in our calculation of standard deviation. And soon. Each value of the 20-day standard deviation will have 19 common returnpoints as the value before and value after. In this sense, the calculation “rolls” witheach day, hence the expression “rolling volatility.”We must make an adjustment to our standard deviation calculation. While theresulting standard deviation is reported in daily units, it is much more common torefer to volatility on an annualized basis. Because volatility scales with the squareroot of time, we multiply our daily standard deviation by the square root of trading days in a year (typically 252 for equity markets). The result of this adjustmentis an annualized standard deviation, or volatility.Note that in the above graph of the S&P 500, volatility tends to meander backand forth around the mean of 18 percent. While periods of low volatility may existfor short periods of time, volatility eventually returns to the long-term mean.Periods of high volatility can also only last so long before returning to the longterm mean. Such mean reversion of standard deviation is the backbone of GARCHmodels. This property of volatility is common to almost all markets. Periods ofhigh volatility are often followed by further periods of high volatility, slowlydecreasing to more normal levels over time. Similarly, periods of low volatility areoften followed by further periods of low volatility, eventually returning to normallevels over time.Research has shown that these GARCH models explain the volatility of markets with strong statistical significance. In addition, academic studies have foundthat when returns are adjusted for this varying volatility, the high peaks near themiddle of the distribution and fat tails towards the extremes diminish and leave amore normal looking distribution. Although GARCH cannot or does not predictmarket returns or prices, the concept of nonconstant volatility has important implications for generating trading signals as well as managing the risk of portfolios.
THE RANGE OF VOLATILITIESIt’s important to understand the statistics behind market prices so we can accurately analyze market information. One important distinction among markets istheir varying degree of volatility. As already noted, markets do not possess constant volatility; rather, their volatilities are constantly changing over time. In theexample above, we see that annualized volatility of the S&P 500 is typicallybetween 15 and 25 percent, but also can spend long periods outside these ranges.In fact, the trend in the late 1990s was one of increasing volatility.Traders should have been aware that trading an equal dollar value of the S&P500 in the late 1990s was riskier than doing so in the early 1990s. Volatility of the
CHAPTER 2 An Introduction to Statistics
51
One Month Volatility: GE80%
Volatility
60%
40%
20%
1/30/01
1/30/00
1/30/99
1/30/98
1/30/97
1/30/96
1/30/95
1/30/94
1/30/93
1/30/92
1/30/91
1/30/90
0%
Date
FIGURE
2.15
One Month Volatility: GE. Volatility tends to bounce between 20% and 40%.
S&P 500, when measured by standard deviation of percent returns, increasedbetween 1995 and 2000. General Electric and Intel have similar volatility storiesas the S&P 500, as shown in Figures 2.15 and 2.16. Matching the rise in volatility of the S&P 500, the volatility of daily returns from GE and Intel (INTC)expanded between 1998 and 2000.If we know a market’s annualized volatility, we can calculate the daily risk ofbeing long or short. For example, the annualized volatility of the S&P 500 isaround 20 percent. The daily standard deviation of being long or short the S&P500 is equal to the value of the portfolio multiplied by the annualized standarddeviation divided by the square root of 252:(Notional $) (␴Annualized)Daily standard deviation = ᎏᎏᎏ252
We divided by the square root of 252 (the number of trading days in ayear) in the above equation to scale the volatility from annual to daily. Becausestandard deviation scales proportionately to the square root of time, we mustadjust volatility numbers to arrive at apples-to-apples comparisons. Suppose wehave $20 million long exposure to the S&P 500. Our daily standard deviationwould be:
PART 1 Structural Foundations for Improving Technical Trading
52
One Month Volatility: INTC150%
Volatility
120%
90%
60%
30%
1/30/01
1/30/00
1/30/99
1/30/98
1/30/97
1/30/96
1/30/95
1/30/94
1/30/93
1/30/92
1/30/91
1/30/90
0%
Date
FIGURE
2.16
One Month Volatility: INTC. Volatility tends to bounce between 30% and 60%.
($20,000,000)(20%)Daily standard deviation = ᎏᎏᎏ = $252,000
252In the example above, we can create what statisticians call “confidenceintervals” around the daily standard deviation of $252,000. Using the normaldistribution, 68 percent of the time we should make or lose between plus andminus one standard deviation ($252,000), and 95 percent of the time we shouldmake or lose between plus and minus two standard deviations ($504,000).Many traders have the misperceived notion that the S&P 500 is a veryvolatile market. Compared to most commodities and individual stocks, this is nottrue. As can be seen in the graph of annualized volatility for various markets from1997 to 2001 (Figure 2.17), it is the fifth least volatile market. When traders saythe S&P 500 futures are volatile, what they’re really talking about is the notionalsize of the futures contract compared to other futures contracts. The notional valueof the S&P 500 futures contract at the time of this writing is ($250)(800) =$200,000, whereas the notional value of crude oil, a more volatile market—asnoted in the graph—is only ($1000)(30) = $30,000.The difference between price volatility, as measured solely by daily percentage returns, and dollar volatility, measured by daily dollar returns of a futures con-
CHAPTER 2 An Introduction to Statistics
53
Annualized Volatility of Selected Markets70%60%
Volatility
50%40%30%20%10%
FIGURE
Natural Gas
INTC
Coffee
Nasdaq 100
Crude Oil
MO
GE
Cotton
Corn
S&P 500
Gold
Yen
T-Bonds
Canadian Dollar
0%
2.17
Annualized Volatility of Selected Markets. Commodities have some of the smallest volatility while individual stocks can be very volatile.
tract, will become important for determining the number of contracts to trade whenentering a market.The annualized volatility during 2001 for the 14 arbitrarily chosen commodities and markets in the graph shows that currencies such as the Canadian dollar and the Japanese yen are among the least volatile markets. Technology stockssuch as Intel and the Nasdaq 100 are among the most volatile.
THE LOGNORMALITY OF MARKET PRICESWhile short-term market returns can best be described as normally distributed,market prices follow a lognormal distribution. This is somewhat similar to the normal distribution in its shape, except the left- and right-hand side of the distributionis not symmetrical, as seen in Figure 2.18. The right-hand tail slowly extends outward, while the left-hand tails toward zero more abruptly. The lognormal distribution of prices is used commonly and is the basis for the Black-Scholes option pricing formula.Readers might question the link that causes short-term returns to follow anormal distribution but prices to follow a lognormal distribution. The answer liesin compounding of returns. Consider a $100 stock and two scenarios:
PART 1 Structural Foundations for Improving Technical Trading
54
Lognormal Distribution5.00%
4.00%
3.00%
2.00%
1.00%
0.00%0
FIGURE
5
10
15
20
2.18
Lognormal Distribution. The lognormal distribution has a longer right-hand side tail than the normaldistribution.
1. A gain of 50 percent on Day 1 and a gain of 50 percent on Day 22. A loss of 50 percent on Day 1 and a loss of 50 percent on Day 2At the end of the two-day period, the stock will finish at $225 = ($100)(1.5)(1.5)in the first example and $25 = ($100)(0.5)(0.5) in the second example. Scenario 1leads to a dollar return of +$125, while Scenario 2 leads to a dollar return of –$75.Though each scenario has a 25 percent chance of occurring, the former generates amore positive return than the latter does a negative return. As a result, the right tail ofa lognormal extends further from the mean than does the left tail.This characteristic will appear on several occasions throughout this book,and as we’ll see later, it has important implications for money management.
CHAPTER
3
Creating Trading StrategiesThe Building Blocks That Generate Trades
T
his chapter will provide an introduction to the concepts behind trading strategies.While the material may be old hack for some, the concepts are nevertheless important for building the better, more reliable systems and strategies we’ll examine later.There are three building blocks for creating any system: entries, exits, and filters. Entries are the signals that generate the opening buy and sell orders for newpositions. Exits indicate that the expected value of a trade has diminished to thepoint that the trade should be closed. Filters persuade the trader to only take theentries with highest expected profits over the life of a system.
THE NEED TO EXPLAIN PRICE CHANGESMy trading experience has uncovered some strange peculiarities on Wall Street.There seems to be a constant need to justify or explain every single movement inthe market. What caused the Dow to fall 25 points? Why are bonds up today?In some instances, such as earnings reports, economic news releases, or cropreports, the answers are in plain sight. But in many, there is no single driver of pricemovement. Such single pieces of information rarely cause large amounts of capitalto be allocated. Instead, it is usually a confluence of days or even weeks of information that change an investment philosophy from bearish to bullish or vice versa.To highlight the confirmation required in the investment process, considerthe following: A strong economic report might cause a mutual fund portfolio man55
Copyright 2003 by Lars Kestner. Click Here for Terms of Use.
PART 1 Structural Foundations for Improving Technical Trading
56
ager to consider buying stocks that may benefit from an upturn in the economy.The fund manager notices two companies in a cyclical industry that report betterthan expected earnings. Not yet convinced, the manager visits the companies’headquarters and also conducts channel checks with suppliers and distributors tosee how orders are tracking in the current quarter. Convinced one of the stocks isnow poised for higher prices, the fund manager begins to accumulate a large position in the stock. The buying pressure drives the company’s stock price up $2 onthe day.Was there any news on this specific trading day to explain the sharp rise inthe stock? No. It was a confluence of many pieces of information over severalweeks that caused the stock to rise.It’s very hard to explain the day-to-day movements of market prices. Ourtrading strategies should not try to do so since prices are noisy and probably bestexplained by the random walk model. Instead, we will focus on predicting the market’s daily direction correctly roughly 52 to 55 percent of the time. Even if we arecorrect only 52 percent of the trading days in the year, we should expect to makemoney 72 percent of the years when we trade. If we can increase the reliability ofour forecasts to 55 percent daily accuracy, we become 94 percent likely to makemoney in any given year. The point is, a small increase in daily accuracy goes along way in increasing our chances of being profitable for the year. The entries,exits, and filters below will aid in our trading decisions and increase our dailyaccuracy when trading.
TRADING STRATEGY ENTRIESEntry signals are the engines that drive trading systems. Often, the bulk of timespent on creating new systems is consumed while devising entry signals. With somany ideas to choose from, testing can become a 24-hour-a-day job.A few of the many entry methodologies used today include moving averages,channel breakouts, momentum, volatility breakouts, oscillators, and price patterns.These basic techniques have become the foundation for wonderfully complex newideas. In order to build better systems later in this book, we must first understandthe basics of current entry techniques.Trend-Following TechniquesTrend-following methods generate buy signals while the market is in a period ofstrength. In contrast, sell signals are generated during periods of weakness.Although trend followers will never buy market bottoms or sell market tops, themiddle of a trend is often enough to produce profitable trading. Typical trend following strategies will employ moving averages, channel breakouts, momentumreadings, or volatility breakouts to signal entries.
CHAPTER 3 Creating Trading Strategies
57
Moving AveragesFor over 50 years, moving averages have been used by traders. Essentially, a moving average is the mean of a time series, but updated and recalculated each tradingday. Moving averages come in many different shapes and sizes. The most commonare: simple, weighted, and exponential.The most basic of the above three, the simple moving average, is an averageof values recalculated every day. As time passes on, older values are replaced bymore current values. The result is that the average “moves” over time. An n-daysimple moving average is calculated by summing the previous n days of values(usually closes) and dividing by the value of n. Shorter values of n produce tightermoving averages, which hug closing data more closely than moving averages created with larger values of n.1Simple n-day moving average = ᎏ ∑ ClosexNNumerous variations of the moving average have been created and studiedover time. Some of the innovations involve the weighting, smoothing formulas, andoverall structure of how the average is calculated. Besides the simple moving average, the second most common form is the exponential moving average. It is calculated using today’s price value and yesterday’s moving average value. The smoothing factor, ␣, determines how quickly the exponential moving average responds tocurrent market prices. Large values of ␣ will track prices more closely than smallervalues of ␣.Exponential moving average = xt = (␣) closet + (1 – ␣) xt-12where ␣ of moving average in days = ᎏ1 ⫹ daysThe weighted moving average is the third most common. Unlike simple movingaverages that weight each price equally, weighted moving averages typicallyassign higher weights to more recent data points. An example of a weighted moving average would be:Weighted moving average =1ᎏᎏ (4 ⭈ closet ⫹ 3 ⭈ closet⫺1 ⫹ 2 ⭈ closet⫺2 ⫹ 1 ⭈ closet⫺3)(4 ⫹ 3 ⫹ 2 ⫹ 1)The basic trading signals for moving averages are triggered when pricescross above or below a moving average. When prices cross above the moving average, higher prices are likely and it signals that it’s time to buy. When prices crossbelow the moving average, a declining market is expected and it’s time to sell.Moving averages of 20 to 100 days are commonly used to generate buy and sellsignals. Shorter moving averages will respond quicker to recent price movement
PART 1 Structural Foundations for Improving Technical Trading
58
and generate more trading signals than will longer moving averages, which produce trading signals infrequently.Other variations of moving average strategies combine more than one movingaverage to generate trading signals when a faster moving average—smaller n in asimple moving average or larger ␣ in an exponential moving average—crossesabove or below a slower moving average (larger n or smaller ␣). Moving averagesof 10 and 40 days are commonly used to generate signals in the dual moving average crossover system (Figure 3.1).Moving Average RulesVariation 1Buy when close crosses above x day moving average■ Sell when close crosses below x day moving average■
Variation 2Buy when x day moving average crosses above y day moving average■ Sell when x day moving average crosses below y day moving average■
Due to the smoothing of prices, moving averages will always lag currentprices. As prices trend higher, the average will always trail current prices. The
Single and Dual Moving Averages
120
Price
110
Price
100Faster Average90Slower Average
FIGURE
3.1
Single and Dual Moving Averages. Averages are often used