modeller implicitly assumes that thefactor sensitivities are constant over the data sample. Hence, in the case of a deterministic relationship where beta would change over time in a predictable way, OLS wouldestimate an average beta over the sample with a large prediction error ε.The problem is illustrated in Figure 7.1. The sensitivity ∂Y /∂X of a dependent variable(say Y ) to an independent one (say X) is represented on the vertical axis. The actualsensitivity is not constant but varies over time. Linear regression measures an average (orunconditional) effect over the estimation data set. In the case of Figure 7.1, this averagesensitivity is close to zero and would probably not be statistically signiﬁcant. However,the actual sensitivity is far from small. For instance, at time t, the true factor sensitivityis strongly negative although the unconditional model’s prediction is slightly positive.Parameter estimation is clearly biased here. At time t, the expected value of beta isquite different from the true value. The model wrongly attributes to the stochastic residualε an effect that is in fact largely deterministic. The importance of the estimation errordepends on the variance of the actual sensitivity over the sample (the more volatile thesensitivity, the more likely sensitivity measured at a particular time differs from theaverage sensitivity) and the size of the estimation window.The example given in the “Sensitivity estimation.xls” ﬁle illustrates standard linearregression (OLS) using Microsoft Excel’s native LINEST function.7 The data set consistsFactorsensitivity∂Y∂X
True factorsensitivitySensitivityestimated bylinear OLS
Best estimation forcurrent sensitivity
t
bias
Time
Estimation window
Figure 7.1 Actual sensitivity (in black) and unconditional sensitivity measured by linear OLSestimation (in white). When the true sensitivity is not constant but conditional on time, OLSestimation, which measures an unconditional (average) sensitivity over time, is biased. Theamplitude of the bias depends on the variance of the true sensitivity and the length of theestimation window7A more user-friendly linear regression tool is also available through the “Analysis Tool pack” add-in thatcomes with Excel.
218
Applied Quantitative Methods for Trading and Investment
of 100 observations of two observable variables, Y and X, representing daily percentagereturns. X and Y are related by a simple time-varying relationship that is mostly deterministic (90% of the variance in Y is related to X):Y (t) = β(t) × X(t) + ε(t)
(7.7)
where β(t) is a linear function of time; ε(t) is a Gaussian random variable (µ = 0%, σ =0.1%). The actual beta (generated in column B and plotted in white in Figure 7.1) increaseslinearly over the period, starting at −0.49, turning positive at time t = 50 and ﬁnishingat 0.50.Cell K3 contains the Excel formula for the beta estimated over the entire data set, i.e.“=LINEST($E$3:$E$102,$C$3:$C$102,TRUE,FALSE)”. Its value is −0.04. If this valueis used to predict Y based on X using the same data set, the R-square is 2%. Cell K4contains the formula for the beta estimated over the last 50 observations only. Its valueis 0.23 and the corresponding R-square (using these 50 observations) is 61%.8 Given allthe information available at time t = 100 (i.e. 100 observations of X and Y ), which oneis the right value of beta to use at time t = 100? And at time t = 101?Actually, neither of the two values is satisfactory as they correspond to past averagesof betas rather than current values or predictions of beta. The true value of beta attime t = 100 is 0.5 and the best value to use when a new value of X and Y becomesavailable at t = 101 would be 0.51. Unfortunately OLS is unsuitable when beta changesover time because it averages out the variations of beta rather than models its timestructure. Furthermore it is backward looking rather than predictive. In other terms, OLSis insensitive to the order of the observations. One could scramble the data, OLS wouldstill yield the same estimate of beta. The potentially useful information contained in thedata chronology is not used.Intuitively, one can see that the estimates are highly sample-dependent. If beta changesrapidly, estimates using short time periods should be better than estimates using longerperiods. Unfortunately, by reducing the estimation period, one increases sampling error.The best window size should correspond to an optimal trade-off between sampling errorand beta variation. On the one hand, if the estimation period is too short, there are notenough observations to separate the information from the noise, resulting in large samplingvariance. On the other hand, if the estimation period is too long, the current beta maysigniﬁcantly differ from its average, resulting in a biased model. This is known as thebias/variance dilemma.9Based on this perception, many practitioners use rolling regressions. This consists ofapplying a linear regression model to a rolling window of observations. The window is anordered subsample of the time series. Figure 7.2 illustrates the rolling window approach.An example of rolling regression is given in column H of the spreadsheet. The corresponding (out-of-sample) R-square is 39.2%.The provided sensitivity estimation tool pack in the “SEToolPack.XLS” ﬁle can also beused to estimate rolling regressions. In order to do so, select “Sensitivity estimation” from8Note that these R-square ﬁgures are in-sample and do not reﬂect the out-of-sample performance of the model.At time t = 75 for instance, the 25 last observations (i.e. 76 → 100) are not yet available. And yet we use thet = 100 estimate of beta in order to compute this R-square.9This issue becomes even more important and difﬁcult to resolve when measuring joint sensitivities that changeat different speeds. Which window size should then be used?
Time-Varying Factor Sensitivities
219
W1→7
W2→8Informationalready in W1→7
1
2
3
4
5
6
7
8
9
Time
Newinformation
Rolling stepVariance in W1→7
New: 1/6Old: 5/6Expected autocorrelation: sqrt(5/6) = 0.91
Figure 7.2 Rolling regressions. Sensitivities are estimated over a rolling window (here W1→7 )of the n last observations (here n = 7). Then, the estimation window is moved forward by arolling step of p observations (here p =1) and a new set of factor sensitivities is estimated (hereon W2→8 ). The procedure goes on until the last observation in the data set is reached. Unlessp = n, consecutive windows are not independent. Actually, (n − p)/n of the variance in awindow is shared with an adjacent window and only p/n of the variance is new information
Figure 7.3 Using the sensitivity estimation tool pack to estimate rolling regressions
the tools menu. In the main menu (Figure 7.3) enter the Y and X ranges into the corresponding edit boxes (dependent and independent variables respectively), select the rangewhere you want the computed sensitivities to be saved to (output range),10 select “Linearrolling regression” option and press “OK” to go to the next dialogue box (Figure 7.4). Inthe “Rolling regression” dialogue box, select “no weighting” and a window size.10The sensitivity estimation tool pack can be used with any worksheet and is not restricted to the example sheetprovided. For instance, it can be used with the “stock beta estimation” sheet or any proprietary data collectedby the reader in other workbooks.
220
Applied Quantitative Methods for Trading and Investment
Figure 7.4
The rolling regression menu of the tool pack
The example provided in the spreadsheet consists of two factors, X1 and X2 and 115observations. The dependent variable Y is generated by a time-varying linear relationshipof the factors. The sensitivities and the additive stochastic noise are stored in columns AFto AI of the example sheet. They have been constructed so that each represents a differenttype of time series. The sensitivity to X1 is a continuous function of time, the sensitivityto X2 is constant except for a large level shift occurring half-way through the sampleand alpha is a slowly growing value. These time series are represented by the white linesin the graph included in the spreadsheet (Figure 7.5). The grey lines correspond to therolling regression estimates. The ﬁrst n observations, where n is the window size, areused to estimate the regression coefﬁcients at time n. “Ini. Period” will therefore appearin the ﬁrst n rows.Rolling regressions, although popular, are still unsuitable for time-varying sensitivityestimation. They give the illusion that they can handle conditional betas while in realitythey are biased estimators of time-varying sensitivities. This is because OLS linear regression estimates an average sensitivity over each rolling window. If actual sensitivities varyover time (e.g. follow a trend) they will depart from their average.
Sensitivity
Actual Sensitivities
Alpha
X1
X2
1.00
0.50Time0.00
−0.50
Figure 7.5 Rolling regression example. The data set is provided in the “Example” sheet of“SEToolPack.xls” ﬁle. It consists of 115 observations. The white lines correspond to the actualsensitivities. The grey lines are the sensitivities estimated by the rolling regression modelprovided in the tool pack. The size of the rolling window is 32 observations
Time-Varying Factor Sensitivities
221
Figure 7.2 shows that sensitivities computed from adjacent windows are highly correlated because they are estimated from data sets that share n − p observations, n being thewindow size and p the rolling shift between two consecutive windows. Autocorrelation inrolling sensitivities increases with the window size n. This autocorrelation is independentof the behaviour of the actual sensitivities. Even√ for random sensitivities, autocorrelationin estimated sensitivities is expected to be (n − p)/p (the correlation squared is thepercentage of variance common to two adjacent windows, i.e. (n − p)/p). The measuredeffect of a particular event persists as long as it remains in the rolling estimation window.The consequences of this persistence are threefold:• “Ghost” effects. If a signiﬁcant event occurs on one particular day, it will remain inthe sensitivity series for p further days, where p is the length of the rolling window.The apparent sensitivity persistence is a measurement artefact and does not necessarilyrelate to a true sensitivity shock. The amplitude of the effect depends on both the sizeof the rolling window and the variance of the actual sensitivity. Figure 7.6 illustratesthe shadow effect. The estimation bias is a function of the difference between thetwo sensitivity levels, their respective durations and the length of the rolling window.Measured sensitivity remains at this level for another 15 days although actual sensitivityhas returned to its normal level. This is the shadow effect. Rolling regression measuresan effect that may have long disappeared.• The difference between two consecutive sensitivity estimates is only determinedby two observations. That is the observation entering the rolling window (i.e. themost recent data point) and the observation leaving the rolling window (i.e. the most
Sensitivity
Actual sensitivityMeasured by rolling OLS
Shadow effect
Estimation bias
1
26
51
76
101Time
126
151
176
Figure 7.6 Estimation bias and shadow effect in rolling estimation. In this controlled experiment, the actual sensitivity of Y to X is represented by the thin line. It has been chosen constantover most of the data set at some level (LL) except for a very short period of 5 days duringwhich it takes an unusually high value (HL). This may be caused, for instance, by incorrect market expectations motivated by unfounded rumours. Sensitivity is estimated by OLS regressionover a rolling window. The window is 20 days long. Sensitivity estimates are represented by thebold line. Rolling estimation clearly underestimates actual sensitivity. At the end of the ﬁfth dayof HL, the estimate reaches its maximum value of (5 × HL + 15 × LL)/20
222
Applied Quantitative Methods for Trading and Investment
remote data point). While it is legitimate that the most recent observation affects thisdifference, there is no reason why the most remote point should have more inﬂuencethan any other observation in the window, especially because the size of the windowhas been chosen somewhat arbitrarily.• Sensitivity estimates lag actual sensitivity. Because rolling regression measures anaverage sensitivity over the estimation period, estimated betas lag actual betas, especiallywhen the latter trend. The lag depends on the length of the rolling window. This effect isclearly visible in Figure 7.5. The beta to variable X2 slowly adapts to the level shift whilethe beta to X1 seems to lag the actual beta by about 20 observations.
7.4 WLS TO ESTIMATE FACTOR SENSITIVITIES: A BETTERBUT STILL SUB-OPTIMAL METHODOne of the major limitations of rolling OLS is that all observations are given equalweight, irrespective of their distance in time. Hence, the most recent observation is giventhe same “credibility” as the most remote observation in the window. A natural improvement of the rolling window procedure would be to give observations different weightsbased on their position in the time series. This procedure is known as weighted leastsquares (WLS) estimation and can better deal with variations of sensitivities (see, amongothers, Pindyck and Rubinfeld (1998)). Remote observations are attributed less weightthan recent observations.The criterion to be minimised, WSSR, is the weighted sum of squared residuals ratherthan the ordinary sum of squared residuals, SSR. WSSR is deﬁned by:WSSR(β) =
T
w(t)(Ŷβ (t) − Y (t))2 = (Y − Xβ) W(Y − Xβ)
(7.8)
t=1
where W is a diagonal matrix containing the weights w(t), i.e.w(1)0...0 0w(2) . . .... W = ...... ...0 T ×T0...0 w(T )The weighted least squares estimator β̂w is:β̂w = (X WX)−1 X WY
(7.9)
and the estimated variance–covariance matrix of β̂w is s 2 (β̂w ):s 2 (β̂w ) = MSEw (X WX)−1
(7.10)
with MSEw being the weighted squared error of the regression model:MSEw =
WSSRTt=1
w(t) − 2
(7.11)
Time-Varying Factor SensitivitiesSensitivity
223Actual Sensitivities
Alpha
X1
X2
1.00
0.50Time0.00
−0.50
Figure 7.7 Weighted regression example. The size of the rolling window is still 32 observations. However, here the observations have been weighted using a linear function of time and adecay rate of 3%
Not surprisingly, when W = I, i.e. when all observations have equal weight, equations (7.8)to (7.10) can be condensed into the OLS equation. The most popular functions, which areprovided by the tool pack, are linear or exponential but other functions such as sigmoidscan be considered. A decay rate determines the speed at which the weight of an observationdecreases with time.11In order to use the sensitivity estimation tool pack, select one of the weighting optionsin the rolling regression menu (Figure 7.3). For instance, select a linear weighting andenter a weight decay of 3%. Figure 7.7 shows the result of the estimation.Weighted least squares estimation induces less autocorrelation in the estimates thanordinary least squares estimation. Depending on the decay rate, shadow effects, lag andpersistence problems are considerably reduced.However, WLS does not provide a way to model the sensitivities time series. It stillmeasures past (weighted) average sensitivities rather than predicting future ones. In addition, all sensitivity coefﬁcients in the regression equation are identically affected by theweighting, regardless of their rate of change as the weights only depend on the position ofthe observation in the time series. Consequently, constant sensitivities suffer from largeweight discount rates while highly variable sensitivities suffer from small decay rates.There is no single weight discount rate that is adapted to all factor sensitivity coefﬁcientswhen their variances differ and some trade-off has to be made.
7.5 THE STOCHASTIC PARAMETER REGRESSION MODELAND THE KALMAN FILTER: THE BEST WAY TO ESTIMATEFACTOR SENSITIVITIESThe procedures that have been described so far involve a single regression equationwith constant betas. These procedures use ordinary or weighted least squares in order torepeatedly estimate new model coefﬁcients from adjacent windows of observations.The stochastic parameter model, however, is based on a conceptually different approach(see Gouriéroux et al. (1997), or Harvey (1989)). The beta coefﬁcients are not assumed11
Although this rate is usually set a priori, one could also estimate it, for instance by minimising the WSSR.
224
Applied Quantitative Methods for Trading and Investment
constant and are optimally adjusted when new information becomes available. The dynamics of the betas are modelled in a second equation. Very much like a GARCH model,12the stochastic parameter regression model is based on a system of equations. Hence, thelinear regression equation (7.2) can be rewritten into a simple time-varying regression byletting β follow a given time process, for example an autoregressive process:13Yt = Xt βt + εt
(7.12)
andβt = β t−1 + ηt
for t = 1, . . . , T
(7.13)
where  is a non-random K × K matrix, K being the number of factors; ηt is a vectorof serially uncorrelated disturbances with zero mean. It is not observable.The addition of the second equation (7.13) allows beta to vary in time according toa process that can be modelled. However,  and ηt are not observable and need tobe estimated. Simple estimation procedures such as OLS which can only handle singleequations cannot be used here but fortunately, the problem can be resolved as it can easilybe put into a state space form. The Kalman ﬁlter can then be used to update the parametersof the model. Moreover, the same Kalman ﬁlter can be employed with a large variety oftime processes for the sensitivities without adding much computational complexity.14Originally developed by control engineers in the 1960s (see Kalman (1960)) for application concerning spacecraft navigation and rocket tracking, state space models have sinceattracted considerable attention in economics, ﬁnance, and the social sciences and havebeen found to be useful in many non-stationary problems and particularly for time seriesanalysis. A state space model consists of a system of equations aimed at determining thestate of a dynamic system from observed variables (usually time series) contaminatedby noise. Time, or the ordering of observations, plays an important role in such models.Two identical state space models applied to the same data are likely to produce differentestimates if observations are presented to the models in different orders (i.e. scramblingobservations alters model estimates). It is therefore not surprising that they are usuallyused for modelling time series. The state of the dynamic system, described by “statevariables”, is assumed to be linearly related to the observed input variables.The stochastic coefﬁcient regression model expressed in a state space form can bedeﬁned by a system of two equations. The ﬁrst equation (e.g. equation (7.12)) is referred toas the observation equation and relates the dependent variable, Yt , to the independent variables by the unobservable states (in the case of equation (7.12), the state is simply the vector of sensitivities βt ). More generally, the observation equation takes the following form:Yt = Ht (st + d) + εt12
for t = 1, . . . , T
(7.14)
See for instance Bollerslev (1986).Equation 7.13 describes one example of a time process that can be used for modelling the sensitivities timeseries. This particular process corresponds to a simple AR1. The Kalman ﬁlter approach can be used with manymore processes.14If ηt has zero variance and  = I, then βt is constant and equation (7.12) is just the familiar equation of alinear regression. The estimates of βt are those of a recursive regression (OLS linear regression using all theobservations from i = 1 to t) and the estimate of βT is identical to the OLS estimate of a linear regression. Ifthe variance of ηt is larger than zero, however, sensitivity coefﬁcients are allowed to change over time.13
Time-Varying Factor Sensitivities
225
where Yt is the dependent observable variable (e.g. an asset return) at time t. st is a Kvector that describes the state of the factor sensitivities of the asset returns at time t. d isa K non-random vector that can account for the long-term mean in sensitivities. Ht is aK vector that contains the factor values at time t. εt is a serially uncorrelated perturbationwith zero mean and variance Rt , i.e.: E(εt ) = 0 and Var(εt ) = Rt , Rt being non-random.It is not observable.The exact representation of Ht and st will depend on model speciﬁcation. For theautoregressive model described by equations (7.12) and (7.13), which is also the mostpopular, Ht and st simply correspond to Xt and βt respectively. However, this is notthe case for more complex models such as the random trend model which requires morestate variables (K = 2N , N being the number of factors). In general, the elements of stare not observable. However, their time structure is assumed to be known. The evolutionthrough time of these unobservable states is a ﬁrst-order Markov process described by atransition equation:st = s t−1 + ct + ηt
for t = 1, . . . , T
(7.15)
where  is a non-random K × K state transition matrix; ct is a non-random K vector; ηtis an m × 1 vector of serially uncorrelated disturbances with zero mean and covarianceQt ; E(ηt ) = 0 and Var(ηt ) = Qt , Qt being non-random. It is not observable.The initial state vector s0 has a mean of s and a covariance matrix P0 , i.e.:E(s0 ) = s
and
Var(s0 ) = P0
Furthermore, the disturbances εt and ηt are uncorrelated with each other in all timeperiods, and uncorrelated with the initial state, i.e.:E(εt , ηt ) = 0 for all elements ηt of ηt ,
and for t = 1, . . . , T
andE(εt , s) = 0 and E(ηt , s) = 0 for t = 1, . . . , TThe system matrices , P and Q, the vectors η, c and d and the scalar R are non-randomalthough some of them may vary over time (but they do so in a predetermined way).Furthermore, the observation noise εt and the system noise ηt are Gaussian white noises.The most popular processes used for time-varying sensitivities are the random walkmodel and the random trend model.The random walk model dictates that the best estimate of future sensitivities is thecurrent sensitivity. To specify this model, vectors d and c are set to zero and the systemmatrices for the random walk model are given by equations (7.16):15st = [βt,1 , βt,2 , . . . , βt,N ]Ht = [1, Ft,1 , . . . , Ft,N−1 ]
(7.16)
=I15Hence the random walk speciﬁcation of the stochastic regression model can be expressed as a system of twoequations: (1) Yt = Xt βt + εt and (2) βt = βt−1 + ηt .
226
Applied Quantitative Methods for Trading and Investment
where  is N × N , Ht is 1 × N and st is N × 1, N being the number of independentvariables, including the constant.The random trend model dictates that the best estimate of future sensitivities is thecurrent sensitivity plus the trend. In the presence of a trend, sensitivities at t + 1 are notequally likely to be above or under the value at t. A simple model which allows thesensitivities to trend is the random trend model. A sensitivity coefﬁcient βit follows arandom trend if:βt = βt−1 + δt−1 + ηt,1(7.17)δt = δt−1 + ηt,2where βt represents the sensitivity vector at time t and δt is the random trend in thesensitivity vector at time t.16In a state space form, the system in (7.17) can be written as:
βtδt
= 0
βt−1δt−1
+
ηt,1ηt,2
(7.18)
where the individual sensitivity random trend state transition matrix is given by1 1.0 =0 1To specify this model, vectors d and c are set to zero and the collection of factorsensitivities and trends are expressed as:s(t) = [β1 (t), δ1 (t), β2 (t), δ2 (t), . . . , βN (t), δN (t)]
(7.19)
H(t) = [1, 0, 1, 0, . . . , 1, 0]0 0 · · · 0 0 00 = .... ..000
(7.20)
(7.21)
where the dimensions of the state of sensitivities has been doubled to include trends inthe sensitivities, i.e.  is 2N × 2N , st is 2N × 1 and Ht is 1 × 2N .Other time processes such as the random coefﬁcient model (Schaefer et al., 1975) orthe mean-reverting coefﬁcient model (Rosenberg, 1973) can also be used within thismodelling framework.The objective of state space modelling is to estimate the unobservable states of thedynamic system in the presence of noise. The Kalman ﬁlter is a recursive method ofdoing this, i.e. ﬁltering out the observation noise in order to optimally estimate the statevector at time t, based on the information available at time t (i.e. observations up toand including Yt ). What makes the operation difﬁcult is the fact that the states (e.g. thesensitivities) are not constant but change over time. The assumed amount of observationnoise versus system noise is used by the ﬁlter to optimally determine how much of thevariation in Yt should be attributed to the system, and how much is caused by observation16Hence the random trend speciﬁcation of the stochastic regression model can be expressed as a system ofthree equations: (1) Yt = Xt βt + εt , (2) βt = βt−1 + δt−1 + ηt,1 and (3) δt = δt−1 + ηt,2 .
Time-Varying Factor Sensitivities
227
noise. The ﬁlter consists of a system of equations which allows us to update the estimateof the state st when new observations become available.Equations (7.22) to (7.31) describe the Kalman ﬁlter.17 These equations are importantto the more technical readers who want to develop and implement the model. Otherreaders may want to skip these equations and directly move to the more intuitive examplegiven later.The ﬁrst two equations deﬁne the state ŝt|t (equation (7.22)) and the state error covariance matrix Pt|t (equation (7.23)):ŝt|t = E(st |Y1 , . . . , Yt )
(7.22)
Pt|t = E((st − ŝt|t )(st − ŝt|t ) ) = Var(st − ŝt|t )
(7.23)
where the notation âb|c denotes an estimate of a at time b conditional on the informationavailable at time c.The predicted state of sensitivities ŝt|t−1 and the corresponding forecasting errorcovariance matrix Pt|t−1 = E((st − ŝt|t−1 )(st − ŝt|t−1 ) ) are given by the predictionequations (7.24) and (7.25):ŝt|t−1 = ŝt−1|t−1 + ct
(7.24)
Pt|t−1 = Pt−1|t−1  + Qt
(7.25)
and the predicted dependent variable is given by equation (7.26):Ŷt|t−1 = Ht ŝt|t−1 + d
(7.26)
The forecast error et at time t and its variance ft can be calculated by equations (7.27)and (7.28) respectively:et = Yt − Ŷt|t−1
ft|t−1 = Pt|t−1  + Rt
(7.27)(7.28)
The new information is represented by the prediction error, et . It can be the result ofseveral factors: random ﬂuctuations in returns, changes in underlying states, or error inprevious state estimates. Given this new information, the estimate ŝt|t of the state vectorand its covariance Pt|t can now be updated through the updating equations (7.29) and(7.30) respectively. The Kalman ﬁlter uses the new information for adjusting the estimatesof the underlying states, where the new information is simply the prediction error of thereturns. The Kalman gain matrix, Kt , optimally adjusts the state estimates in order toreﬂect the new information:ŝt|t = ŝt|t−1 + Kt et
(7.29)
Pt|t = (I − Kt Ht )Pt|t−1
(7.30)
The Kalman gain matrix is calculated from the observation noise variance Rt and thepredicted state error covariance Pt|t−1 with the recursive equation (7.31):Kt = Pt|t−1 Ht (Ht Pt|t−1 Ht + Rt )−117
Further explanations can be found in Harvey (1989) or Gouriéroux et al. (1997).
(7.31)
228
Applied Quantitative Methods for Trading and InvestmentInitialisation: t = 1s1|0 = s, P1\0 = P0
Non-random values of:Φ, d, ct , Rt , Qt
Predict Y:
Yt |t − 1 = Ht st |t − 1 + d
Compute error:
et = Yt − Yt |t − 1
Compute gain:
Kt = Pt |t − 1H′t (Ht Pt |t − 1H′t + Rt )−1
t=t+1Observationof Yt and Ht
No
Updating
st |t = st |t − 1 + Kt et
equations:
Pt |t = (I − Kt H′t )Pt |t − 1
Prediction
st |t − 1 = Φ