Lag
5
FIGURE 2.1B
15
20
White Noise ACF.
Moving Average Process (MA)We now generate another time series from the white noise series above. Thevalue yt of this time series at time t is given by the ruleyt = et + bet–1
(2.2)
In words, the time series value is the sum of the current white noise realization plus beta2 times the white noise realization one time step ago. Note that2
Beta in this connotation is a nondescript Greek symbol denoting a constant and hasno relationship to the CAPM model.
BACKGROUND MATERIAL
18
when b = 0, this is the same as the white noise series. In Figure 2.2a is a plotof a time series of this type. This specific time series was generated from thewhite noise sequence in Figure 2.1 using the formula yt = et + 0.8et–1. The correlogram of the series is plotted in Figure 2.2b. In the correlogram, note thatthere is a steep drop in the value after t = 1. To see why that is, let us consider the time series values for the three consecutive time steps t, t + 1, andt + 2.yt = et + bet–1yt+1 = et+1 + betyt+2 = et+2 + bet+1
(2.3)
3210–1–2–310
30
FIGURE 2.2A
50
70
90
MA(1) Series.
1.0Auto Correlation
0.80.60.40.2–0.0–0.20
5
FIGURE 2.2B
10Lag
MA(1) Series ACF.
15
20
Time Series
19
Observe that the values one time interval apart (t = 1) have in their termsone common white noise realization value (albeit with different coefficients).Between yt and yt+1 the common white noise realization is et. Similarly, between yt+1 and yt+2 there is et+1. Because of this, we expect there to be somecorrelation between them.However, between yt and yt+2, values two time intervals apart (t = 2), wehave no common white noise realizations. They are independent drawingsfrom normal distributions and are therefore uncorrelated (correlation = 0).Thus, after exhibiting strong correlation after one time step, the correlationgoes to zero from the next time step onward. This would explain the steepdrop in correlation after t = 1.To examine the predictability of this time series, we again ask the samequestion: Does knowledge of the past realization help in the prediction ofthe next time series value? The answer here is a resounding yes. At time stept we know what the white noise realization was at time step t – 1. Thus ourprediction for time step t would be a value that is normally distributed withthe mean, ytpred = βε t −1. The variance of the predicted value would be thevariance of the et, which is same as the variance of the white noise used toconstruct the time series. Since these values are based on the condition thatwe know the past realization of the time series, they are called the conditional mean and the conditional variance of the time series. To conclude,knowledge of the past definitely helps in the prediction of time series.Summing up, the preceding series was constructed using a linear combination (moving average) of white noise realizations. The series is thereforecalled a moving average (MA) series. Also, because we used the currentvalue and one lagged value of the white noise series, the series qualifies as afirst-order moving average process, denoted as MA(1). This idea is easilygeneralized to a series where the value is constructed using q lagged valuesof white noise realizations.yt = et + b1et–1 + b2et–2 +...+bqet–q
(2.4)
Such a series is called the moving average series of order q or an MA(q)series.
Autoregressive Process (AR)In the previous example we had constructed a time series by taking a linearcombination of a finite number of past white noise realizations. In this section we will construct the series using a linear combination of infinite pastvalues of the white noise realization. In practice, though, infinity is approximated by taking a very large number of values. A question that immediatelypops to mind is that if we add an infinite sequence of numbers, will thesum not go to infinity? In some instances it might go to infinity. There are,
BACKGROUND MATERIAL
20
however, cases where the sum of an infinite sequence of numbers is actuallya finite value.3 Let us denote the value of the time series at instant t asyt = et + aet–1 + a2 et–2 +...
(2.5)
The infinite moving average representation above is called the MA(•) representation. To simplify Equation 2.5, consider the value of the time seriesat t – 1. It is given asyt–1 = et–1 + aet–2 + a2 et–3 +...
(2.6)
Examining the two equations, note that we can write yt in terms of yt–1 asfollows:yt = ayt–1 + et
(2.7)
In words, the value at time t is alpha times the value at time t – 1 plus a whitenoise term. Note that alpha may be viewed as the slope of the regression between two consecutive values of the time series. Since the next value in thetime series is obtained by multiplying the past value with the slope of theregression, it is called an autoregressive (AR) series. Figure 2.3a is the plot ofthe AR time series, generated using the white noise values seen in Figure 2.1.The corresponding correlogram is shown in Figure 2.3b. Notice that thecorrelation values fall off gradually with increasing lag values; that is, thereis not much of a sharp drop. To get an insight into why that is, let us applythe same kind of reasoning as we did for the MA model. Every time step hasin it additive terms comprising all the previous white noise realizations.Therefore, there will always be white noise realizations that are common between two values of the time series however far apart they may be. Naturally, we can expect there to be some correlation between any two values inthe time series regardless of the time interval between them. It is thereforenot surprising that the correlation exhibits a slow decay.To answer the predictability question, here, too, as in the moving average case, knowledge of the past values of the time series is helpful in predicting what the next value is likely to be. In this case we have ytpred = α yt −1.The conditional variance of the predicted value would be the variance of theet, which is same as the variance of the white noise used to construct the timeseries.The one-step autoregressive series may be extended to an autoregressive(AR) series of order p, denoted as AR(p). The value at time t is given as3
We touch upon this topic very briefly in the appendix. However, for a full-blowndiscussion on stability analysis, we recommend that the reader follow up with thereferences.
Time Series
21
420–2–410
30
50
FIGURE 2.3A
AR(1) Series.
70
90
1.0Auto Correlation
0.80.60.40.2–0.0–0.20
5
FIGURE 2.3B
10Lag
15
20
AR(1) Series ACF.
yt = et + a1yt–1 + a2yt–2 +...+ apyt–p
(2.8)
It is, however, important to bear in mind that the generalized AR seriesis generated from a white noise series using linear combinations of pastrealizations.
The General ARMA ProcessThe AR(p) and MA(q) models can be mixed to form an ARMA(p, q) model.By extrapolation it is easy to see that the generation rule for an ARMA (p, q)process is given as
BACKGROUND MATERIAL
22
[
]
y t = α 1y t −1 + α 2 y t − 2 + … + α p y t − p +
[
+ ε t + β1ε t −1 + β 2ε t − 2 + … + β qε t − q
(2.9)
]
We once again underscore the main point (hoping to drive it home) by quoting our constant refrain pertaining to Weiner filtering: The preceding models are all constructed using a linear combination of past values of the whitenoise series. An important consequence of that fact is that the sum of two independent ARMA series is also ARMA.
The Random Walk ProcessAn important and special ARMA series that merits discussion is the randomwalk. The random walk has been studied extensively by scientists from various disciplines. Phenomena ranging from the movement of molecules tofluctuations of stock prices have been modeled as random walks. Let ustherefore discuss this in some detail.A random walk is an AR(1) series with a = 1. From the definition of anAR series given, the value of the time series at time t is thereforeyt = et + et–1 + et–2 +... = et + yt–1
(2.10)
In words, the random walk is essentially a simple sum of all the white noiserealizations up to the current time. The AR representation provides an alternate way to look at the random walk. It is the value of the time series onetime step ago plus the white noise realization at the current time step. Thewhite noise realization at the current time step in the case of the randomwalk is known as the innovation. Figure 2.4 is a picture of the random walkgenerated using the white noise series in Figure 2.1.Let us now begin to examine some properties of the random walk. Whatdo we expect the variance of the random walk to be at time t? Applying theformulas from the appendix in Chapter 1 on the MA(∞) (infinity) representation of the random walk, along with the fact that white noise drawings areuncorrelated, we have
( )
( )
( )
(
)
( )
var yt = var ε t + var ε t −1 + var ε t − 2 + L + var ε1
(2.11)
Since these random white noise drawings all have the same variance, thevariance of the random walk at any time t is clearly
( )
( )
var yt = t var ε t
(2.12)
Note that in this case the variance depends on the time instant, and it increases linearly with time t. (If the variance increases linearly with t, then the
Time Series
23
15
10
5
0
–510
30
FIGURE 2.4
50
70
90
Random Walk Series.
standard deviation increases linearly with t ). In this case, unlike all theprevious cases, the variance increases monotonically with time; that is, thevalues are capable of moving to extremes with the passage of time. Also, thestatistical parameters like the unconditional mean and variance are not timeinvariant, or stationary. The series is therefore called a nonstationary timeseries.The correlation between a value and its immediate lagging value is 1.Our prediction for the next time step would then be a value with meanequal to the current time step; that is, ytpred = yt −1. The variance, of course,is the variance of the white noise realizations. As a matter of fact, our prediction for any number of time steps would be a distribution whose mean isthe current value of the series. However, because the variance increases linearly with time, the error in our prediction progressively increases with thenumber of time steps.Of the different time series reviewed so far, the random walk is the onlyseries in which the prediction of the mean value for the next time step is thecurrent value. Such series where the expected value at the next time step isthe value at the current time step are known as martingales. The randomwalk qualifies as a martingale.The random walk also exhibits a strong trending behavior. Let us examine that statement by contrasting the behavior of the random walk with
24
BACKGROUND MATERIAL
other time series. The other time series tend to oscillate about the mean ofthe series; that is, they exhibit mean reversion. To see what we mean, wesuggest that the reader examine the time series plots and see how manytimes the different time series cross the mean (zero in this case). It is easyto see that the random walk has the least number of zero crossings. Eventhough the increments to the series at each time instance have equal odds ofbeing positive or negative, it is not uncommon for the random walk series tostay positive (or negative) during the entire time.
FORECASTINGHaving discussed the stochastic time series models, let us now direct our attention to the problem of forecasting. The classical forecasting problem maybe stated as follows: We are given historical time series data with values upto the current time. We are required to predict the value of the next time stepvalue as closely as possible. In the stochastic time series context, this meansthat we first identify the ARMA model that is most likely to have resulted inthe data set and then use the estimated parameters of the model to forecastthe next value of the time series.Let us now formally lay down the steps involved in forecasting problems involving stochastic time series. The solution method is best describedas a three-step process. The first step involves transforming the time seriessuch that it is amenable to analysis. We call this the preprocessing step. Thedata are then analyzed for patterns that may clue us in on the dynamics ofthe time series. This means that we identify the ARMA model that is likelyto have resulted in the data. This is the analysis step. Finally, we make ourprediction in the prediction step. We now discuss each of the three steps indetail.Preprocessing involves dealing with pesky issues like checking for missing values, weeding out bad data, eliminating outliers, and so forth. It mayalso involve transforming the time series to prepare it for analysis. A simpletransformation may be to subtract the mean of the series. Other methodsmay involve creating a new time series by a functional transformation. Theapplication of the logarithmic function to values of the given series prior toanalysis is a good example. In the context of ARMA models, an importanttransformation technique that is frequently used is known as differencing. Itis a process by which a new series is constructed by taking the difference between two consecutive values in the given series. Let us discuss the motivation for doing that. The ARMA model based forecasting is typically focusedon the stationary time series. If we are given a series that is deemed nonstationary, differencing helps transform the nonstationary series into a stationary series. The output from the differencing operation may be viewed as the
Time Series
25
series of increments to the current value. Thus, analyzing the differencedoutput amounts to studying the changes in the values as opposed to the values themselves.The next step is the analysis step. It involves identifying the ARMAmodel used to generate the given time series data. An ARMA model is completely identified when we are given the white noise series and the rule togenerate the time series from the white noise realizations. Sometimes, thewhite noise series is implicit. The estimated ARMA parameters are, however, stated explicitly. But why should we try to fit an ARMA model to agiven data set? The answer is simply that ARMA models provide an empirical explanation for the data without concerning themselves with theoreticaljustifications. This makes them readily applicable to a variety of situations.Also, the fact that ARMA models are empirical is not necessarily a badthing, as insights from the model fitting exercise can be later used to construct a plausible theory.Once the underlying ARMA model is identified, we can proceed to theprediction step. We use the model parameters to predict the next value in theseries. This completes the forecasting exercise. As seen earlier in our discussion of the ARMA model, the prediction of the next time step value is ratherstraightforward once the model is identified. Therefore, insofar as forecasting is concerned, identifying the correct model is key to obtaining a goodforecast. Not surprisingly, a good portion of the field of time series analysisis focused on model identification.
GOODNESS OF FIT VERSUS BIASWe noted that identifying the right model is key to obtaining a good forecast. There are quite a few software packages4 that estimate parameter values for ARMA models. While they are based on a variety of approaches, thebasic underlying theme in all of them remains the same; that is, the goal tofind the most appropriate ARMA model. Note the use of the term most appropriate. Let us focus on what it actually means.Intuitively, a model may be deemed appropriate based on the accuracywith which it is able to account for the given data set. Let us call the number that quantifies this accuracy the “goodness of fit” measure. An exampleof the goodness of fit measure is the least squares criterion, which is simplythe sum of squares of the prediction error. Prediction error is defined asthe difference between the actual observation and the value predicted by themodel. The idea then is to find a model that minimizes the least squares4Eviews, S-Plus, and SAS are some software packages that deal with time series modeling and forecasting.
26
BACKGROUND MATERIAL
criterion (sum of squared errors) for the given data. Another example of thegoodness of fit measure is the maximum likelihood criterion. This is a number representative of the probability that the given data set was producedby a particular set of parameter values. The idea here is to find the parameters that maximize the probability, or the maximum likelihood criterion.Thus, the goodness of fit measure helps identify the best model for the givendata set.Of course, the preceding statement is not without caveats. Let us saythat we are required to choose the best four-parameter model fitting thedata. The goodness of fit criterion would do a wonderful job in helping usachieve that. It is, however, very likely that the best five-parameter modelwould have a better goodness of fit score. As a matter of fact, we can in alllikelihood keep improving our goodness of fit score by increasing the number of explanatory variables. Therefore, using the goodness of fit score without reservation amounts to advocating the philosophy of the more themerrier for explanatory variables.Is that necessarily a good thing? What happens when we apply themodel to out-of-sample data? Will we get the same level of accuracy? To seethe logic more clearly, let us discuss an extreme case where we fit 100 datapoints with a 100th-order polynomial (100 explanatory variables). Withthat, we can get an exact fit to the data and the best possible goodness of fitscore ever. However, as a working model for prediction, it is probably notmuch use to us. Increasing the parameters indefinitely may result in a modelthat fits the current data set but performs poorly when used outside the current sample. Restating, we could say that our model with a large number ofexplanatory variables is hopelessly biased to the current data set. So, here isour dilemma: We can improve the goodness of fit by increasing the numberof explanatory variables and run the risk of bias, or we can use few explanatory variables and possibly miss further reduction in forecast error.The question at this point is, “How do I know the point at which I have areasonable goodness of fit, and at the same time know that I am not overlybiased to the current data set?” The resolution of this forms the topic of discussion in the following section.
MODEL CHOICEThe model choice process attempts to achieve a trade-off between goodnessof fit and bias. In order to decide whether to increase the number of explanatory variables, we pose the question, “Am I getting sufficient bang forthe buck in terms of fit error reduction for the addition of the new explanatory variable?” If I am, then let us go with the additional variable; otherwise,we stick with the model at hand.
Time Series
27
The Akaike information criterion (AIC) quantifies the preceding tradeoff argument.5 In general, every model with k parameters is associated withan AIC number as follows: n ei2 AIC = n log + 2k i =1 n 
∑
(2.13)
where ei is the forecast error on the ith data point. Here, the first term represents the goodness of fit, and the second term is the bias. For every additional variable, the second term increases by a value of 2. However, when avariable is added, we expect the fit to improve and the variance of the forecast error to go down. If this reduction is more than 2, then the AIC valuefor the model with an additional variable will be lower, and we will have gotour proverbial bang for the buck. If the value is higher, then the trade-off isnot worth it, and we stick with the current model.The rationale for the AIC formula and the quantitative value used fortrade-off has a strong foundation in information theory and is far fromarbitrary. Further follow-up material on this can be found in the referencesection.
ExampleThe application of the AIC idea is illustrated in the following exercise. AnAR(3) time series that was generated is shown in Figure 2.5a. AR models ofvarious orders were fit to it and the AIC values calculated. The result is plotted in Figure 2.5b. The x-axis denotes the number of parameters in the AR
420–2–40
20
40
FIGURE 2.5A5
60
80
100
AR(3) Series.
AIC is but one of many cost functions. The Schwartz information criterion (SIC)and the Bayesian information criterion (BIC) are also popular.
BACKGROUND MATERIAL
28
RAINING ON THE PARADEIf you ever happen to make a presentation involving data analysis,here is a situation that you might encounter. After all the preparationinvolving umpteen coffees, and bleary-eyed but vigorous mouse clicking at statistical packages as you present your forecasting model, thereis a wise guy in the audience who quips, “I am sure I can fit any modelto the degree of accuracy I want by adding a lot of variables. I do notsee how your model is any good.” While you would like to stare himdown until he sulks and quietly leaves the room, more often than notthe wise guy happens to be the boss. Unfortunately for you, more oftenthan not he is also correct.The key, however, is to be one up on the wise guy! Based on thepreceding discussion you can now wax eloquently about the tug of warbetween goodness of fit and the evil of bias and how you have meticulously taken into account the effect of adding multiple variables in theforecasting model. Dazzle everyone with your slides on AIC calculations and top it off with an out-of-sample test.If your presentation is close to end of fiscal year, you can chuckleto yourself about the bump in bonus you are likely to see due to this.
AIC Value
Time Series
29
80.0070.0060.0050.0040.0030.0020.0010.000.000
2
4
6
8
10
Number of Parameters
FIGURE 2.5B
AIC Plot.
model and the y-axis is the AIC value. Note that the AIC value registers aminimum at four parameters. This is three AR parameters and a constantvalue for the mean of the series. Using more parameters will result in a better goodness of fit but will not help in forecasting. In some instances, it mightactually hurt the forecasting results.
BACKGROUND MATERIAL
30
MODELING STOCK PRICESThe model that is most commonly assumed for stock price movement iscalled a log-normal process; that is, the logarithm of the stock price is assumed to exhibit a random walk. Let us discuss the implications of such anassumption.First, this says that the logarithm of the stock price is a martingale. Thisis to say that the observed price of a stock at the next time period is roughlyequal to the price at the current time, give or take a few. That is definitelyreasonable.Next, let us examine the resulting time series when we difference therandom walk. Differencing the random walk yields the increment to the random walk at each time step. The set of increments by definition are drawingsfrom a normal distribution. But this is exactly how white noise is defined.Thus, differencing a random walk results in a white noise series. Also, bearin mind that the differencing output of the log-normal process (the differencein the logarithm of the prices) can be interpreted as the stock return.6 Puttingthe two together, the implication of the log-normal assumption is that stockreturns are essentially a white noise process. Let us look at the plausibility ofthis implication. Figure 2.6a is a plot of the logarithm of the price of GE(General Electric) over a 100-day period. The series is then differenced,yielding the differenced plot in Figure 2.6b. To quickly check the nature ofdifferenced values (returns), we urge the reader to examine Figure 2.6d. It isa Q-Q plot of the returns versus the normal distribution. The closer thepoints are to the straight line, the more the actual distribution behaves likea normal distribution. The autocorrelation plot of the returns is depicted inFigure 2.6c. Note that the correlation values are negligible, signifying that anassumption of white noise for the differenced series in a random walk is definitely plausible.Now, let us discuss the issues surrounding predictability in a randomwalk. We know that for a random walk the predicted value at the next timestep is the value at the current time step. That is all fine, but the purpose ofprediction is to make profits, and profits are made by correctly predictingthe increment to the random walk in the next time period. However, becausethe random walk is a martingale, the mean value of the predicted incrementis zero. The actual realized value of the increment is anybody’s guess. Doesthe situation improve when we try to predict values two time steps ahead?Not very much really. The mean value of the predicted increment is still
6 log(p ) − log(p ) ≈21
to be the return.
p2 − p1. Hence the difference in the logarithm may be construedp1
Time Series
31
3.303.253.203.153.103.05
Returns
10
30
50
FIGURE 2.6A
GE Series.
70
90
8
10
0.080.060.040.020.00–0.02–0.04–0.060
2
4
FIGURE 2.6B
6
Returns.
1.0Auto Correlation
Auto Correlation
0.8
0.05
0.00
0.60.40.2–0.0–0.2
–0.05
0
–2.9
FIGURE 2.6C
–0.8Lag
1.3
GE Returns Q-Q Plot.
FIGURE 2.6D
5
10Lag
15
GE Returns ACF.
20
BACKGROUND MATERIAL
32
zero. If anything, the variance of the normal distribution two time stepsaway increases, and the plausible range of values that the increment can assume actually increases, further increasing our prediction error. Therefore,knowing the past history of a random walk is not much help in predictingthe forward-looking increments.The situation is very different for stationary processes. Armed with theknowledge that stationary processes are mean reverting, one can predict theincrement to be greater than or equal to the difference between the currentvalue and the mean. The prediction is guaranteed to hold true at some pointin the future realizations of the time series.However, stock prices are modeled as a log-normal process, and that isdefinitely not stationary. So, where does that leave us in terms of makingprofits? Definitely not anywhere close to making money. The reader is probably wondering what the point of this whole chapter is. If the logarithm ofstock prices is assumed to be random walk, there is no need to go at it in aroundabout way. Just say it is futile trying to predict stock returns and leaveit at that. But all hope is not lost. We shall see in the later chapters that itmay be possible to construct portfolios whose time series are actually stationary, and the returns for those portfolios are indeed predictable. Let usstop here with this teaser.
SUMMARYA time series is constructed by periodically drawing samples from probability distributions that vary with time.The white noise process is the most elementary form of time series andis generated by drawing samples from a fixed distribution at every timeinstance.ARMA time series are generated using fixed linear combinations ofwhite noise realizations.Time series forecasting for ARMA processes involves deciphering thelinear combination and the white noise sequence used to generate thegiven data and using it to predict the future values.A random walk process is the time series where the current value is asimple sum of all the white noise realizations up to the present time.A random walk is a nonstationary time series.Nonstationary time series are usually transformed to stationary time series using differencing.The logarithm of the stock price series is usually modeled as a randomwalk.
Time Series
33
FURTHER READING MATERIALTime SeriesDiebold, Francis X. Elements of Forecasting. (Cincinnati, Ohio: South Western College Publishing, 1998).AICAkaike, H. “A New Look at Statistical Model Identification.” IEEE Transactions onAutomatic Control 19 (1974): 716–723.
BACKGROUND MATERIAL
34
APPENDIXLag 1 Correlation in a MA(1) SeriesThe variance of yt can be calculated using the preceding formulas as
( )
(
)= (1 + β ) var(ε )
( )
( )
(
)
var yt = var ε t + βε t −1 = var ε t + β 2 var ε t −1 + 2β cov ε t , ε t −1 =2
t
(
)
] = βE(ε ) =var(y ) var(y )var(y ) var(y )β var(ε )β==1+ βvar(y )[
2t −1
E yt yt −1
corr yt , yt −1 =
t −1
t
t
t −1
t
2
t
Therefore, unlike the white noise series, this series has a nontrivial correlation structure.
Lag 1 Correlation in a AR(1) SeriesThe variance at each time instant is given as
( ) (
( )
) ( )
var yt = E yt2 = 1 + α 2 + α 4 + α 6 + … var ε t
If a > 1, the series explodes and the variance becomes infinity. However,when a > 1, the variance can be calculated as the sum of an infinite geometric series and written as
( )
var yt =
( )
var ε t
1 − α2
The covariance is given as
(
)
[
]
[(
) ]
( )
cov yt , yt −1 = E yt yt −1 = E αyt −1 + ε t yt −1 = α var yt
Time Series
35
The correlation is therefore
(
)
corr yt , yt −1 =
(
) = α var(y ) = αvar(y )var(y )
cov yt , yt −1
t
t
t
Conditions under Which the Maximum LikelihoodIs Equivalent to Minimizing Sum of SquaresThe substitution of the logarithm of the likelihood criterion with the sum ofthe squared errors hinges on a key assumption. The assumption is that theerrors follow a normal distribution with a zero mean. Based on this assumption, every error value may be assigned a probability of occurrence.
( )
p ei =
(
)
 e / σ 2iexp−22π
1
We now make another assumption that the errors are independent of eachother. Then the probability (likelihood) of obtaining the following error sequence is the product of these probabilities.
(
N
) ∏ p(e )
p error =
i =1
i
Now, taking the logarithm of the above equation on both sides we have anexpression for logarithm of the likelihood.
(
N
N
) ∑ log[p(e )] = −2N log(2π ) − 2σ1 ∑ e
log likelihood =
i =1
i
2
i =1
2i
Let us examine our motivation for doing that. If we arrange a sequence ofnumbers in ascending or descending order and take their logarithms in sequence, the logarithms are guaranteed to be in ascending or descendingorder, as the case may be. We might say that transforming a set of numbersinto their logarithms preserves their ranks. Therefore, maximizing the loglikelihood is equivalent to maximizing the likelihood. We shall see in the following discussion that the log-likelihood can be simpler to deal with.
BACKGROUND MATERIAL
36
Examining the expression for logarithm of the likelihood, we see thatN
the only variable term is −
∑ ei2 . Note that this is the sum of squared errorsi =1
multiplied by the negative sign. Thus, maximizing the logarithm of the