excess returns on theRussell 3000 (RUS2) and the change in the 10-year interest rate over the 6month swap rate (∆MAT) are significant for the four indices. Not surprisingly, these two factors are also important in explaining the CTA returns inthe subperiods. ∆MAT is included as a factor in all the subperiods and isconsistently significant. RUS2 helps explain the variations in returns duringthe “Weak Bull” and “Moderate Bull” periods. The two subperiods during which the tailor-made factor model best captures the return variationsin the four indices are the “Weak Bull” and “Strong Bull” periods, whichshow adjusted R-squared of up to 40.4 percent. This leads us to concludethat given the appropriate risk factors, we are able to explain a considerable proportion of CTA returns in a linear setup. However, the results inTable 6.6 show that the factors having the best explanatory power changewith market conditions.As we noted earlier, the return characteristics of the Currency index andDiscretionary index are considerably different from those of the other fourindices, hence the factors that best capture their behavior are different.Tables 6.7 and 6.8 present the results for the tailor-made models for thesetwo indices for the entire period as well as the four subperiods.The Currency index proves to be the index for which the factors wereleast successful at explaining the excess returns (Table 6.7). For the entireperiod, the adjusted R-squared of the tailor-made model is 0.099. The resultsindicate that the returns on the currency index seem to exhibit an optionlikepayoff distribution as the series of returns on the one-month ATM call writ-
120
0.1220.1280.1940.133
0.3260.2930.4040.325
0.1500.1830.2240.187
CTA IndexSystematicFin/MetalDiversified
CTA IndexSystematicFin/MetalDiversified
R2adjRUS
−0.052−0.052−0.029−0.099
————
————
Alpha
−0.094−0.030−0.035−0.025
−0.1810.0950.2010.164
−0.019−0.014−0.172−0.0540.012*0.021*0.019*0.028**
Moderate Bull
0.0230.037*0.022**0.028
Weak Bull
0.014**0.018**0.015**0.021**
————
————
————
RUS3
Entire Period
RUS2
————
0.246*0.2380.0680.297*
0.0480.0620.0060.058
UMD
————
0.382**0.536**0.258**0.516*
0.0450.0630.0420.039
HDMZD
−0.181**−0.228**−0.196**−0.233*
0.044−0.045−0.115*0.089
−0.115**−0.161**−0.147**−0.154**
∆MAT
Tailor-Made Specification Results for CTA, Systematic, Financial and Metals, and Diversified Indices
CTA IndexSystematicFin/MetalDiversified
TABLE 6.6
————
————
————
∆DEF
121
RUS
————
————
Alpha
−0.506−0.757−1.011*−0.624
−0.141−0.140−0.054−0.139
** The values are significant at the 5 percent level.
** The values are significant at the 10 percent level.
0.1540.1630.1940.173
CTA IndexSystematicFin/MetalDiversified
R2adj
0.3350.3710.3330.358
(continued)
CTA IndexSystematicFin/MetalDiversified
TABLE 6.6
————
————Bear−0.001−0.001−0.001−0.002
————
RUS3
Strong Bull
RUS2
————
————
UMD
————
————
HDMZD
−0.153−0.202*−0.180−0.199
0.289*0.376**0.274**0.417*
∆MAT
————
0.522*0.591*−0.2630.781*
∆DEF
122TABLE 6.7
PERFORMANCE
Tailor-Made Model Results for Currency IndexR2adj
EntirePeriodWeakBullModerateBullStrongBullBear
Alpha
ATMC
DEF MAT
0.099 −3.188 −0.485** 2.364* —
FX
UMD HDMZD RUS2
0.099 0.083* 0.122**
—
0.372 −0.757*
—
—
—
—
—
—
—
—
—
—
—
—
0.090—
3.923—
0.273—
— −3.172 ————
——
——
——
0.332
0.409* 0.569** 0.030
ATMC = series of returns on the one-month ATM call written on the Russell 3000index. DEF = U.S. Moody’s Baa corporate bond yield. MAT = U.S. 10-year/6-monthInterest Rate Swap Rate. FX = monthly change in the U.S. dollar/Swiss francexchange rate. UMD (Up Minus Down) = average return on the two high priorreturn portfolios minus the average return on the two low prior return portfolios.HDMZD (High Dividend Minus Zero Dividend) = average return of the highestdividend-paying stocks versus the stocks that do not dispense dividends. RUS2 =square of the excess returns on the Russell 3000.** The values are significant at the 10 percent level.**The values are significant at the 5 percent level.
ten on the Russell 3000 index (ATMC) is a significant explanatory variable.Similar to the four previous indices, the “best-fit” regression is most successful at capturing the dynamics of the returns in the “Weak Bull” subperiod, with the adjusted R-squared equal to 0.332. For the “Moderate Bull”and “Bear” markets, no combination of risk factors manages to provide anyinsight into the return structure of the Currency index returns.Table 6.8 presents the tailor-made regression results for the Discretionary index. Although the results are not impressive when we consider theentire period (adjusted R-squared of 0.097), the market factors are successful at explaining the Discretionary index returns for all the subperiodswith the exception of “Strong Bull.” The results during the “Bear” periodare particularly impressive as the regression results report an adjustedR-squared of 0.47. The adjusted R-squared of the “Weak Bull” and “Moderate Bull” subperiods are comparable to those found for the previousindices; however, the factors that explain the variations in the returns aredifferent across the indices. Overall we find that the factors that bestexplain the excess returns on the discretionary index are the currency riskpremium (FX), the square of the excess returns on the Russell 3000 (RUS2),and the returns on the two commodity indices (GSCI and MCOM).
123
0.0970.3450.211—0.472
ATMC
————−0.202
Alpha
−0.212*−0.025−0.184—−0.092——−0.11**—0.166**
FX—0.117−0.123—−0.052*
UMD—0.089−0.096*——
HDMZD—0.092−0.091**——
∆MATRUS2
RUS3
0.091** 0.007**——0.018** −0.002**0.069————————
GSCI
————0.267**
MCOM
ATMC = series of returns on the one-month ATM call written on the Russell 3000 index. FX = monthly change in the U.S.dollar/Swiss franc exchange rate. UMD (Up Minus Down) = average return on the two high prior return portfolios minus theaverage return on the two low prior return portfolios. HDMZD (High Dividend Minus Zero Dividend) = average return of thehighest-dividend-paying stocks versus the stocks that do not dispense dividends. ∆MAT = change in the U.S. 10-year/6-monthInterest Rate Swap Rate. GSCI = return on the Goldman Sachs Commodity Index. RUS2 = square of the excess returns on theRussell 3000. RUS3 = cube of the excess returns on the Russell 3000. MCOM = return on Moody’s Commodity Index.** The values are significant at the 10 percent level.** The values are significant at the 5 percent level.
Entire PeriodWeak BullModerate BullStrong BullBear
R2adj
TABLE 6.8 Differentiated Model Results for Discretionary Index
124
PERFORMANCE
PERFORMANCE MEASUREMENTPerformance under Changing Market ConditionsThanks to the effort put in the previous section to explain CTA expectedreturns over the subperiods, we can go beyond the use of the Sharpe ratioto characterize abnormal performance as extensively used in the CTA performance literature. This ratio is extremely useful for ranking purposes, butnot to quantify the extent to which a given index has exceeded a benchmarkreturn. Furthermore, the pervasive departure from normality of CTAreturns casts doubt on the reliability of this performance measure, whichuses variance as the measure of risk.Here we apply four types of performance measures to each period:1. The alpha of the regressions;2. The Information Ratio (IR) (Grinold and Kahn 1992, 1995) defined asthe ratio of alpha over the standard deviation of residuals;33. The Generalized Treynor Ratio (GTR), which extends the originalTreynor ratio to a multi-index setup (Hübner 2003), defined as theratio of the alpha over the total required return; and4. The Treynor and Mazuy (1966) measure of market timing, which issimply the coefficient of the squared market return, proxied by RUS2in our specification.Although the alpha, the IR, and the GTR provide different portfolio rankings, the test for significance is essentially the same as it reduces to testingwhether alpha = 0, which is typically performed using a Student t-test.The analysis of Table 6.6 reveals unambiguous results on alphas. For allstrategies, the regression results never allow us to reject the hypothesis ofzero abnormal performance. The only noticeable exception is observed forthe Finance/Metals strategy, which underperforms the market at the 10 percent significance level in the “Strong Bull” subperiod. Notice that all thealphas of the four strategies are negative during this bullish period, whilethe three substrategies display positive, yet relatively small in magnitude andinsignificant, alphas during the “Weak Bull” period. This finding indicatesthat these types of CTA strategies tend to amplify market movement in theadverse direction. Not only are their required returns negatively correlatedwith market movement, but their abnormal performance is also contrarian.The Finance/Metals strategy seems to experience larger swings in both directions. The (insignificant) negative performance in the “Bear” market contra3Of course, the same caveat as for the Sharpe ratio applies to this measure as itimplicitly uses the variance as a risk measure.
The Performance of CTAs in Changing Market Conditions
125
dicts this analysis, as the CTAs did not benefit from market conditions thatshould have favorably influenced their market contrarian strategies.At the aggregate level, the magnitude of the (negative) alphas is ratherlow, but this has to be related to the low significance levels of the regressions resulting from the extreme heterogeneity of CTA behavior from onesubperiod to another. Of course, these conclusions can be generalized to theIR and GTR performance measures, as none of the alphas is significant.The analysis of Tables 6.7 and 6.8 is very different. The Currency indexpresented a negative (insignificant) alpha over the whole period, but mostlydue to times in which we could not find any significant linear relationshipwith the factors (“Moderate Bull” and “Bear”). During the “Weak Bull”and “Strong Bull” periods, alphas were positive although not significantlydifferent from zero. This is at least evidence that Currency CTAs, on average, did not follow the same amplifying strategies as the ones displayed inTable 6.6 but that they could extract some additional returns. The Discretionary index, on the other hand, exhibited negative abnormal performanceover all subperiods, and the aggregate abnormal return over the entireperiod is even significantly negative (Table 6.8).The Treynor and Mazuy (1966) measure of market timing ability, captured by the coefficient for RUS2, is much more informative. As a reminder,this coefficient is meant to account for the loading of the skewness-relatedrisk premium: The greater this value, the more likely it is that the portfolioreturns will have a positive (right) asymmetry, thus putting more weight tothe more positive returns. When considered in the context of performancemeasurement, RUS2 captures the manager’s market timing abilities, as itgives an asymmetric weight to positive and negative deviation from the meanmarket excess return. This interpretation is valid provided the expectedmarket excess return is positive. For example, with a mean return of 1 percent and a coefficient of 1, a deviation of +1 percent with respect to thisvalue will provide a positive return of 1 × (1% + 1%)2 = 4%, while a deviation of −1 percent will provide a return of 1 × (1% − 1%)2 = 0%. Thus, apositive coefficient signals positive market timing when markets are bullishand negative market timing ability otherwise.For the CTA strategies reported in Table 6.6, market timing abilities arepervasive during the total period, mainly due to the “Weak Bull” and “Moderate Bull” periods. During the (much shorter) “Strong Bull” and “Bear”periods, this effect completely fades away; it does not even intervene in thetailor-made regressions. Very noticeable is the same positive sign of thealpha and the market timing coefficients during the “Weak Bull” period, afinding that contrasts with many previous studies of abnormal performanceof managed portfolios.44See Bello and Janjigian 1997 for a review.
126
PERFORMANCE
Tables 6.7 and 6.8 display again very different results, as the Currencyindex does not provide any evidence of market timing abilities while theregression for the Differentiated index supports positive market timing abilities for the total period, mainly driven by the “Weak Bull” period.To summarize, available evidence seems to indicate that CTAs couldgenerate asset selection as well as market timing performance during thefirst part of the sample period, but this performance seems to have fadedaway. There is no indication of positive or negative alpha or Jensen-Mazuycoefficient during the “Strong Bull” and “Bear” periods, even though consistently, yet not significantly, negative alphas do not suggest any positiveportfolio abnormal performance of CTA funds during this period.
Performance during Extreme EventsIn the previous section we studied the performance of CTA indices under different market conditions. Now we seek to take the investigation one stepfurther and examine the behavior of these funds when exposed to extrememarket fluctuations. Earlier we identified three specific events that causedsignificant short-term shocks in the overall market during our sample period:the Russian debt crisis, the Asian currency crisis, and the September 2001terrorist attacks in the United States. These three events caused a considerable drop in market indices (we use the S&P 500 as our benchmark), and itgenerally took two months for the markets to revert to their preevent levels.We therefore seek to investigate the performance of the different CTA indicesduring the two-month period comprising the event and the recovery.To measure the abnormal performance of a CTA index, we calculate itsstandardized abnormal returns over T months as:T
SARi ,T =
∑ ARi ,t
t =1
s(ARi ) T
T = 1, 2(6.3)k
with
ARi , t = Ri , t − α i − ∑ β i , j Fj , tj =1
where, for index i,ARi,t = the abnormal return in month tRi,t= the return in month tai= unexplained return by asset-class factorsbi,j= factor loading on the jth asset-class factorFj,t= value of the jth asset-class factor in month ts(ARi ) = standard deviation of abnormal returns over entiresample period
127
The Performance of CTAs in Changing Market Conditions
TABLE 6.9
Abnormal Performance during Extreme EventsT
CTA Index
1 month2 months
Systematic
1 month2 months
Fin/Metal
1 month2 months
Diversified
1 month2 months
Discretionary
1 month2 months
RussianCrisis
AsianCrisis
TerroristAttack
−2.78(2.32)−2.01(3.28)
−0.01(2.32)0.38(3.28)
−0.54(2.24)1.45(3.17)
−2.75(2.66)−1.94(3.77)
0.02(2.66)0.17(3.77)
−0.12(2.76)3.14(3.91)
−3.55*(2.11)−3.03(2.98)
−0.45(2.11)1.59(2.98)
−0.28(2.16)3.19(3.06)
−3.27(3.10)−3.00(4.39)
0.34(3.10)0.92(4.39)
0.32(2.98)3.69(4.22)
−1.21(1.13)−1.72(1.59)
2.03*(1.13)3.55**(1.59)
0.57(0.96)0.53(1.36)
Table 6.9 presents the results for the measures of abnormal performance for the different CTA indices for one-month and two-month periodsfollowing the extreme events.According to the results in Table 6.9, no abnormal performance for theCTA indices appears to exist, with the noticeable exceptions of the Financial/Metal index during the first month of the Russian crisis and the Discretionary index during the Asian crisis. For the latter index, the abnormalperformance is significantly positive and robust during the entire Asian crisis. It sharply contrasts the very low abnormal returns achieved by all otherindices under the same circumstances.In general, the Russian crisis appears to have a negative effect on CTAabnormal performance. Although the individual coefficients are not significant, they are uniformly negative. On the other hand, the Asian crisis, andmore surprisingly the terrorist attacks, yield very small t-values for all theCTA indices.
128
PERFORMANCE
CONCLUSIONThroughout our analysis of the behavior of CTA indices during the 1990 to2003 period, we have outlined that the splitting of the time window into atleast four subperiods is beneficial to capture the sensitivity of CTA returnsto broad sources of risk. With our tailor-made specifications, we can explainan average of 25 percent of the variance of returns, which is much greaterthan the accuracy obtained using the traditional multifactor or multimoment analyses.Thanks to this improvement over classical specifications, we can soundlyassess the abnormal performance of CTA strategies during changing marketconditions. Among the indices studied in this chapter, only the CurrencyCTA index seems to exhibit significant security selection as well as markettiming abilities. Although it is usually not significant, the performance ofCTA indices during the most extreme market fluctuation,—“Strong Bull”and “Bear” market conditions—is typically negative and does not suggest thatthese investment vehicles could benefit from either type of market condition.No severe market crisis seems to have affected CTA performance withthe noticeable exception of the Asian crisis, whose exploitation by the Discretionary CTA strategy caused significant abnormal returns for investors.Overall, this study indicates that most of the variance of CTA returnsremains unexplained by traditional risk factors, at least in a linear setup.There is, however, considerable evidence of positive market timing abilityassociated with these types of securities.
CHAPTER
7
Simple and Cross-Efficiencyof CTAs Using DataEnvelopment AnalysisFernando Diz, Greg N. Gregoriou, Fabrice Rouah,and Stephen E. Satchell
e apply data envelopment analysis and use the basic and crossefficiency models to evaluate the performance of CTA classifications.With the ever-increasing number of CTAs, there is an urgency to providemoney managers, institutional investors, and high-net-worth individualswith a trustworthy appraisal method for ranking their efficiency. Dataenvelopment analysis can achieve this, eliminating the need for benchmarks, thereby alleviating the problem of using traditional benchmarks toexamine nonnormal returns. This chapter studies CTAs and identifies theones that have achieved superior performance or have an efficiency score of100 in a risk/return setting.
W
INTRODUCTIONResearch into the performance persistence of commodity trading advisors(CTAs) is sparse, so there is little information on the long-term diligence ofthese managers (see, e.g., Edwards and Ma 1988; Irwin, Krukemeyer, andZulaf 1992; Irwin, Zulauf, and Ward 1994; Kazemi 1996). It is generallyagreed that during bear markets, CTAs provide greater downside protectionthan hedge funds and have higher returns along with an inverse correlationto equities. The benefits of CTAs are similar to those of hedge funds, in thatthey improve and may offer a superior risk-adjusted return trade-off tostock and bond indices and can act as diversifiers in investment portfolios(Schneeweis, Savanayana, and McCarthy 1991; Schneeweis, 1996).
129
130
PERFORMANCE
Investors who have chosen to include CTAs in their portfolios haveallocated only a small portion of their assets. This can be attributed to themediocre performance of CTAs during the early 1990s (Georgiev 2001).Others are unaware that during periods of increased stock market volatility, careful inclusion of CTA managers into investment portfolios canenhance their returns especially during severe bear markets (Schneeweis andGeorgiev 2002). Moreover, extreme volatility in international financialmarkets of this past decade, such as that experienced during the Asian currency crisis of 1997 and the Russian ruble crisis of August 1998, did notsignificantly affect CTA performance. In fact, during these periods of highvolatility, CTAs make most of their money and produce superior returnsrelative to traditional market indices.Much recent debate has centered on how to measure and evaluate theperformance of CTAs. Comparing CTAs to standard market indices could beerroneous since CTAs are viewed as an alternative asset class and possess different characteristics from traditional stock and bond portfolios. Unlikemutual funds, it is difficult to identify factors that drive CTA returns(Schneeweis, Spurgin, and Potter 1996). Fung and Hsieh (1997b) applySharpe’s factor “style” analysis to CTAs and find that very little of the variability in CTA returns can be attributed to variability of financial assetclasses (in marked contrast to what Sharpe (1992) finds for mutual funds).They attribute the low R-squared values to the dynamic strategies of CTAs.Investors and analysts placing too much faith in these models are thereforeat risk of being misled by biased alphas (Schneeweis, Spurgin, and McCarthy1996). The underlying question of which benchmarks would be appropriatefor each CTA strategy continues to be a controversial one.How performance is measured also can be the reason for divergentresults. Excess returns can display performance persistence when in fact itis nonexistent. A recent study by Kat and Menexe (2002) suggests that thepredictability in returns is low.The nonnormal returns that CTAs often display make it difficult toapply linear factor models that use traditional market indices since these donot offer a sufficient measure of CTA risk exposure. Fung and Hsieh(1997b) argue that the explanatory power of these models is weak and propose an extension of Sharpe’s model to CTAs whereby specific CTA“styles” are defined. The traditional Sharpe ratio usually overestimates andmiscalculates nonnormal performance, because this well-known riskadjusted measure does not consider negative skewness and excess kurtosis(Brooks and Kat 2001).Using CTA indices to examine performance persistence also can introduce biases. CTA indices are rebalanced and cannot properly reproduce the
Simple and Cross-Efficiency of CTAs Using Data Envelopment Analysis
131
same composition during an entire examination period; consequently persistence could be wrongly estimated.Regardless of the capability of existing and frequently used models toexplain CTA returns, the dynamic trading strategies and skewed returnsremain critical issues in the CTA performance literature, and further investigation is warranted.We use simple and cross-efficiency DEA models to handle the problemsencountered when using multifactor models to predict CTA returns. DEAallows us to appraise and rank CTAs in a risk-return framework withoutusing indices. The efficient frontier is generated from the most efficientCTAs, and DEA calculates the efficiency of each CTA relative to the efficient frontier, thereby producing an efficiency score according to the inputand output variables used. The selection of variables is discussed in themethodology and data section. DEA is a nonparametric technique thatmeasures the relative efficiency of decision-making units (DMUs) on thebasis of observed data and then presents an efficiency score as a single number between 0 and 100.1 The main benefit of DEA is that it identifies thebest-performing CTA and determines the relative efficiencies of a set of similar CTAs (peers). DEA, also called frontier analysis, was originally developed by Charnes, Cooper, and Rhodes (CCR) (1978). It was later adaptedby Banker, Charnes, and Cooper (BCC) (1984), who expanded the Farrell(1957) technical measure of efficiency from a single-input, single-outputprocess to a multiple-input, multiple-output process. The CCR and BCCmodels are the simple (or basic) DEA models and were developed originallyfor nonprofit organizations. Later we discuss an alternative DEA model:cross-efficiency.The power of DEA is in its ability to deal with several inputs and outputs while not requiring a precise relation between input and output variables. DEA produces an efficiency score which takes into account multipleinputs and outputs, and uses the CTAs themselves as the benchmark. Usingan alternative performance measure like DEA is beneficial because itenables investors to potentially pinpoint the reasons behind a CTA’s poorperformance. Once the weaknesses are recognized, the CTA can attempt toreach a perfect efficiency score by comparing itself to CTAs that haveachieved an efficiency score of 100. Furthermore, numerous DEA softwareprograms, such as the DEA solver in Zhu (2003), and Banxia’s Frontier
1An efficiency score of 100 refers to an efficient fund (or best-performing fund thatlies on the frontier); a score of less than 100 signifies the fund is inefficient.
132
PERFORMANCE
Analyst, provide an improvement summary that can pinpoint the weaknesses from the CTA’s inputs and outputs.For institutional investors considering using CTAs as downside protection in bear markets, it is critical that a performance measure provide notonly a precise appraisal of the CTA’s performance, but also an idea of thequality of its management with respect to certain criteria (variables such asinputs and outputs). Using DEA could present investors with a useful toolfor ranking CTAs, not by historical returns, but by peer group appraisal.In the next section we discuss the different DEA methodologies. Thenwe describe the data, discuss the empirical results, and summarize ourconclusions.
METHODOLOGYIn its most rudimentary form, DEA calculates an efficiency score thatdescribes the relative efficiency of a CTA when compared to other CTAs inthe sample. The first step in DEA is to obtain an efficient frontier from theinputs and outputs identified by Pareto optimality.2 DEA then calculates theefficiency score of each DMU relative to the efficiency frontier. In this chapter, the DMUs are CTAs.The efficiency frontier consists of the “best-performing” CTAs—themost efficient at transforming the inputs into outputs (Charnes, Cooper, andRhodes, 1981). Any CTA not on the frontier would have an efficiency scoreless than 100 and would be labeled inefficient. For example, a CTA with anefficiency score of 80 is only 80 percent as efficient as the top-performingCTA. A best-performance frontier charts the maximum or minimum level ofoutput (input) produced for any assumed level of input (output), where outputs represent the degree to which the CTA’s goal has been achieved.How the inputs and outputs are used in the efficiency analysis areessential because they establish the grounds on which the efficiency of thefund is calculated. The most extensively used DEA technique to measureefficiency takes the weighted sum of outputs and divides it by the weightedsum of inputs (Golany and Roll, 1994). In its simplest form, DEA calculatesweights from a linear program that maximizes relative efficiency with a set
2Pareto optimality means the best that can be attained without putting any group at
a disadvantage. In other words, a group of funds becomes better off if an individualfund becomes better off and none becomes worse off.
Simple and Cross-Efficiency of CTAs Using Data Envelopment Analysis
133
of minimal weight constraints.3 Charnes, Cooper, and Rhodes (1978) proposed reducing the multiple-input, multiple-output model to a ratio with asingle virtual input and a single virtual output.
Simple EfficiencyThe main distinction between the two simple DEA models is that the BCCmodel uses varying returns to scale to examine the relative efficiency ofCTAs, while the CCR model uses constant returns to scale.4 To obtainrobust results, a proper working sample ought to be on the order of threetimes the number of CTAs as the number of input and output variables(Charnes, Cooper, and Rhodes, 1981). In addition, DEA uses a comparative measure of relative performance framework.We adapt the notation from Adler, Friedman, and Stern (2002) for thesimple and cross-efficiency models. By comparing n CTAs with s outputs,denoted by yrk in equation 7.1, where r = 1, . . . , s, and m inputs denotedby xik, i = 1, . . . , m,