Top Bottom Top Bottom Top BottomCOR COR Return Return Skew Skew MinRet MinRet Kurt Kurt
Top Std. Dev. 1BottomStd. Dev. −0.051 1TopPerNeg−0.057 −0.009 1BottomPerNeg−0.076 0.158 −0.027 1Top COR0.152 0.126 −0.061 −0.001 1BottomCOR−0.137 0.038 0.044 −0.044 −0.142 1TopReturn−0.135 0.101 −0.078 −0.189 0.076 0.238 1BottomReturn0.088 0.258 0.122 –0.082 0.089 −0.098 0.063 1Top Skew−0.060 −0.163 −0.102 −0.138 0.146 −0.055 −0.052 0.070 1BottomSkew0.009 −0.002 −0.138 0.181 0.039 0.179 0.183−0.028 1TopMinRet0.052 −0.094 −0.201 0.106 −0.021 −0.014 −0.127 0.220 −0.012 −0.001BottomMinRet0.149 0.119 −0.041 −0.064 0.067 −0.037 −0.171 0.206 0.021 0.116Top Kurt−0.112 0.131 −0.069 0.004 −0.009 0.071 0.032 0.076 −0.094 0.001BottomKurt0.091 0.003 −0.139 0.114 0.146 0.088 0.003 −0.080 0.092 0.168
2000–2002
TABLE 5.7 (continued)
95
CTA Performance Evaluation with Data Envelopment Analysis
TABLE 5. 8 Summary Statistics for Variables Used in Regression AnalysisVariables
Mean
Std. Dev.
Min
Max
BetaBeta SquaredAverage FundManagedLength ofManagerTrack Record
−0.0680.046
0.2050.096
−0.7820.000
0.4700.612
$90,659,049
$175,566,905
$86,542
$1,172,390,042
11.055
4.362
5.667
22.167
Variables
Mean
Std. Dev.
Min
Max
BetaBeta SquaredAverage FundManagedLength ofManagerTrack Record
−0.0630.090
0.2940.159
−0.8700.000
0.8680.756
$92,303,454
$222,082,600
$92,542
$2,078,385,875
11.055
4.362
5.667
22.167
METHODOLOGYBrief Background of Data Envelopment AnalysisData envelopment analysis, a mathematical programming approach, wasfirst developed by Charnes, Cooper, and Rhodes (1978) to measure the efficiency or performance of individual decision-making units (DMUs) in producing multiple outputs from multiple inputs. Unlike a parametricapproach (like regression-based methods), which requires the researcher tomake sometimes arbitrary assumptions about the functional relationshipbetween inputs and outputs, the DEA approach does not require suchassumptions. It allows us to create an efficient frontier based on the inputoutput combinations of the observed DMUs, without any apriori assumptions regarding the functional form of the relationship between them.Consider an industry producing a vector of m outputs y = (y1,y2, . . . , ym) from a vector of n inputs, x = (x1, x2, . . . , xn). Let the vectorsxj and yj represent, respectively, the input and output bundles of the j-thdecision-making unit. Suppose that input-output data are observed for NDMUs. Then the technology set can be completely characterized by the production possibility set T = {(x, y) : y can be produced from x} based on afew regularity assumptions, which in case of variable returns to scale are:
96
PERFORMANCE
1. All observed input-output combinations are feasible.(xj, yj) Œ T; (j = 1,2, . . . N)2. T exhibits free disposability with respect to inputs.(x0, y0) Œ T and x1 ≥ x0 ⇒ (x1, y0) Œ T3. T exhibits free disposability with respect to outputs.(x0, y0) Œ T and y1 ≤ y0 ⇒(x0, y1) Œ T4. T is convex.(x0, y0) ŒT and (x1, y1) Œ T⇒ (lx0 + (1 − l)x1, ly0 + (1 − l)y1) ŒT; 0 ≤ l ≤ 1Within the DEA approach, efficiency16 can be measured based on eitherof two orientations. The first yields an output-oriented measure of efficiencythat describes the maximum proportional increase in outputs that can beachieved for the given level of inputs from the DMU. The second orientationyields an input-oriented measure for the maximum proportional reductionin inputs that can be achieved for the given level of outputs of the DMU.Following Banker, Charnes, and Cooper (BCC) (1984) we can measurethe output-oriented efficiency of the ith DMU by solving this linear programming problem:17Max fiSubject toN
∑ λ j yrj ≥ φi yri
r = 1, 2, . . . , m
j =1N
∑ λ j xsj ≤ xsi
s = 1, 2, . . . , n
j =1
(5.1)
N
∑ λj = 1j =1
λj ≥ 0
j = 1, 2, . . . , N
For an efficient DMU fi = 1, whereas for an inefficient DMU fi > 1.On the other hand, an input-oriented measure of efficiency can beobtained for the ith DMU by solving the linear programming problem:
16The concept of efficiency used here is that of technical efficiency. It is used in thecontext of an expanded efficient frontier with n variables across n dimensions,rather than just the two familiar mean and variance dimensions.17While the Charnes, Cooper, and Rhodes, (1978) model assumes constant returnsto scale, the model proposed by Banker, Charnes, and Cooper (1984) allows forvariable returns to scale.
CTA Performance Evaluation with Data Envelopment Analysis
97
Min qiSubject toN
∑ λ j yrj ≥ yri
r = 1, 2, . . . , m
j =1N
∑ λ j xsj ≤ θ i xsi
s = 1, 2, . . . , n
j =1
(5.2)
N
∑ λj = 1j =1
λj ≥ 0
j = 1, 2, . . . , N
In this case an efficient DMU will have qi = 1, whereas an inefficientDMU will have qi < 1.One requirement of these two models is that the inputs and outputsmust not be negative. However, the BCC output-oriented model (5.1) isinvariant to input translations, and the BCC input-oriented model (5.2) isinvariant to output translations (see Ali and Seiford 1990). By choosing theappropriate model, we are able to handle the case of negative outputs ornegative inputs by translation.
Application of DEA to the Study of CTA PerformanceIn this study our objective is to obtain a multicriteria measure of efficiencyfor each individual CTA in our sample. Wilkens and Zhu (2001) provide amotivation for applying DEA to measure the efficiency of an individualCTA based on multicriteria. They also provide a detailed illustration of howDEA can be used for the evaluation of CTA efficiency. Following a similarapproach, we measure the efficiency of each CTA by treating the standarddeviation of returns and proportion of negative returns as “inputs” in theDEA model; we treat return (average monthly return), minimum return,skewness, and kurtosis “outputs” in the DEA model.18Since many of our outputs were negative for several CTAs, we had totranslate them to obtain positive values.19 (Table 5.1 shows the summary18Our model differs from that of Wilkens and Zhu (2001) because we use kurtosis
as an additional “output” in our model.19These translations were used to make each of our outputs positive: (1) return: Weadded 0.04 (i.e., 4 percent) to the return of each CTA; (2) minimum return: We added1 to the minimum return of each CTA; (3) skewness: We added 5 to the skewnessof each CTA; (4) kurtosis: We added 3 to our original measure of excess kurtosis foreach CTA (thus obtaining measures of kurtosis rather than excess kurtosis).
98
PERFORMANCE
statistics for the original data; Table 5.9 shows the summary statistics forour translated data.) As a result of this translation, we chose the input-oriented BCC model to measure the efficiency of each individual CTA since itis invariant to output translations.20We follow Wilkens and Zhu (2001), but also add kurtosis to the model.Although extreme value theory generally views kurtosis as indicative of morerisk, we take a more neutral approach by controlling for skewness, kurtosis,and return outputs while minimizing standard deviation and the percent ofnegative returns. One reason that we treat kurtosis as an output rather thanas an input to the DEA model is the fact that our input-oriented DEA modelonly has limited ability to translate negative inputs. Another more compellingreason is that in our sample of CTA returns, the mean skewness is positive,indicating that extreme values are more often positive than negative.
Tobit Regressions: Explaining the Differencesin Efficiency of CTAsOnce we measure the input-oriented efficiency scores for the individualCTAs in our sample, we address the question of what leads to the differences in efficiencies. We explore the potential for the size of the fund, thelength of the fund’s track record, its investment style, and investment strategy to explain the degree of efficiency in terms of the DEA criteria (maximizing monthly returns, minimum returns, skewness, and kurtosis andminimizing standard deviation of returns and percentage of negativemonthly returns). However, we cannot carry out standard ordinary leastsquares (OLS) regression of efficiency scores (qi ≤ 1) on the explanatoryvariables because the efficiencies scores of a number of CTAs in our sampleare clustered at the upper limit of 1. Because the dependent variable, whichis the efficiency score, is censored, the appropriate model to use in this context is a Tobit regression model, which is a limited-dependent-variablemodel. (See Greene 2000.) In this study, therefore, we use Tobit regressionmodels to explain the differences in efficiencies across CTAs.
RESULTSTable 5.10 presents the frequency distribution for the efficiency scores of all157 CTAs. Overall, the scores are higher during the first (up-market) time20We recognize that standard deviation and percentage of negative returns are notreally inputs that are used to produce the outputs (returns, minimum returns, skewness, and kurtosis). Nevertheless, we use the terms “inputs” and “outputs” heresimply to convey clearly how each of these criteria is being used within the constructof the DEA model.
99
Mean of CTAsStandardDeviationMinMax
Mean of CTAsStandardDeviationMinMax
Mean of CTAsStandardDeviationMinMax
Mean of CTAsStandardDeviationMinMax
All 216 CTAsfor 1998–2000
All 216 CTAsfor 2000–2002
157 CTAsfor 1998–2000
157 CTAsfor 2000–2002
Original Inputs& Adjusted Outputs
0.0310.0130.191
0.058
0.0260.0220.155
0.056
0.0340.0040.245
0.055
0.0340.0050.193
0.055
StandardDeviation
0.0940.2080.750
0.481
0.0920.2920.750
0.453
0.1180.1250.750
0.456
0.1240.0420.750
0.427
PercentNegative
0.0090.0080.067
0.045
0.0080.0160.072
0.048
0.0110.0080.113
0.046
0.0130.0160.151
0.050
AverageMonthlyReturn
TABLE 5.9 Summary Statistics for Translated Data Values of the DEA Model Criteria
0.5963.4306.471
5.247
0.7053.4847.224
5.420
0.8970.5586.981
5.183
0.9292.8808.694
5.460
Skewness
0.0670.6150.982
0.891
0.0490.7540.988
0.907
0.0760.5170.997
0.897
0.0630.4700.994
0.909
MinimumReturn
1.1601.6727.748
3.635
1.6441.5029.249
4.040
2.3651.67223.812
4.223
2.7321.47619.370
4.637
Kurtosis
100
PERFORMANCE
TABLE 5.10 Frequency Distribution for Efficiency ScoresEfficiency Range
1998–2000
2000–2002
<0.40.4 – <0.50.5 – <0.60.6 – <0.70.7 – <0.80.8 – <0.90.9 – <11
02292735261325
11242422617413
OverallMeanStandard DeviationMinMax
1570.7650.1580.4121
1570.6820.1530.3841
period with an average efficiency of 76.5 percent, in contrast to an averageof 68.2 percent during the second (down-market) period.Table 5.11 breaks the results down by investment style21 (diversifiedversus nondiversified) and shows that the mean and the standard deviationof the two groups are very close. There is virtually no difference in the efficiency scores between the two investment style groups.Table 5.12 breaks the results down by investment strategy (systematic,discretionary, and trend-following). There is weak evidence that the systematic strategy outperforms the other strategies on the basis of the performancecriteria used in this study. In both periods, the systematic strategy has thehighest mean efficiency score with a relatively low standard deviation.Determinants of the efficiency scores (theta) are investigated usingTobit regressions with efficiency score as the dependent variable. The variables include beta, beta-squared, average funds under management, lengthof manager track record, and dummy variables for the investment styles andstrategies. Table 5.13 presents a correlation matrix for all of these variables.Tables 5.14 through 5.16 provide the results of three Tobit regressionsand indicate that beta-squared is a significant factor inversely affecting theefficiency scores during both time periods. Beta and the length of the manager’s track record (maturity) also inversely impact the efficiency scores, but21Because there are so many diversified CTAs in the database, we group together all
of the CTAs that are not labeled as diversified. This results in only two groups:diversified and nondiversified.
101
CTA Performance Evaluation with Data Envelopment Analysis
TABLE 5.11 Frequency Distribution of Efficiency Scores, by Investment StyleDiversified
Nondiversified
Efficiency Range
1998–2000
2000–2002
1998–2000
2000–2002
<0.40.4 – <0.50.5 – <0.60.6 – <0.70.7 – <0.80.8 – <0.90.9 – <11SumMeanStandard DeviationMinMax
01231621111011930.7450.1560.4121
1102025181027930.6790.1550.3841
016111415314640.7930.1570.4431
0222178726640.6870.1510.4691
TABLE 5.12 Frequency Distribution of Efficiency Scores, by Investment StrategyDiscretionary
Systematic
Trend Based
EfficiencyRange
1998–2000
2000–2002
1998–2000
2000–2002
1998–2000
2000–2002
<0.40.4 – <0.50.5 – <0.60.6 – <0.70.7 – <0.80.8 – <0.90.9 – <11
01320303
11511003
01181426151019
062628181438
008119833
0511137312
SumMeanStandardDeviationMinMax
120.755
120.671
1030.778
1030.695
420.736
420.655
0.2040.4121
0.2220.3841
0.1570.4431
0.1480.4021
0.1450.5081
0.1430.4501
TABLE 5.13 Correlation Matrix for Variables Used in the Regression Analysis1998–2000
Theta
Beta
AveMgFunds
Maturity
BetaSquared
ThetaBetaAveMgFundsMaturityBeta-Squared
1−0.079−0.004−0.186−0.174
10.051−0.046−0.524
10.298−0.073
10.057
1
2000–2002
Theta
Beta
AveMgFunds
Maturity
BetaSquared
ThetaBetaAveMgFundsMaturityBeta-Squared
10.0280.024−0.088−0.245
1−0.1040.143−0.144
10.205−0.055
10.093
1
Note: Theta represents the efficiency score from the DEA analysis.
TABLE 5.14 Results of Tobit RegressionsVariables
1998–2000
2000–2002
Intercept
0.7446(0.0544)−0.1651***(0.0595)0.0008(0.0006)−0.0060**(0.0025)0.0192(0.0222)0.0421(0.0442)0.0212(0.0462)−0.3477***(0.1235)
0.6276(0.0502)−0.0336(0.0362)0.0005(0.0004)−0.0024(0.0024)−0.0081(0.0209)0.0750*(0.0433)0.0460(0.0464)−0.1448**(0.0649)
BetaAvFundsMaturityIS2S3Beta-squared
The figures in parentheses are the standard errors.I, S2, and S3 are dummy variables for non-diversified investment style, systematic investment strategy, and trend-basedinvestment strategy, respectively.***The coefficient is significant at 1 percent.***The coefficient is significant at 5 percent.***The coefficient is significant at 10 percent.
103
CTA Performance Evaluation with Data Envelopment Analysis
TABLE 5.15 Results of Tobit RegressionsVariables
1998–2000
2000–2002
Intercept
0.7912(0.0292)−0.1517***(0.0588)0.0010(0.0006)−0.0066***(0.0025)−0.3515***(0.1221)
0.6916(0.0280)−0.0305(0.0346)0.0005(0.0005)−0.0026(0.0024)−0.1692***(0.0626)
BetaAvFundsMaturityBeta-squared
The figures in parentheses are the standard errors.***The coefficient is significant at 1 percent.***The coefficient is significant at 5 percent.***The coefficient is significant at 10 percent.Note that Average Funds is significant at the 10.08 percentlevel of significance during 1998 to 2000.
TABLE 5.16 Results of Tobit RegressionsVariables
1998–2000
2000–2002
Intercept
0.7727(0.0348)−0.1619***(0.0595)0.0010*(0.0006)−0.0062**(0.0025)0.0217(0.0224)−0.3622***(0.1221)
0.6574(0.0335)−0.0428(0.0349)0.0005(0.0004)−0.0021(0.0024)0.0393*(0.0217)−0.1531**(0.0625)
BetaAvFundsMaturityS2Beta-squared
The figures in parentheses are the standard errors.*** The coefficient is significant at 1 percent.*** The coefficient is significant at 5 percent.*** The coefficient is significant at 10 percent.
104
PERFORMANCE
these results are statistically significant during the first time period only.The coefficient for dummy variable for the systematic strategy is positiveand statistically significant during the second time period.
CONCLUSIONWe illustrate the use of the DEA methodology in an application for evaluating CTA performance, and we explore the relationship between such performance and fund size, length of the manager’s track record, investmentstyle and strategy, and measures of the covariance of CTA returns withequity market returns. Performance is quantified by efficiency scoreswhere 1 (or 100 percent) indicates perfect efficiency and scores lower than1 represent relatively less efficient CTAs based on the performance criteriachosen.We find some evidence that emerging hedge fund managers outperformestablished managers. Specifically, the length of the managers’ track recordis negatively related to our DEA model efficiency scores, but the size of thefund is not related to efficiency scores.We also find that beta-squared is a significant factor inversely affectingthe efficiency scores during both strong and weak equity market environments. Beta also inversely impacts the efficiency scores, but the results forbeta are statistically significant during the first time period only.Our preliminary results indicate that strategies (systematic, discretionary, trend-based) and styles (diversified, financial, currency, etc.) do notaffect efficiency scores calculated by the DEA methodology presented here.One exception is that the systematic strategy does well relative to the otherstrategies in a statistically significant sense during the second time period offlat and weak equity markets.
CHAPTER
6
The Performance of CTAsin Changing Market ConditionsGeorges Hübner and Nicolas Papageorgiou
his chapter studies the performance of 6 CTA indices during the period1990 to 2003. Four distinct phases of financial markets are isolated, aswell as three extreme events. We show that traditional multifactor as wellas multimoment asset pricing models do not adequately describe CTAreturns for any of the subperiods. With a proper choice of risk factors, wecan, however, explain a significant proportion of CTA returns and assessthe abnormal performance of each strategy. Most indices display null ornegative alphas, but they seem to exhibit positive market timing abilities.The currency index reports both types of positive performance during thefirst subperiod. Severe market crises do not seem to affect abnormal CTAreturns, except the Asian crisis, which benefited investors in the discretionary index. The Russian crisis has a uniform, although insignificant,negative impact on CTA abnormal returns.
T
INTRODUCTIONSince the blossoming of an extensive literature on hedge funds, commoditytrading advisors (CTAs) have profited from renewed interest amongresearchers. Following the initial studies by Brorsen and Irwin (1985) andMurphy (1986), Elton, Gruber, and Rentzler (1987) ascertained that commodity funds were not likely to provide a superior return to passively managed portfolios of stocks and bonds. As a result of these discouragingfindings, for over a decade very little research was devoted to the analysisof CTAs.Fung and Hsieh’s paper (1997a) on the analysis of hedge fund performance rekindled academic interest in CTAs. In their paper the authors noticethat the return distributions of certain hedge funds share some important
105
106
PERFORMANCE
characteristics with those of CTAs. Subsequently, Schneeweis and Spurgin(1997), Brown, Goetzmann, and Park (2001), and Edwards and Caglayan(2001) performed studies on a joint sample of CTA and hedge fund data.Fung and Hsieh (1997b) analyzed these two investment vehicles independently and discovered that CTA returns exhibit optionlike dynamics that mayprovide them with a peculiar role in portfolio management. Liang (2003)explicitly separated CTAs and hedge funds in his analysis and concluded thataside from the particular management rules that differentiate them fromhedge funds, CTAs exhibit very low correlation with hedge funds strategies.Although they seem to underperform hedge funds and even funds-of-fundsstrategies in bullish markets, Edwards and Caglayan (2001) and Liang(2003) discovered that their creditable behavior in bearish market conditionsindicates that CTAs could represent precious hedging instruments whenmarkets are in a downtrend. This atypical behavior can be attributed at leastin part to the nonnormality of the return structure of CTAs.Although the particular return distributions of CTAs are now recognized, the measurement of their performance has yet to be adapted. Bymimicry with the large stream of performance studies on mutual funds, virtually all studies on hedge funds have adopted the classical Sharpe ratio(1966) and Jensen’s alpha (1968) as relevant performance measures. Thesequestionable choices become all the more inaccurate when they are appliedto CTAs [see Edwards and Liew (1999); Edwards and Caglayan (2001);Liang (2003)] because their underlying distributional properties, and, mostof all, very low correlation with traditional risk factors do not support thesemeasures. Edwards and Caglayan (2001) use catastrophic loss measures toassess the hedging properties of these funds, but this type of measure isapplicable only to extremely risk-averse agents, which is not a frameworkthat corresponds to real portfolio management constraints. The positiveaspect of these measures is that they do not require prior knowledge of theunderlying return-generating process, which eliminates most of the difficulties associated with the discovery of a proper pricing model for CTAs.In this chapter we test a joint set of pricing models and performancemeasures that aim to better capture the distributional features of CTAs. Theidentification of risk premia and of the sensitivities of CTA returns tothese factors will clear the way toward the use of less utility-based performance measures than the Sharpe ratio and to a more proper use of stochastic discount factor–based performance measures, such as Jensen’salpha, the Treynor ratio, or the Treynor and Mazuy (1966) measure of market timing ability.The next section of this chapter examines the market trends and crisesover the sample period and presents the descriptive statistics of the CTAindex returns. An examination of the explanatory power of market factors
107
The Performance of CTAs in Changing Market Conditions
as well as trading strategy factors in describing CTA returns follows. Thenext section looks at different performance measures on the CTAs.
DATA AND SAMPLE PERIODThe data set that we use is the Barclay’s Trading group CTA data for theperiod from January 1990 to November 2003. The data set is composedof end-of-month returns for the CTA index as well as for five subindices1:the Barclay Currency Traders Index, the Barclay Financial and MetalTraders Index, the Barclay Systematic Traders Index, the Barclay Diversified Traders Index, and the Barclay Discretionary Traders Index.We divide the sample period into subperiods to investigate the behavior of the CTA indices under specific market conditions (see Table 6.1).
TABLE 6.1
Summary of SubperiodsPanel A: Bull and Bear Markets
Market Trend
Start
Finish
Ann. Return
# Obs
Weak BullModerate BullStrong BullBear
01:199001:199409:199803:2000
12:199309:199803