example, because CTAs may desire a higher beta in up-market environments but negative betas in down-market periods.In addition to applying the DEA methodology to evaluate CTA performance, we explore the relationship between the efficiency scores andfund size, investment style and strategy, length of the manager’s track
1Another important dimension of styles is the time frame. There are long-term,
short-term, and medium-term traders and those who combine time frames.
CTA Performance Evaluation with Data Envelopment Analysis
81
record, and measures of the covariance of CTA returns with equity marketreturns. We ask:Do emerging hedge fund managers2 really do better than larger, established managers?■ Is there a relationship between efficiency scores and equity markets,and if so, does the market environment impact the relationship?■ Do strategies (systematic, discretionary, trend-based) or styles (diversified, financial, currency, etc.) matter in different market environments?■
We analyze monthly CTA returns in two different market environments: over 24 months beginning in 1998, when equity market returns arepredominantly positive, and over 24 months beginning in 2000, when theyare more often negative. We find that emerging managers perform betterthan well-established managers in the sense that funds with shorter trackrecords have a greater efficiency score. Fund size and manager tenure areweakly positively correlated. In contrast with the conventional wisdom,however, larger funds have better efficiency scores. These results providesome insight into capacity issues concerning optimal fund size. The fundsize and manager tenure coefficients are, however, statistically significantonly during the first (1998–2000) time period, indicating that capacityissues may be less important during flat equity markets.For both time periods, squared equity beta is inversely related to theefficiency scores and the coefficient is highly significant. This result appearsto be influenced by the risk-minimizing design of our DEA model. The styledummy variable (diversified versus nondiversified) was not a significant factor impacting efficiency scores. The systematic strategy variable was significant, but only during the second (2000–2002) down-market period. Weconsider these results as preliminary because several issues may be affectingtheir significance. Notably, when our sample size is broken down by investment style and strategy, the number of CTAs representing each group is verysmall. Nevertheless, we believe that the approach is a promising avenue forfurther research.The next section of this chapter provides a background discussion on various risk measures and performance evaluation issues. The variables chosenas inputs to the DEA model and the regression model are then discussed inthe context of prior research, and the data are described. The variable descrip-
2We consider managers with short track records to be emerging CTAs. This category is distinctly different from managers who invest in emerging markets.
82
PERFORMANCE
tion is followed by an explanation of the DEA methodology and Tobit regressions used to explore determinants of the efficiency scores obtained from theDEA model. Results are presented and the final section concludes.
RISK MEASURES AND PERFORMANCE EVALUATIONA multitude of investment fund performance models and metrics exist in partbecause some measures are more appropriate for certain purposes than others.For example, the Sharpe ratio is arguably more appropriate when analyzingan entire portfolio, while the Treynor ratio is appropriate when evaluating asecurity or investment that is part of a larger portfolio.3 The multitude of performance measures and approaches also suggests that more than one measure of risk may be needed to accurately assess performance. Conversely, somemeasures can be redundant. For example, Daglioglu and Gupta (2003b) findthat returns of hedge fund portfolios constructed on the basis of some riskmeasures are often highly correlated, and sometimes perfectly correlated,with returns of portfolios constructed on the basis of others. Burghart, Duncan, and Liu (2003) illustrate that the theoretical distribution of drawdownscan be replicated with a high degree of accuracy given only a manager’s average return, standard deviation of returns, and length of track record.In this section we begin by briefly reviewing some of the traditionalportfolio performance measures and analysis techniques. We review singleparameter risk measures based on modern portfolio theory, we discussexpanded performance models that account for time-varying risk, discussconcerns over assuming mean-variance sufficiency, and consider multifactormodels of style and performance attribution. This short review exposes aplethora of performance measures. The question of appropriateness andredundancy is revisited in the section that describes the data used in thisstudy. The current section also discusses the seemingly paradoxical issue ofusing benchmarks to evaluate absolute return strategies4 and concludeswith a discussion of potential determinants of performance.
Alpha and BenchmarksTraditional asset managers seek to outperform a benchmark, and their performance is measured relative to that benchmark in terms of an alpha.3The Sharpe measure is appropriate when analyzing an entire portfolio, because thestandard deviation, or total risk, is in the denominator whereas beta is the denominator of the Treynor measure, and beta measures the systematic risk that will contribute to the risk of a well-diversified portfolio.4Absolute return strategies seek to make positive returns in all market conditions.In contrast, relative return strategies seek only to outperform a benchmark.
CTA Performance Evaluation with Data Envelopment Analysis
83
While CTAs follow absolute return strategies that seek to make positivereturns in all market conditions, benchmarks now exist for CTAs and otherhedge fund strategies. Before considering benchmarks for absolute returnstrategies, we first review the concepts in the context of traditional assetmanagement. Jensen’s (1968) alpha is generally a capital asset pricing model(CAPM)-based performance measure of an asset’s average return in excessof that predicted by the CAPM, given its systematic risk (beta)5 and themarket (benchmark) return. Alphas also may be measured relative to additional sources of risk in multi-index models.Whereas various single-index models are based on the CAPM andassume that security returns are a function of their co-movements6 with themarket portfolio, multi-index (or multifactor) models assume that returnsare also a function of additional influences.7 For example, Chen, Roll, andRoss (1986) develop a model where returns are a function of factors relatedto cash flows and discount rates such a gross national product and inflation. The purposes of multi-index models are varied and, in addition toperformance attribution, include forming expectations about returns andidentifying sources of returns.Sharpe (1992) decomposes stock portfolio returns into several “style”factors (more narrowly defined asset classes such as growth and incomestocks, value stocks, high-yield bonds) and shows that the portfolio’s mixaccounts for up to 98 percent of portfolio returns. Similarly, Brinson,Singer, and Beebower (1991) show that rather than selectivity or markettiming abilities, it is the portfolio mix (allocation to stocks, bonds, andcash) that determines over 90 percent of portfolio returns. However, Brownand Goetzmann (1995) identify a tendency for fund returns to be correlatedacross managers, suggesting performance is due to common strategies thatare not captured in style analysis.Schneeweis and Spurgin (1998) use various published indexes (Goldman Sachs Commodity Index, the Standard & Poor’s 500 stock index, the5Within the Markowitz (1952) framework, total risk is quantified by the standard
deviation of returns. Tobin (1958) extended the Markowitz efficient frontier byadding the risk-free asset, resulting in the capital market line (CML) and paving theway for the development of the capital asset pricing model, developed by Sharpe(1964), Lintner (1965), and Mossin (1966). The CAPM defines systematic risk,measured by beta (b), as the relevant portion of total risk since investors can diversify away the remaining portion.6Usually CAPM-based performance models describe covariance with the marketportfolio, however, as noted earlier, they can attempt to describe coskewness andcokurtosis as well.7Arbitrage pricing theory (APT) establishes the conditions under which a multiindex model can be an equilibrium description (Ross, 1976).
84
PERFORMANCE
Salomon Brothers government bond index, and U.S. dollar trade-weightedcurrency index, the MLM Index8) with absolute S&P 500 returns andintramonth S&P return volatility in a multifactor regression analysis todescribe the sources of return to hedge funds, managed futures, and mutualfunds. The index returns employed in the regression analysis are intendedto be risk factors that explain the source of natural returns. The explanatory variable, absolute equity returns, captures the source of return thatderives from the ability to go short or long. Returns from the use of optionsor intramonth timing strategies are proxies for the intramonth standarddeviation. The MLM Index, an active index designed to mimic trendfollowing strategies, is used to capture returns from market inefficiencies inthe form of temporary trends.Seigel (2003) provides a comprehensive review of benchmarking andinvestment management. Despite the fact that CTAs and many hedge fundmanagers follow absolute return strategies, various CTA benchmarks nowexist, as described by Seigel (2003).
Addressing Time-Varying RiskSingle-parameter risk measures are problematic if managers are changingfund betas over time, as they would if they were attempting to time the market. For example, when equity prices are rising, the manager might increasethe fund’s beta and vice versa. Although market risk can be measured if theportfolio weights are known, this information is generally not publiclyavailable and other techniques must be employed.9
8Mount Lucas Management IndexTM is based on a concept conceived in 1988 of anindex methodology that involves changing (commodity) market sides long and shortto measure economic return.9Treynor and Mazuy (1966) added a quadratic term to the basic linear regressionmodel to capture nonlinearities in beta resulting from market timing activities. Konand Jen (1978, 1979) use a switching regression technique. Merton (1981) and Henriksson and Merton (1981) develop nonparametric and parametric option-basedmethods to test for directional market timing ability. The nonparametric approachrequires knowledge of the managers’ forecasts. The more commonly employedparametric approach involves adding an extra term to the usual linear regressionmodel and is CAPM based. Ferson and Schadt (1996) note that fund betas maychange in response to changes in betas of the underlying assets as well as fromchanging portfolio weights. They modify the classic CAPM performance evaluationtechniques to account for time variation in risk premiums by using a conditionalCAPM framework. This method removes the perverse negative performance oftenfound in earlier tests and suggests that including information variables in performance analysis is important.
CTA Performance Evaluation with Data Envelopment Analysis
85
Mitev (1998) uses a maximum likelihood factor analysis technique toclassify CTAs according to unobservable factors. Similarly, Fung and Hsieh(1997b) also use a factor-analytic approach to classify hedge funds. In bothcases, the results identify general investment approaches or trading strategies (e.g., trend-following, spread strategies, or systems approaches) assources of returns to these alternative investment classes. Factor analysisand multifactor regression analysis differ in their approach to identifyingthe factors (benchmarks) that serve as proxies for risk. In multifactorregression analysis, the factors are specified in advance. Factor analysis willidentify funds that have common yet unobservable factors, although thefactors can be inferred from the qualitative descriptions of the funds. Whilethis may seem redundant, the clustering of funds is done independently ofthe qualitative descriptions in a formal data-driven process.The data envelopment analysis methodology used in this chapter, anddescribed in more detail in Wilkens and Zhu (2001, 2004), incorporatesmultiple criteria and “benchmarks” funds or other securities according tothese criteria. This is distinctly different from multifactor analysis. Herebenchmarks are not risk factors but rather are efficient securities as definedin n dimensions where each dimension represents risk and return criteria.Recently Gregoriou (2003) used the DEA method in the context of benchmarking hedge funds.
Skewness and Kurtosis:Questioning Mean-Variance SufficiencyThe standard CAPM framework assumes that investors are concerned withonly the mean and variance of returns. Ang and Chau (1979) argue thatskewness in returns distributions should be incorporated into the performance measurement process. Even if the returns of the risky assets within aportfolio are normally distributed, dynamic trading strategies may producenonnormal distributions in portfolio returns. Both Prakash and Bear (1986)and Stephens and Proffitt (1991) also develop higher-moment performancemeasurements.Fishburn (1977), Sortino and van der Meer (1991), Marmer and Ng(1993), Merriken (1994), Sortino and Price (1994), and others also havedeveloped measures that take into account downside risk (or semivariance)rather than the standard deviation of returns. Although some differencesexist among these measures, the Sortino ratio captures their essence.Whereas the Sharpe ratio is defined as excess return10 divided by standard
10Return minus the risk-free rate.
86
PERFORMANCE
deviation, the Sortino ratio is defined as return divided by downside deviation. Downside deviation (DD) measures the deviations below some minimal accepted return (MAR). Of course, when the MAR is the averagereturn and returns are normally distributed, the Sharpe and Sortino ratioswill measure the same thing. Martin and Spurgin (1998) illustrate that evenif individual asset or fund returns are skewed, the skewness tends to bediversified away at the portfolio level. However, they also illustrate thatmanagers may choose to follow strategies that produce skewed returns as aform of signaling their skill. Note that coskewness remains irrelevant if itcan be diversified away, but skewness may have some signaling value. Additionally, the popularity of the related value at risk (VaR) measure11 and thecommon practice of reporting drawdown12 information for various alternative investments suggest that skewness may be important, whether interms of investor utility or skill signaling.Beta-Squared Coefficient The classic paper by Fama and MacBeth (1973),and several other early papers (e.g., Carroll and Wei 1988; Shanken 1992)empirically test a two-pass regression methodology for stock returns.Assuming a nonlinear relationship between stock returns, the tests includebeta-squared in the second-pass regression. These tests find that the coefficient for beta-squared is negative and statistically significant, providing evidence of a nonlinearity in stock returns.Schneeweis and Georgiev (2002, p. 7) provide evidence that CTAs havenonlinear returns with respect to the equity market: “When S&P 500returns were ranked from low to high and divided into four thirty-threemonth sub-periods, managed futures offered the opportunity of obtainingpositive returns in months in which the S&P 500 provided negative returnsas well as in months in which the S&P 500 reported positive returns.”We include equity beta-squared in our Tobit regressions where thedependent variable is not the expected return of the CTA, but is ratherthe efficiency score obtained in the DEA models. Although the dependentvariable is not the same as in the earlier stock studies, we might hypothesize that CTA efficiency scores are also negatively related to beta-squared.
11See Chung (1999) for a concise review of VaR methodologies.12Drawdown information is generally reported as the maximum drawdown over aperiod and is defined as the return from a fund’s net asset value peak to trough. TheCalmar ratio is a similar measure that CTA investors are often interested in and isdefined as the average annual return over the past three years divided by theabsolute value of the maximum drawdown during that period.
CTA Performance Evaluation with Data Envelopment Analysis
87
We infer a direct correspondence between the efficiency score and expectedreturn. The CTA returns observed by Schneeweis and Georgiev (2002),therefore, imply a positive coefficient. Finally, we note that the efficiencyscores used in this study minimize variability. This leads to the hypothesisthat the beta-squared coefficient is negatively correlated with the efficiencyscore, unless the enhanced return from high (absolute) betas is an offsetting factor.Fund Size In his chapter “The Lure of the Small,” Jaeger (2003) describeshow small firms and small portfolios are desirable features of hedge funds.Small firms satisfy hedge fund managers’ entrepreneurial spirit, and smallportfolios are often necessary to enable hedge funds to implement theirstrategies, especially if they trade in markets that are sometimes illiquid.Gregoriou and Rouah (2002) find, however, that fund size does not matterto hedge fund performance. Being a subset class of hedge funds, CTAs areexamined in this chapter to see if fund size or length of manager trackrecord is related to the DEA efficiency scores.Determinants of Performance Based on the discussion above, we choose asbases for performance evaluation in a DEA model monthly returns, kurtosis, minimum return, skewness, standard deviation of returns, and percentage of negative monthly returns. We then investigate the potential of fundsize, length of track record, strategy, and style to impact performance scoresof funds created by the DEA model.
DATA DESCRIPTIONMonthly CTA return data for 216 CTAs over two periods surroundingMarch 2000 are obtained from the Center for International Securities andDerivatives Markets (CISDM) Alternative Investment Database.13 The firstperiod is an up-market period for the equity market (March 31, 1998, toFebruary 28, 2000) and the second period is a down market environment(April 30, 2000, to March 31, 2002). The daily high for the S&P 500occurred in March 2000, as illustrated in Figure 5.1. The mean monthlyreturn for the S&P 500 was 1.28 percent and −1.11 percent for the first andsecond periods, respectively.
13We selected funds from the database with the most complete information on
investment styles and strategies.
88
PERFORMANCE
1,550
Daily Closing Value
1,4501,3501,2501,1501,050950850
12-Nov-02
1-Jul-02
6-Sep-02
24-Apr-02
5-Dec-01
14-Feb-02
18-Jul-01
28-Sep-01
5-Mar-01
10-May-01
17-Oct-00
22-Dec-00
5-Jun-00
10-Aug-00
20-Jan-00
28-Mar-00
7-Sep-99
11-Nov-99
30-Jun-99
23-Apr-99
7-Dec-98
16-Feb-99
24-Jul-98
30-Sep-98
11-Mar-98
18-May-98
750
Date
FIGURE 5.1
S&P 500 Daily Closing Values, from 1998 to 2002
Performance criteria used in the DEA model were calculated from theCTA returns for each of the two periods. The DEA approach to “estimating” the efficient frontier is a nonstatistical approach. As a result, all deviations from the efficient frontier are measured as inefficiency (i.e., there isno allowance for statistical noise). The efficiency measures obtained fromthis method are, therefore, very sensitive to the effect of outliers. Hence, foreach performance criterion used in the DEA model, particular effort wasmade to detect any outliers. CTAs with outliers in one subperiod weredeleted from both subperiods so as to have the same group of CTAs. Ourfinal sample consisted of 157 CTAs that were used for analysis in the DEAmodel and the subsequent Tobit regression analysis. Table 5.1 providesdescriptive statistics for the DEA model criteria over both periods and forthe full and final sample.Other information we use from the CISDM Alternative InvestmentDatabase includes the assets under management over time, the datesthe funds were established, and information on the investment style14
14We follow the terminology established by Sharpe (1992) and call the market focus
investment style.
89
Mean of CTAsStandardDeviationMinMax
Mean of CTAsStandardDeviationMinMax
Mean of CTAsStandardDeviationMinMax
Mean of CTAsStandardDeviationMinMax
All 216 CTAsfor 1998–2000
All 216 CTAsfor 2000–2002
157 CTAsfor 1998–2000
157 CTAsfor 2000–2002
Original Data
0.0310.0130.191
0.058
0.0260.0220.155
0.056
0.0340.0040.245
0.055
0.0340.0050.193
0.055
StandardDeviation
0.0940.2080.750
0.481
0.0920.2920.750
0.453
0.1180.1250.750
0.456
0.1240.0420.750
0.427
%Negative
TABLE 5.1 Descriptive Statistics for the DEA Model Criteria
0.009−0.0320.027
0.005
0.026−0.0180.115
0.016
0.011−0.0320.073
0.006
0.013−0.0240.111
0.010
AverageMonthlyReturn
0.596−1.5701.471
0.247
0.705−1.5162.224
0.420
0.897−4.4421.981
0.183
0.929−2.1203.694
0.460
Skewness
1.160−1.3284.748
0.635
−0.1090.067−0.385−0.018
1.644−1.4986.249
1.040
−0.0930.049−0.247−0.012
2.365−1.32820.812
1.223
−0.1030.076−0.483−0.003
2.732−1.52416.370
1.637
−0.0910.063−0.530−0.006
Kurtosis
MinimumReturn
90
PERFORMANCE
TABLE 5.2 Number of CTAs, by Investment StyleInvestment Style
# ofCTAs
% ofCTAs
AgricultureCurrencyDiversificationFinancialStocksOverall
62093335157
41359213100
(agriculture, currencies, diversified, financial, and stocks) and strategy(discretionary, systematic, and trend-based15) of the fund. The diversifiedinvestment style is most common, accounting for 59 percent of the CTAsin our final sample, as illustrated in Table 5.2. Comprising 66 percent ofour final sample, the systematic investment strategy is the most common,as indicated in Table 5.3. Table 5.4 describes the distribution of thelength of the managers’ track record (maturity) in years, and Table 5.5 presents the distribution of the average funds under management for thetwo periods.Table 5.6 presents correlation coefficients for the DEA model criteria.We see that in both periods, minimum return and standard deviation arehighly (negatively) correlated, as one might expect. Kurtosis and skewnessare also highly (positively) correlated, but only in the first period. We notethat we are therefore potentially including redundant information in themodel. That is, by maximizing the minimum return, we may not necessarily need to minimize correlated measures such as the standard deviation.Following Daglioglu and Gupta (2003b), however, we sort the portfolios bythe various performance criteria and find that the returns to the sorted port-
15We follow Fung and Hsieh (1997a) and refer to the type of active management
followed as the strategy, and we use the classification scheme available in theCISDM database.
91
CTA Performance Evaluation with Data Envelopment Analysis
TABLE 5.3 Number of CTAs, by Investment StrategyStrategy
# ofCTAs
% ofCTAs
DiscretionarySystematicTrend BasedOverall
1210342157
86627100
folios are not as highly correlated as the variables themselves are. Table 5.7presents these results.After computing efficiency scores with the DEA methodology describedin the following section, determinants of the scores are explored by regressing them against four additional variables: beta, beta-squared, averagefunds managed, and length of manager track record. Table 5.8 presents thesummary statistics for these variables.
TABLE 5.4 Length of Managers’ Track Record (Maturity) in YearsLength of ManagerTrack Record
# ofCTAs
% ofCTAs
<66 – <77 – <88 – <99 – <1010 – <1111 – <1212 – <1313 – <1414 – <1515 – <1616+Overall
8191528109131752922157
51210186681131614100
92
PERFORMANCE
TABLE 5.5 Distribution of the Average Funds under Management1998–2000AverageFundManaged(000,000)
# ofCTAs
<2.52.5 – <55 – <1010 – <2020 – <3030 – <4040 – <5050 – <100100 – <150150 – <200200 – <400400+Overall
1914132581462772148157
2000–2002
% ofCTAs
AverageFundManaged(millions)
# ofCTAs
% ofCTAs
129816594174195100
<2.52.5 – <55 – <1010 – <2020 – <3030 – <4040 – <5050 – <100100 – <150150 – <200200 – <400400+Overall
23171715171191485156157
151111101176953104100
TABLE 5.6 Correlation Coefficients for the DEA Model Criteria1998–2000
Std. Dev. Per Neg
Return Skewness Min Return Kurtosis
Standard DeviationPercent NegativeReturnSkewnessMinimum ReturnKurtosis
1.0000.3201.0000.354 −0.4780.2430.422−0.838 −0.2450.210 0.181
1.0000.002−0.133−0.065
2000–2002
Std. Dev. PerNeg
Return Skewness Min Return Kurtosis
Standard DeviationPercent NegativeReturnSkewnessMinimum ReturnKurtosis
1.0000.217 1.0000.271 −0.4400.124 0.308−0.846 −0.1670.057 −0.133
1.0000.2350.037−0.161
1.0000.1240.648
1.0000.287−0.417
1.000−0.088
1.000−0.326
1.000
1.000
93
TopBottomTop BottomStd. Dev. Std.Dev. PerNeg PerNeg
Top Std. Dev. 1BottomStd. Dev.0.125 1TopPerNeg−0.102 0.041 1BottomPerNeg−0.094 0.054 −0.085 1Top COR−0.109 0.138 0.004 0.111BottomCOR−0.027 0.082 0.015 −0.067TopReturn0.147 −0.021 0.039 −0.042BottomReturn0.041 −0.066 0.048 −0.122Top Skew0.185 0.008 −0.065 0.138BottomSkew−0.067 −0.087 −0.008 0.111TopMinRet0.023−0.126 −0.206BottomMinRet−0.061 −0.025 0.038 0.035Top Kurt−0.006 0.076 0.083 0.013BottomKurt−0.006 0.276 0.059 0.040
1998–2000
1
0.050 −0.029 0.025
0.050
1
−0.001 0.044 −0.010
0.1640.086
110.0570.025 0.066 −0.111 −0.178 −0.023
0.016 0.122 0.064 −0.097 −0.099 0.099−0.154 0.168 −0.006 −0.050 0.042 −0.018
0.089 −0.031 −0.002 −0.040 −0.089
0.034 0.098
−0.073 0.080 0.100 10.047 0.045 −0.076 −0.041 1
−0.110 −0.003
−0.075 1
1
0.218 1
1
Top Bottom Top Bottom Top Bottom Top Bottom Top BottomCOR COR Return Return Skew Skew MinRet MinRet Kurt Kurt
TABLE 5.7 Top and Bottom Correlation Matrix for the DEA Model Criteria, by Portfolio
94
TopBottomTop BottomStd. Dev. Std. Dev. PerNeg PerNeg
10.049
10.057 −0.035 −0.182 1
0.0030.186
1
Top Bottom Top Bottom