the efficiency measure for fund k is:
∑r =1 ur yrkm∑i =1 vi xiks
hk = Max
(7.1)
where the weights ur and vi are positive. An additional set of constraintsrequires that the same weights, when applied to all CTAs, not allow anyCTA with an efficiency score greater than 100 percent and is displayed inthis set of constraints:
∑r =1 ur yrj ≤ 1 for j = 1, . . . , n.m∑i =1 vi xijs
3Linear programming is the optimization of a multivariable objective function, sub-
ject to constraints.4The BCC model permits a greater number of potential optimal solutions. With theBCC model, the number of funds with an efficiency score of 100 will, on average,be higher than with the CCR model. Choosing between these models requiresinsight into what the process will involve. For example, if the increase in inputs doesnot provide the same increase in outputs, then the variable returns to scale modelshould be used.
134
PERFORMANCE
The efficiency score falls between 0 and 100, with CTA k regarded asefficient on receiving an efficiency score of 100. Therefore, each CTA selectsweights to maximize its own efficiency.s
hk = Max ∑ ur yrk + ckr =1
subject to the constraints:m
s
i =1
r =1
∑ vi xij − ∑ ur yrj − ck ≥ 0 for j = 1, . . . n,
(7.2)
m
∑ vi xik = 1,i =1
ur ≥ 0 for r = 1, . . . , s,vi ≥ 0 for i = 1, . . . , m.An extra constant variable, denoted by ck, is added in the BCC modelto allow variable returns to scale between inputs and outputs. For a CTA tobe BCC technically efficient; its only requirement is to be efficient; for aCTA to be efficient in the CCR model, it must be both scale and technicallyefficient (Bowlin 1998).A CTA is considered scale efficient if the level of its operation is optimal. If the scale efficiency is reduced or increased, the efficiency willweaken. A scale-efficient CTA will function at most favorable returns toscale. In essence, the distance on a production frontier between the constantreturns to scale and the variable returns to scale frontier establishes thecomponent labeled scale efficiency. A CTA is considered technically efficientif it is able to maximize each of its outputs per unit of input, thus signifying the efficiency of the conversion process of the variables. In this chaptertechnical efficiency is calculated using the BCC model.In a production frontier, constant returns to scale implies that anyincrease in the inputs of a CTA will result in a proportional increase in itsoutputs. In other words, a linear relationship would be present betweeninputs and outputs. If a CTA were to increase its inputs by 5 percent,thereby producing a similar increase in outputs, the CTA would be operating at constant returns to scale. Consequently, irrespective of what scale theCTA operates at, its efficiency will stay the same.If an increase in the inputs of a CTA does not induce a proportionaltransformation in its outputs, however, then the CTA will display variable
Simple and Cross-Efficiency of CTAs Using Data Envelopment Analysis
135
returns to scale, which implies that as the CTA alters its level of day-to-dayoperations, its efficiency can increase or decrease. Therefore, since CTAsvary their leverage at different times to magnify returns, we employ theBCC model (varying returns to scale).
Cross-EfficiencyThe cross-evaluation, or cross-efficiency, model was first seen in Sexton,Silkman, and Hogan (1986) and later in Oral, Ketani, and Lang (1991),Doyle and Green (1994), and Thanassoulis, Boussofiane, and Dyson (1995).It establishes the ranking procedure and computes the efficiency score ofeach CTA n times using optimal weights measured by the linear programs.A cross-evaluation matrix is a square matrix of dimension equal to thenumber of CTAs in the analysis. The efficiency of CTA j is computed withthe optimal weights for CTA k. The higher the values in column k, the morelikely that CTA k is efficient using superior operating techniques. Therefore,calculating the mean of each column will provide the peer appraisal scoreof each CTA. The cross-efficiency method is superior to the simple efficiency method because the former uses internally generated weights asopposed to forcing predetermined weights.The cross-evaluation model used here is represented by equation 7.3:
∑r =1 urkyrj ,hkj =m∑i =1 vikxijs
k = 1, . . . , n, j = 1, . . . , n,
(7.3)
where hkj = score of CTA j cross-evaluated by the weight of CTA k.In the cross-evaluation matrix, all CTAs are bounded by 0 ≤ hkj ≤ 1, andthe CTAs in the diagonal, hkk, represent the simple DEA efficiency score, sothat hkk = 1 for efficient CTAs and hkk < 1 for inefficient CTAs. Equation7.3 shows that the problem of trying to distinguish the relative efficiencyscores of all CTAs is generated n times.The DEA method renders an ex-post evaluation of a CTA’s efficiencyand specifies the precise input-output relation. The relation must be realizedwithout a level of efficiency greater than 100 when the coefficients areadapted to the CTAs in our sample. Efficiency scores, as they are relative tothe other CTAs in the sample, are by no means absolute.Papers on DEA have been published in many sectors, and the use ofsuch analysis often has resulted in technical and efficiency improvements.DEA also has been used recently to evaluate the performance of mutual
136
PERFORMANCE
funds and determine the most efficient funds (see, e.g., McMullen andStrong 1997; Bowlin 1998; Morey and Morey 1999; Sedzro and Sardano2000; Basso and Funari 2001). Barr, Seiford, and Siems (1994), however,suggest that using a single input/output ratio to assess management qualityis impractical; instead they propose a multidimensional approach.However, the CCR model is one of the first DEA models based on efficiency. It allows a set of optimal weights to be calculated for each input andoutput to maximize a CTA’s efficiency score. If these weights were appliedto any other fund in our database, the efficiency score would not exceed100. The CCR score aggregates technical and scale efficiency. Despite themany modified DEA models in existence, the CCR model is the mostbroadly known and used. Basically, the BCC and CCR models offer twoways of considering the same problem.As we noted earlier, cross-evaluation DEA is superior to either simpleDEA method because efficiency is still measured relative to the CTA withthe highest efficiency score, but having more than one combination ofweights of a fund that maximizes its own efficiency adds an extra dimension of flexibility. The main idea of DEA is that it is flexible and can branchout to other CTAs to evaluate their individual performance. CTAs with highaverage efficiency from a cross-efficiency matrix can be considered as goodexamples for inefficient CTAs to work toward and improve their methods.We adopt and expand the methodology of Sedzro and Sardano (2000),who investigated mutual funds, and apply it to CTA classifications. SinceCTAs exhibit nonnormal distribution of returns and display fat tails, we usevariables different from those used for mutual funds (Fung and Hsieh1997a). In previous studies skewness was shown to have an influence onmonthly average returns in stock markets (see Sengupta 1989).The inputs and outputs must correspond to the activities of CTAs forthe analysis to make sense. We use six variables in a risk-return framework,three for inputs and three for outputs, because a larger number might clutter the analysis. Three times the number of inputs and outputs will result inhaving sufficient observations (degrees of freedom) to get a good evaluation. Having a greater number of variables could result in an overlap ofmeasuring inputs and outputs, thereby producing some problems in interpreting the results. If too many variables are used, the analysis could resultin many CTAs being rated efficient.Modern portfolio theory measures the total risk of a portfolio by usingthe variance of the returns. But this method does not separate upside risk,which investors seek, from the downside returns they want to avoid. Variance is not usually a good method for measuring risk, but semivariance isgenerally accepted and frequently used because it measures downside risk.Returns above the mean can hardly be regarded as risky, but variance belowthe mean provides more information during extreme market events. This is
Simple and Cross-Efficiency of CTAs Using Data Envelopment Analysis
137
important for investors who worry more about underperformance thanoverperformance (Markowitz 1991).5 Because CTAs can obtain positivereturns in flat or down markets, they induce negative skewness in portfolioreturn. Adding CTAs to a traditional stock and bond portfolio to obtainhigher risk-adjusted returns and lower volatility will therefore result in atrade-off between negative skewness and diversification of the portfolio(Diz 1999).The mean and standard deviations of CTA returns can be misleading;examining higher moments such as skewness is recommended (Fung andHsieh 1997a). The introduction of skewness in inputs and outputs mightpresent some signaling assessment of each CTA classification because skewness does not penalize CTA by the upside potential returns. Although CTAsattempt to maximize returns and minimize risk, this comes at a trade-off;adding CTAs to traditional investment portfolios will likely result in highkurtosis and increased negative skewness, which are the drawbacks of thisalternative asset class.
DATAWe use CTA data from the Barclay Trading Group/Burlington Hall AssetManagement and examine five CTA classifications during the periods from1998 to 2002 and 2000 to 2002. The subtype CTA classifications includeDiversified, Financials, Currency, Stocks, and Arbitrage. We choose thesetime periods because we wish to determine whether the extreme marketevent of August 1998 had any impact on each of the classifications. Thedatabase provider warned us that using a longer time frame, for example,a 7- or 10-year examination period, would have resulted in significantlyfewer CTAs. Our data set consists of monthly net returns, for which bothmanagement and performance fees are subtracted by the CTAs and forwarded to Barclay. We do not examine defunct CTAs.The data were aggregated into separate DEA runs for the three-yearand five-year periods for each classification. Both examination periods contain the same CTAs in each classification, which enables us to compare CTArankings and efficiency scores across periods. The inputs are (1) lower meanmonthly semiskewness, (2) lower mean monthly semivariance, and (3)mean monthly lower return. The outputs are (1) upper mean monthly semiskewness, (2) upper mean monthly semivariance, and (3) mean monthlyupper return. The value of outputs is the value added of each CTA.
5Extreme market events include the Asian currency crisis of 1997 and the Russianruble crisis of 1998.
138
PERFORMANCE
EMPIRICAL RESULTSAn efficiency score of 100 signifies that a CTA is efficient and that no otherCTA has produced better outputs with the inputs used. It does not implythat all CTAs with a score of 100 provide the same return during the examination period, merely that the return is at the maximum of the incurredrisk. The efficiency score is not absolute. A CTA with an efficiency score of100 returning 20 percent is considered more risky than a CTA with a scoreof 100 returning 15 percent. Note that the results obtained from DEA donot guarantee future efficiency; nonetheless, DEA is a very valuable selection and screening tool for institutional investors. Every CTA with an efficiency score of 100 can be considered to be as one of the best.Simple efficiency is perhaps not quite enough to assess the performanceappraisal of CTAs, though, because cross-efficiency goes beyond selfappraisal to peer appraisal (Vassiloglou and Giokas 1990; Sedzro and Sardano 2000). CTAs with an efficiency score of 100 in the simple model dropin value when the average cross-efficiency measure is used. However, thecross-efficiency scores signify the peer appraisal of each CTA, thus revealing a CTA’s all-around performance in all areas.Table 7.1 displays the number of efficient and nonefficient CTAs for bothexamination periods. The results indicate that a greater majority of CTAs arenonefficient according to the inputs and outputs we use. The reason possiblycan be attributed to the various extreme market events, such as the Russianruble crisis of August 1998, which led to increased volatility in commoditiesmarkets. To assess the performance of CTAs properly, the time series of eachCTA classification must be long enough to include at least one extreme negative market event, as is the case during the 1998 to 2002 period. Althoughwe find a low number of efficient CTAs in each classification, we are comforted by an earlier study that found only 8.9 percent of mutual funds investigated to have efficiency scores of 100 (Sedzro and Sardano 2000).Tables 7.2 through 7.6 present basic statistics and simple and crossefficiency scores for the five CTA classifications. A high score means theCTA performs well relative to its peers, based on the inputs and outputsused.6 Some CTAs are rated as efficient by the simple BCC model, but
6The Babe Ruth analogy is a classic example. Babe Ruth was a great home run hitter.
In terms of simple efficiency (basic DEA model), he would have achieved a score of 100.However, if he were to be compared to other players on the team, he may not have beenan all-around player, thus making his cross efficiency score low compared to a goodall-around player.
139
15.17%9.21%10.62%0.95%14.54%—
12.17%5.73%8.53%8.41%14.49%—
Classification
StocksCurrencyFinancialsDiversifiedArbitrageTotal
StocksCurrencyFinancialsDiversifiedArbitrageTotal
7.08%0.60%2.52%2.83%8.68%—
6.30%4.15%3.90%5.71%1.01%—
CrosssectionMedian
CrosssectionMin
−35.22%−48.33%−37.07%−75.30%−7.45%—
17.50%13.01%16.32%20.08%8.47%—
−35.22%−48.33%−37.07%−68.35%−5.63%—
1999–2001
18.99%14.56%16.57%7.60%12.91%—
1997–2001
CrosssectionSTD
55.49%29.41%31.41%36.61%7.14%—
55.49%48.45%33.60%80.29%16.89%—
CrosssectionMax
53107328
4385323
83635430122
93736450127
133945503150
134044503150
Efficient Nonefficient Total
Number of Efficient and Nonefficient and Summary Statistics for CTAs, 1997–2001 and 1999–2001
CrosssectionMean
TABLE 7.1
140
Allied Irish Capital Mgmt. Ltd.Analytic Investment Mgmt.Arcanu Investment Mgmt.Michael N. Trading Co. LtdTrading Solutions
Stocks
CompoundedReturn3.068.8710.2636.2526.40
AverageAnnualizedReturn
9.3428.2932.24181.0199.83
4.7814.0046.8215.13
26.2894.99843.8625.47
CompoundedReturn
AverageAnnualizedReturn0.110.892.810.23
AnnualizedSharpeRatio
4.0710.5812.8616.8625.90
AnnualizedStandardDeviation
3-Year SimpleEfficiencyScore BCCModel100100100100100
−1.000.440.471.900.86
100100100100
5-Year SimpleEfficiencyScore BCCModel
AnnualizedSharpeRatio
1999–2001
4.6810.9115.1347.87
AnnualizedStandardDeviation
1997–2001
Basic Statistics and Simple and Cross-Efficiency Scores for Stocks, 1997–2001 and 1999–2001
Allied Irish Capital Mgmt. Ltd.Analytic Investment Mgmt.Michael N. Trading Co. Ltd.Minogue Investment Co.
Stocks
TABLE 7.2
14.4120.6625.7743.6417.03
3-Year CrossEfficiencyScore BCCModel
54.9054.5577.1012.57
5-Year CrossEfficiencyScore BCCModel
141
DKR Capital Inc.KMJ Capital Mgmt. Inc.OSV Partners Inc.
Currency
CompoundedReturn9.3319.027.95
AverageAnnualizedReturn
31.5772.5226.75
21.5416.8715.16
177.86119.93110.37
CompoundedReturn
AverageAnnualizedReturn1.230.851.70
AnnualizedSharpeRatio
5.6712.312.10
AnnualizedStandardDeviation
0.901.201.76
AnnualizedSharpeRatio
1999–2001
15.0814.776.41
AnnualizedStandardDeviation
1997–2001
100100100
3-Year SimpleEfficiencyScore BCCModel
100100100
5-Year SimpleEfficiencyScore BCCModel
Basic Statistics and Simple and Cross-Efficiency Scores for Currency, 1997–2001 and 1999–2001
Hathersage Capital Mgmt. LLCKMJ Capital Mgmt. Inc.OSV Partners Inc.
Currency
TABLE 7.3
17.6219.8817.95
3-Year CrossEfficiencyScore BCCModel
9.8311.968.41
5-Year CrossEfficiencyScore BCCModel
1427.2632.948.7317.350.7014.3016.778.70
40.90312.1153.07120.0749.9982.32121.1653.32
CompoundedReturn
AverageAnnualizedReturn
15.2812.884.96
8.8330.346.2717.932.30
AnnualizedStandardDeviation
0.650.970.90
0.340.720.720.730.53
AnnualizedSharpeRatio
1997–2001
100100100
100100100100100
5-Year SimpleEfficiencyScore BCCModel
Basic Statistics and Simple and Cross-Efficiency Scores for Financials, 1997–2001 and 1999–2001
Appelton Capital Mgmt.Carat Capital LLCCity Fund Mgmt. Ltd.Eckhardt Trading CompanyIIU Asset StrategiesMarathon CapitalGrowth Partners LLCMoore Capital Mgmt. Inc.Vega Asset Mgmt. (USA) LLC
Financials
TABLE 7.4
48.5953.1152.39
50.0956.6648.4755.9363.96
5-Year CrossEfficiencyScore BCCModel
143
(continued)
Carat Capital LLCCity Fund Mgmt. Ltd.Eckhardt Trading CompanyIIU Asset StrategiesInternational Trading AdvisorsBVBAInvesco Inc.IXORCAPMarathon CapitalGrowth Partners LLCParthian Securities, S.A.Vega Asset Mgmt. (USA) LLC
Financials
TABLE 7.4
6.848.969.577.00−3.339.561.7813.8746.828.31
21.7130.1529.6022.17−11.0428.654.9745.73158.6128.07
CompoundedReturn
AverageAnnualizedReturn
16.1049.652.59
10.6615.365.74
7.425.5013.927.99
AnnualizedStandardDeviation
0.600.871.56
−1.000.35−1.00
0.350.860.380.34
AnnualizedSharpeRatio
100100100
100100100
100100100100
3-Year SimpleEfficiencyScore BCCModel
49.5753.4126.25
36.0151.3154.17
37.4123.1967.5152.11
3-Year CrossEfficiencyScore BCCModel
144
AIS Futures Mgmt. LLCBeach Capital Mgmt. Ltd.Brandywine Asset Mgmt. IncFort Orange Capital Mgmt. Inc.Friedberg Commodity Mgmt. Inc.Marathon CapitalGrowth Partners LLCMississippi River Investments Inc.0.6717.00
22.82
−0.6458.88
5.43
−1.87157.2353.93−21.58
14.1717.64−17.40−7.64−12.01
1.8619.9012.082.74
CompoundedReturn
44.1064.42−43.02−28.19−44.04
AverageAnnualizedReturn
0.07
−1.001.170.29−1.00
AnnualizedSharpeRatio
16.2517.65
20.1014.1815.6726.6337.61
−1.000.72
0.490.94−1.00−1.00−1.00
1999–2001
16.42
21.4413.3426.8440.76
AnnualizedStandardDeviation
1997–2001
100100
100100100100100
100
100100100100
5-Year SimpleEfficiencyScore BCCModel
Basic Statistics and Simple and Cross-Efficiency Scores for Diversified, 1997–2001 and 1999–2001
AIS Futures Mgmt. LLCBeach Capital Mgmt. Ltd.Fort Orange Capital Mgmt. Inc.Friedberg Commodity Mgmt. Inc.Marathon CapitalGrowth Partners LLC
Diversified
TABLE 7.5
45.1547.75
54.6166.9149.5859.0834.91
34.5
53.1464.5144.9737.57
5-Year CrossEfficiencyScore BCCModel
145
BAREP Asset Mgmt.DKR Capital Inc.TWR Mgmt. Corp.
Arbitrage
CompoundedReturn3.6717.4811.50
AverageAnnualizedReturn
9.8167.2638.86
8.5914.2114.54
49.4774.4298.12
CompoundedReturn
AverageAnnualizedReturn0.421.460.79
AnnualizedSharpeRatio
10.576.7110.23
AnnualizedStandardDeviation
3-Year SimpleEfficiencyScore BCCModel100100100
−1.001.970.71
100100100
5-Year SimpleEfficiencyScore BCCModel
AnnualizedSharpeRatio
1999–2001
10.346.8212.91
AnnualizedStandardDeviation
1997–2001
Basic Statistics and Simple and Cross-Efficiency Scores for Arbitrage 1997–2001 and 1999–2001
BAREP Asset Mgmt.DKR Capital Inc.TWR Mgmt. Corp.
Arbitrage
TABLE 7.6
—79.3441.96
3-Year CrossEfficiencyScore BCCModel
67.0565.8788.03
5-Year CrossEfficiencyScore BCCModel
146TABLE 7.7
PERFORMANCE
Champion CTAs 1997–2001 and 1999–20011997–2001
CTAMichael N. Trading Co. Ltd.KMJ Capital Mgmt. Inc.Marathon CapitalGrowth Partners LLCAIS Futures Mgmt. LLCN/A
Reference Set
Classification
1229
StocksCurrency
2830N/A*
FinancialsDiversifiedArbitrage1999–2001
CTATrading SolutionsKMJ Capital Mgmt. Inc.Marathon CapitalGrowth Partners LLCAIS Futures Mgmt. LLCN/A
Reference Set
Classification
618
StocksCurrency
2431N/A*
FinancialsDiversifiedArbitrage
*N/A = the sample set is too small.
when using the cross efficiency model their scores are among the lowest.The tables suggest that CTAs are more efficient in the shorter three-yearperiod. This is due to the absence of extreme market events during thistime frame. In Table 7.7 we identify the “champion” CTAs in each classification (except Arbitrage) by the number of times an efficient fund hasbeen part of an inefficient CTA’s reference set, derived by simple DEA. Asthe frequency of a CTA appearing in a reference set increases, the likelihood of the fund being a good performer increases. The efficient CTAappearing in the most reference sets can be considered the overall “champion.” Inefficient funds can learn from that CTA’s superior managementand investment practices.
CONCLUSIONData envelopment analysis is a novel method that pension funds, endowment funds, institutional investors, and high-net-worth individuals can use
Simple and Cross-Efficiency of CTAs Using Data Envelopment Analysis
147
to select efficient CTAs. We believe DEA is an excellent complement toother risk-adjusted measures because it can present a more complete pictureof hedge fund performance appraisal. DEA can provide reliable results evenwhen using nonnormal returns.As DEA becomes accepted and used by more academics, money managers, and institutional investors, CTAs with high cross-efficiency scores willbecome desirable. Future research could examine the efficiency of CTAindices from other database vendors, for example.
PART
TwoRisk and ManagedFutures Investing
Chapter 8 uses a unique data set from the Commodity Futures TradingCommission to investigate the impact of trading by large hedge funds andcommodity trading advisors (CTAs) in 13 futures markets. Regressionresults show there is a small but positive relationship between the tradingvolume of large hedge funds and CTAs and market volatility. Further resultssuggest that trading by large hedge funds and CTAs is likely based on private fundamental information.Chapter 9 examines the dynamic nature of commodity trading programsthat tend to mimic a long put option strategy. Using a two-step regressionprocedure, the authors document the asymmetric return stream associatedwith CTAs and then provide a method for calculating value at risk. Theauthors also examine a passive trend-following commodity index and findit to have a similar put optionlike return distribution. The authors also demonstrate how commodity trading programs can be combined with otherhedge fund strategies to produce a return stream that has significantly lowervalue at risk parameters.Chapter 10 examines the relationships between various risk measuresfor CTAs. The relationships are extremely important in asset allocation. Iftwo measures (e.g., beta and Sharpe ratio) produce identical rankings for asample of funds, then the informational content of the two measures aresimilar. However, if the two measures produce rankings that are not identical, then the informational content of each measure as well as asset alloca-
149
150
RISK AND MANAGED FUTURES INVESTING
tion decisions may be unique. Interdependence of risk measures has beenexamined previously for equities and recently for hedge funds. In this chapter the authors analyze 24 risk measures for a sample of 200 CTAs over theperiod January 1998 to July 2003.Chapter 11 provides a simple method for measuring the downside protection offered by managed futures. Managed futures are generally considered to help reduce the downside exposure of stocks and bonds. Thechapter also measures the downside protection provided by hedge fundsand passive commodity futures indices. In each case, considerable downsideprotection is offered by each of these three alternative asset classes.
8
CHAPTER
8
The Effect of Large Hedge Fundand CTA Trading on FuturesMarket VolatilityScott H. Irwin and Bryce R. Holt
his study uses a unique data set from the CFTC to investigate the impactof trading by large hedge funds and CTAs in 13 futures markets. Regression results show there is a small but positive relationship between the trading volume of large hedge funds and CTAs and market volatility. However,a positive relationship between hedge fund and CTA trading volume andmarket volatility is consistent with either a private information or noisetrader hypothesis. Three additional tests are conducted to distinguish betweenthe private information hypothesis and the noise trader hypothesis. The firsttest consists of identifying the noise component exhibited in return variancesover different holding periods. The variance ratio tests provide little supportfor the noise trader hypothesis. The second test exam