69
CTA Performance, Survivorship Bias, and Dissolution Frequencies
TABLE 4.11 CTA Persistence in Performance, January 1986 to December 2002
D1D2D3D4D5D6D7D8D9D10D1aD1bD1cD10aD10bD10c
Mean
Std. Dev.
Alpha
CTA Index
R2adj
1.241.021.071.101.051.351.211.671.872.671.301.331.963.171.901.31
4.713.343.003.223.223.794.084.565.666.185.495.095.9919.9914.777.72
−0.33−0.20**−0.09−0.19**−0.25***−0.18**−0.47***−0.19*−0.40***0.201.82−0.090.16−0.39−0.270.07
0.97***0.76***0.71***0.80***0.80***0.94***1.04***1.15***1.40***1.52***0.82***1.23***0.71***1.04***0.99***1.17***
0.570.700.770.840.850.840.890.870.840.820.020.090.110.490.510.52
This table reports the performance analysis of the performance decile regressedagainst the CTA Global Index.t-stat are heteroskedasticity consistent.***Significant at the 1 percent level.**Significant at the 5 percent level.*Significant at the 10 percent level.Numbers in the table are monthly percentages.
Subperiod AnalysisTable 4.12 contains the persistence analysis over various subperiods. Wereport a bull market period (January 1998 to March 2000), a bear marketperiod (April 2000 to December 2002), and the 10-year period endingDecember 2002. This analysis aims at determining if the previous resultsremain stable over different market environments. The left-hand side of thetable indicates that worst-performing funds significantly underperformtheir peers over the bull market period. D1 to D4 and D6 have significantlynegative intercept over the January 1998 to March 2003 period. D9 andD10 have positive alphas. Moreover, the alpha of D10 is significantlypositive. These interesting results indicate that the previous year’s bestperforming funds (around 10 percent of the whole database) significantlyoutperform their peers over the bull market period. The results of subdecile
70
−1.05**−0.40**−0.21**−0.31**0.00−0.26***−0.15−0.21**0.020.79**−0.022.17−0.780.41−0.61**0.29
0.91**0.201.01*** 0.780.83*** 0.840.98*** 0.860.91*** 0.891.05*** 0.940.80*** 0.871.18*** 0.871.19*** 0.841.34*** 0.642.13*−0.020.02−0.040.94**0.030.64*** 0.150.64*** 0.350.43*** 0.01
R2adjD1D2D3D4D5D6D7D8D9D10D1aD1bD1cD10aD10bD10c
Dec 2002
Apr 2000–
−0.45−0.08−0.19*−0.01−0.17**0.08−0.15−0.12−0.09−0.06−0.28−1.731.15−0.13−0.070.501.22***0.83***0.83***0.90***0.87***1.03***1.07***1.11***1.22***1.07***2.27**0.88**0.321.04***0.54***0.74***
Index0.550.610.870.910.950.910.920.910.850.750.180.00-0.030.680.440.46
R2adj
BearMarket PeriodAlphaD1D2D3D4D5D6D7D8D9D10D1aD1bD1cD10aD10bD10c
Dec 2002
Jan 1993–
−0.47**−0.28***−0.13*−0.05−0.12***−0.10*−0.26***−0.15***−0.25***0.38**−0.210.040.74**2.570.670.641.12***0.97***0.85***0.81***0.79***0.90***0.98***1.13***1.29***1.21***0.99***0.44***0.70***1.001.21**0.28
Index
Ten-Year PeriodAlpha
0.470.680.820.850.890.880.890.890.860.710.440.200.180.000.020.00
R2adj
This table reports the persistence in performance analysis of the performance decile regressed against the CTA Global Index overthe January 1993–December 2002 period.t-stat are heteroskedasticity consistent.***Significant at the 1 percent level.**Significant at the 5 percent level.*Significant at the 10 percent level.Numbers in the table are monthly percentages.
D1D2D3D4D5D6D7D8D9D10D1aD1bD1cD10bD10cD10c
Index
Bull Market Period
Mar 2000 Alpha
Jan 1998–
TABLE 4.12 Persistence in Performance Subperiod Analysis
CTA Performance, Survivorship Bias, and Dissolution Frequencies
71
analyses are less significant.9 The table also indicates that each decile is significantly exposed to the CTA Global Index. The R2 is particularly high,especially for the upper deciles, but is generally low for the subdeciles.The central part of Table 4.12 reports the decile analysis over the April2000 to December 2002 period. This period corresponds to a bear marketsince the technology bubble exploded in March 2000. It indicates that allthe deciles but D6 have negative alphas. The only one significantly negativeis D5. This result indicates that no group of funds offers persistent returnsduring the bear market that began in the first half of 2000. As expected, thetop-performing subdecile (D10c) yields a positive (but not significant)alpha. Nevertheless, each decile is significantly positively exposed to theCTA Global Index.The right-hand part of Table 4.12 reports the analysis for the 10-yearperiod ending December 2002. In this last case, all deciles but D10 are negative, and most of them significantly destroy value (D1, D2, D5, D7, D8,and D9 have all significantly negative alphas). As in the bull period analyzed before, D10 has a significantly positive alpha. This indicates that thefunds in this particular decile persistently create value compared to theirpeers. The exposure to the market is significantly positive for all deciles,and as in all the other cases, R2 is high for each decile.
Strategies AnalysisOnce these results are obtained, we regress the same data over the variousstrategies returns10 to determine if some strategies are statistically more represented in some deciles. Results are reported in Table 4.13.The first column contains the alphas. These increase monastically acrossD1 to D8. Alphas for D9 and D10 are negative. Few of them are significant.D2, D3, and D4 are weakly negatively significant, and D8 is significantlypositive at the 10 percent significance level. Interestingly, subdeciles D10a
9Subdeciles sometimes contain few funds when many of the funds were dissolved in
the year after their classification in the top- or worst-performing decile. As we notedin analyzing the survivorship bias, this bias is important in CTA data and we haveto take this factor into account.10We have analyzed the relationship between the various strategies in the correlation analysis. We do not find high correlation between the strategies. The only correlations that could lead to problems in estimation are the high coefficients betweenthe CTA Global Index and some strategies. To deal with this issue, we ran two estimations, with and without the CTA Global Index. The results obtained are qualitatively the same. We report only the results without the index.
720.030.080.07−0.06*−0.030.070.03−0.020.010.29−0.030.050.070.75−0.230.09
−0.58 −0.100.85** −0.020.08***−0.490.00 −0.090.050.00−0.27** 0.030.030.030.000.22* −0.030.10*** 0.000.000.17 −0.06** 0.11** −0.010.03***0.05 −0.040.040.00 −0.030.000.030.10* −0.01 −0.010.21*0.020.14** 0.00 −0.01−0.140.050.17*0.010.01−0.27 −0.020.31** 0.03 −0.01−0.74*0.050.210.11** 0.00−0.03 −0.02 −0.210.010.01−1.01** −0.05 −0.36** −0.05 −0.081.66 −1.641.410.05 −0.34−2.17 −0.394.06−0.49 −0.110.95 −0.431.02** 0.00 −0.09
Fund0.50**0.370.58***0.71***0.49***0.56***0.75***0.80***0.66***0.160.56**0.30*−0.57−3.69−0.54−0.03
System0.02−0.030.28*** 0.38**0.13** 0.10−0.03−0.05−0.030.13*−0.030.16**0.00−0.03−0.13*0.08−0.060.160.060.17−0.080.21−0.050.08−0.020.95**0.010.09−0.010.170.25−0.500.210.00−0.020.05**0.08−0.01−0.040.00−0.070.02***−0.140.01−0.12** 0.00−0.030.000.160.000.26** 0.01−0.080.040.01−0.05*0.78*** −0.021.89−0.091.06−0.17−0.220.09
R2adj
0.380.560.340.74−0.010.84−0.050.87−0.020.830.060.840.010.88−0.050.87−0.110.760.31** 0.590.56* 0.340.260.190.460.283.77* −0.162.5*0.110.88 −0.09
Techcur Techdiv Techfin Techoth Nocat
This table reports the persistence in performance analysis of the performance decile regressed against the CTA Global Index and theCTA substrategy indices.t-stat are heteroskedasticity consistent.***Significant at the 1 percent level.**Significant at the 5 percent level.*Significant at the 10 percent level.Numbers in the table are monthly percentages. Arb = arbitrage; Discret = discretionary; Fund = fundamental; Option = option strategy; Techdiv. = technical diversified; Techfin = technical financial/metals; Techcur = technical currency; Techoth = other technical;System = systematic; Stock = stock index; Nocat = no category.
D1D2D3D4D5D6D7D8D9D10D10aD10bD10cD1aD1bD1c
Discret
Stocks
Arb
Option
Alpha
TABLE 4.13 Decile Performance Analysis, January 1986 to December 2002
CTA Performance, Survivorship Bias, and Dissolution Frequencies
73
and D10c are significantly negative. All these figures are different from theones obtained in the performance or performance persistence analysis.The other columns report the exposition of each decile to the strategiesdefined earlier. We analyze the table horizontally, then vertically, but first wewant to underline the fact that negative significant exposure of a decile to astrategy means that the decile negatively contributes to the creation of alpha.Decile D1 (the worst-performing funds) is significantly positively exposed todiscretionary and systematic funds and significantly negatively exposedto option funds. The mean return for decile D1 is 1.24 percent (see Table4.11). Once we take the strategy performance into account, the alpha is−0.58 (See Table 4.13). The difference between these two numbers comesmainly from the exposure to fundamental and systematic funds.11 D2 is significantly positively exposed to technical currency, technical diversified, andtechnically other funds. Interestingly, this decile is not significantly exposedto systematic funds. D3 is significantly positively exposed to systematicfunds and to technical currency funds. D4 is positively exposed to discretionary funds and to systematic funds. D5 is significantly negativelyexposed to arbitrage funds and significantly positively exposed to discretionary, option strategies, and systematic funds.D6 is significantly positively exposed to systematic funds and technically diversified funds and negatively exposed to technical financial/metalfunds. D7 is significantly positively exposed to systematic funds and negatively exposed to technical financial/metals, whereas D8 is positively exposed to discretionary funds and systematic funds. In this particular case,the strategies reported cannot completely explain the alpha (since it is stillweakly significantly positive). D9 is significantly positively exposed to discretionary funds and systematic funds. Finally D10 is significantly positivelyexposed to discretionary funds, technical financial/metals, and to the no category. Note that it is the only decile exposed to the no-category strategy.If we analyze the results in Table 4.13 by columns rather than by rowsto detect the presence of certain strategies in particular deciles (top, middle,or bottom deciles), we find some interesting features. First of all, each strategy has at least one significant coefficient across the deciles (some of themhave only weak coefficients). Interestingly, for most, significance appearsonly once out of the 10 deciles (arbitrage, fundamental, option, stocks, nocategory). Note, however, that most of these strategies contain only a few
11Recall that the systematic strategy is 0.98 correlated with the CTA Global Index
(see Table 4.3). This means that the systematic strategy can be seen as the index inthis particular case.
74
PERFORMANCE
funds, which explains the nonsignificance of these factors for most of thedeciles. Moreover, the funds of those strategies with very few funds are concentrated among one decile.The evolution of certain coefficients is interesting. For the option strategy, for example, the coefficient is significantly negative for D1 and positivefor D5. This indicates that this strategy impacts significantly inversely onsome deciles. For technical currency funds, coefficients are significantly positive for D2 and D3. Then the coefficients decrease and the one of D8 isweakly negatively significant. This also indicates a pattern in the repartitionof this particular strategy. The no category coefficient is high (but not significant) for D1 and D2 and then it decreases and becomes significantlypositive for D10. The pattern of coefficient changes across deciles is perhaps a reflection of over- and under-representation of strategies within thedeciles. The only exception is systematic funds that contain more than halfof the observations. This strategy is presented across almost all deciles.The R2 reported in the last column indicates that the returns of thedeciles are well explained by the model. Except for D1 and D10 they are allgreater than 0.70 percent. The R2 are lower in the subdeciles. This can beexplained by the fact that the subdeciles contain fewer funds.
DISSOLUTION FREQUENCIESBefore concluding, we analyze the dissolution frequencies in our database,defined as the number of funds that stopped reporting to the database. Thismeasure is similar to survivorship bias, the difference being that we analyzeit per decile each year based on the previous year’s performance. This analysis is interesting because it helps us determine if bad performance leads to ahigher dissolution rate the following year and if good performance is a protection against dissolution.
Average Dissolution FrequenciesCapocci and Hübner (2004) have analyzed the dissolution frequencies inhedge funds. They found an average dissolution frequency of 15 percent forbad performing funds and 7 percent for good ones. They concluded that badperformers were more frequently dissolved but that good performance wasnot a protection against dissolution.Figure 4.2 reports mean dissolution frequencies. The frequenciesdecrease monastically from 46.6 percent for the worst-performing funds to11.6 percent for D6. They are at 12.1 percent for D7, 11 percent for D8,and 8.8 percent for D9. D9 has the lowest dissolution frequencies of all the
75
CTA Performance, Survivorship Bias, and Dissolution Frequencies
45%
30%
15%
Mean
D10
D9
D8
D7
D6
D5
D4
D3
D2
D1
0%
FIGURE 4.2
Average Dissolution FrequenciesCTA dissolution frequencies by year t as a function of year t − 1 decile. At thebeginning of year t, all funds are placed into decile rankings on the basis of theirreturns in year t − 1. If a hedge fund ceases to report returns at any time before theend of year t, it is counted as dissolved.
deciles. Interestingly, D10 has a dissolution frequency of 14.9 percent. Theaverage dissolution frequency is 19.2 percent.These numbers indicate some interesting patterns. First, in absoluteterms, the dissolution frequencies for CTA are impressively high. An examination of yearly rates per decile indicates that the maximum annual dissolution frequencies range from 21.9 percent for D8 (in 1998 and 1999) to74.6 percent for D1 (in 2001). Second, poorly performing funds faceimpressively high dissolution frequencies. As indicated in the figure, theaverage dissolution frequency over the 1986 to 2002 period is more than46 percent for previous year’s worst-performing funds. This means thatalmost half of the poorly performing funds are dissolved the year followingtheir bad performance. In recent years this trend is even stronger, with amaximum of dissolution of 74.6 percent in 2001. Third, the dissolution rateof D1 is much higher than that of the other deciles. The closest dissolutionfrequency is that of D2, at 23.9 percent. We can conclude that bad performance leads to dissolution. Finally, the dissolution rates diminish fromD1 to D9 but increase for decile D10. This indicates that good performance is not a protection against dissolution. The dissolution frequency ofD10 (best-performing funds) is higher than those obtained for D5 to D9.Capocci and Hübner (2004) found some qualitative results for hedge fundswith lower values.
76
PERFORMANCE
Yearly Dissolution FrequenciesFigure 4.3 reports the evolution over time of the yearly dissolution frequencies across deciles illustrated in Figure 4.2. Figure 4.3 indicates that thepercentage of dissolved funds was close to zero in the first year studies.Then the dissolution frequencies increase across most deciles until 1993. In1994 the dissolution rates increase for most deciles but decrease for bothD1 and D2 (from 55.3 percent in 1993 to 47 percent in 1994 for D1 andfrom 44.6 percent in 1993 to 40 percent in 1994 for D2). Since then,depending on the year and on the decile considered, the dissolution frequencies increase or decrease.Rates are particularly high in 1999 for the best- and worst-performingfunds, with dissolution frequencies of respectively 39.5 percent (against 30.3percent in 1998 and 29.6 percent in 2000) and 69.4 percent (against61.4 percent in 1998 and 64.8 percent in 2000). Poorly performing fundsface high dissolution frequencies in 1996 (72.9 percent), 1999 (69.4 percent), and 2001 (74.6 percent). Interestingly, D2 has a higher dissolutionfrequency at 60 percent in 1997, equal to the dissolution frequency of D1for that year. Otherwise, in each particular year, D2 has dissolution frequencies always lower than D1, and D9 always has dissolution frequencieslower than D10.
75%65%55%45%35%25%15%5%D2D4D6D8D10
–5%86
FIGURE 4.3
88
90
92
94
96
98
00
02
Evolution of the Yearly Dissolution Frequencies across Decilesbetween 1986 and 2002
CTA Performance, Survivorship Bias, and Dissolution Frequencies
77
60%50%40%30%20%10%0%−10%86 87 88
FIGURE 4.4
89 90 91
92 93 94
D10–D995 96 97
98 99 00
D1–D201 02
D1–D10
Spread between Dissolution Frequencies
Spread AnalysisFigure 4.4 reports the spreads between D1 and D2, D10 and D9, and D1and D10. This figure is interesting because it shows how these spreadsevolve over time. The front part of the figure (D10 minus D9) indicates thatthis spread is almost null (even negative at −1.3 percent in 1992) before1995 and that it increases heavily in 1999, in the heart of the bull market.Spread D1 minus D2 is particularly low in 1994 and 1997. Spread D1minus D10 is particularly high in 1993 (47.4 percent), 1996 (52.9 percent),and 2001 (47.8 percent).This analysis of dissolution frequencies points to several interestingresults. The dissolution frequencies are high for CTAs. Bad performanceleads to dissolutions, and good performance is not a protection against it.The yearly dissolution frequencies increase heavily in the first year of analysis and then vary over time, but they are higher between 1999 and 2001depending on the deciles considered.
CONCLUSIONIn this study, we investigate CTA survivorship bias, performance, and performance persistence. After having made a literature review and analyzingthe descriptive statistics, we have analyzed the correlation between the various CTA strategies. Our results indicate that most of the strategies definedare weakly correlated, indicating a need to separate the funds into investment strategies. The survivorship bias analysis indicates that our CTA database contains a bias of 5.4 percent per year over the whole January 1985 to
78
PERFORMANCE
December 2002 period. The three-year rolling period survivorship analysisindicates that this bias varies significantly over time.Our performance analysis has attempted to determine if some strategies outperform the CTA Global Index, which consists of all the funds inthe database. Our results indicate that out of 11 strategies, 7 significantlyoutperform the index and 2 significantly underperform the index. Subperiod analysis indicates that over- or underperformance remains constantover time. Most strategies are significantly positively exposed to the CTAGlobal Index. Individual fund performance analysis indicates that 21.7percent of the funds offer significantly positive alpha and 16.4 percent significantly underperform the index over the period studied at the 5 percentsignificance level.To perform our persistence analysis, we apply the decile classificationas suggested by Carhart (1997). Then we determine if certain deciles significantly out- or underperform the index over time. Our results indicatethat most deciles significantly underperform the index over the wholeperiod. Subperiod analysis indicates that the best-performing decile, D10,is the only one that significantly outperforms the CTA Global Index inmost time periods. We also have analyzed the exposure of the deciles to thevarious strategy indices and have found that some deciles are moreexposed to various strategies. Finally we have analyzed the dissolution frequencies in each decile over time. Our results indicate that the dissolutionfrequencies are particularly high for CTA, that bad performance leads todissolution, and that good performance is not a protection against it.Three results are particularly relevant for investors or fund of fundmanagers who want to allocate part of their portfolio to CTAs.1. Some CTA strategies outperform the average over time. However,few funds offer persistence in performance. Most significantly underperform over time.2. Over time, more than 20 percent of the individual funds significantlyoutperform their peers, but almost all funds classified in a decile on thebasis of their previous year’s performance underperform the CTAGlobal Index.3. Dissolution is a real issue in CTA performance since dead funds significantly underperform existing ones, and dissolution frequencies canreach 60 percent in difficult months for poorly performing funds.The next step in analyzing CTA performance is to apply our modeland the decile analysis to individual strategies, to determine if there are differences in the results for individual strategies and to test the robustness ofthe results.
CHAPTER
5
CTA Performance Evaluationwith Data Envelopment AnalysisGwenevere Darling, Kankana Mukherjee, and Kathryn Wilkens
e apply data envelopment analysis to a performance evaluation framework for CTAs. The technique allows us to integrate several performance measures into one efficiency score by establishing a multidimensionalefficient frontier. Two dimensions of the frontier are consistent with thestandard Markowitz mean-variance framework, while additional risk andreturn dimensions include skewness and kurtosis. We also illustrate amethod of analyzing determinants of efficiency scores. Tobit regressions ofefficiency scores on equity betas, beta-squared, fund size, length of managertrack record, investment style (market focus), and strategy (discretionary vs.systematic) are performed for CTA returns over two time frames representing different market environments. We find that the efficiency scores arenegatively related to beta-squared in both time periods. Results also indicate that emerging CTAs (those with shorter manager track records) tend tohave better efficiency scores as defined by the DEA model used in our study.This relationship is strongest during the period from 1998 to 2000, but notstatistically significant during the period from 2000 to 2002. For both timeperiods, fund size is not related to efficiency scores.
W
INTRODUCTIONIndustry performance reports for commodity trading advisors (CTAs)present multiple performance measures such as return, standard deviation,drawdowns, betas, and alphas. Investors and fund managers recognize theimportance of considering a multitude of performance measures to analyzefund risk from various perspectives. It is particularly important for thegrowing alternative investment class of managed futures, which have dif-
79
80
PERFORMANCE
ferent risk/return profiles from those of traditional mutual funds as well asthose of many hedge fund strategies. For all asset classes, however, the academic literature has done little to offer a comprehensive framework thatincorporates multiple risk measures in an integrated fashion (Arnott2003). Too often, studies focus on single measure of risks, arguing for onerelative to another.“Managed futures” are a subset of hedge funds that uses futures contracts as one among several types of trading instruments (including swapsand interbank foreign exchange markets) and for which futures are ameans, rather than an end, with which to implement their strategy. Thename wrongly suggests that futures are the dog rather than the tail. Managed futures encompass the broad set of individual commodity tradingadvisors (CTAs). CTAs are also unfortunately named because, on balance,most of their trading is in the financial markets, not the commodity markets. Like any other class of alternative investments, managers are represented by a variety of styles and substyles. For example, there are systematicand discretionary CTAs, CTAs who exclusively try to capture trends, thosewho identify countertrend opportunities, and those who combine the twoapproaches.1In this study we look at the performance of CTAs based on multiple criteriausing data envelopment analysis (DEA). DEA establishes a multidimensionalefficient frontier and assigns each CTA an efficiency score whereby 1 (or 100percent) indicates perfect efficiency and scores lower than 1 represent relatively less efficient CTAs based on the performance criteria chosen.The criteria we choose as bases for performance evaluation are monthlyreturns, kurtosis, minimum return, skewness, standard deviation of returns,and percentage of negative monthly returns. Although there are many otherpossibly appropriate criteria, those not included here are likely either to beredundant with variables included or to not make sense in an optimizationframework. Criteria that make sense in this framework are those that aredesirable to maximize or minimize across various market conditions. Thisaspect leads us to reject equity betas as a criterion in the DEA model, for