Estimated betas and optimal portfolio I ran a regression on 20 assets to estimate their beta with different methods. I would like to see the differences of these estimation differences in terms of mean-variance optimal portfolio. How can I do that?

The problem is that I do not see clearly the role that the estimated beta plays in portfolio optimization empirically. A mean-variance optimization with N assets should be something like

$min_{\textbf{w}}$ $\textbf{w}^{'}\textbf{V}\textbf{w}$

subject to

$\textbf{w}^{'}E(R)=\bar{R}$,$\textbf{w}^{'}\textbf{I}=1$ 

Where w are the weights of the N assets, our choice variable. $\textbf{V}$ is the covariance matrix of the returns, $\bar{R}$ is the target return, $\textbf{I}$ is just a vector of ones and $E(R)$ is the vector of expected returns of our assets.

The "unknowns" are V and E(R), should I use historical returns to replace them? If yes, then what is the use of the betas in portfolio optimization? none? or maybe should I use the betas to get the values of E(R) and V?

I do not understand, could someone provide a mini-guide to get from betas to the optimizing portfolio?
Let me begin with noting that models such as the CAPM have been extensively falsified, beginning with Mandelbrot in 1963 and ending with the Fama-MacBeth testing in 1973.  Other falsifications followed, but really, it has been a zombie theory since the early seventies.  The difficulty is that the falsification has been like the Michaelson-Morley experiments in the 1880s.  Once they were done, classical physics was falsified, but Plank and Einstein were not around yet.

I have a paper in peer review proposing a new stochastic calculus for options pricing and portfolio construction, but there is nothing resembling a consensus on a replacement.  The math is very different.

The core of your problem can understood in two equations.  The first is $x_{t+1}=\mu{x}_t+\epsilon_{t+1}$.  The second is $r_t=\frac{p_{t+1}q_{t+1}}{p_tq_t}-1$.  

For the first equation, for someone to be willing to invest, $\mu&gt;1$.  It was proven in 1958 that this problem has no meaningful solution in Frequentist statistics.  The CAPM is a Frequentist model.  For the CAPM, this means that $\tilde{w}=R\bar{w}+\varepsilon$ only has a solution when $R$ is known, which is an assumption of the math underlying the model.  In other words, models such as the CAPM and Black-Scholes depend on it being true that people know the true values of the parameters.  If they do not, it is known that these models cannot be solved.  The argument has been that the market behaves as if it knew the parameters.  A Bayesian derivation of the CAPM results in a Cauchy likelihood not a normal one.

The second equation makes returns a function of data rather than a known parameter.  As such, it must be derived.  I have a paper deriving the distribution of returns.  The distribution is the product of the ratio of prices and the ratio of volumes if dividends are ignored.  Dividends create their own complications.

However, from auction theory, it can be determined that the dominant element of the resulting mixture distribution is the truncated Cauchy distribution.  It has no first moment, so $\beta$ cannot exist as described in the CAPM.  That is different from saying that it cannot be found.  Such a thing implies the non-existence of the population parameter as the model describes it.

I left out the logarithmic models for two reasons.  First, the CAPM has additive errors.  Second, the distribution created by taking the log of Cauchy distributed variables is the hyperbolic secant distribution. It has no covariance structure in the multivariate form, so the CAPM is defeated anyway.

A short proof that the CAPM cannot function would be to note that if $q_t=q_{t+1}$ which rules out mergers and bankruptcies and $\delta_t=0,\forall{t}$ and there are no dividends, then return is $\frac{p_{t+1}}{p_1}$.  Because stocks are sold in a double auction, there is no winner’s curse.  The rational behavior is to bid your expectation.  Using the assumption of many buyers and sellers from the CAPM, the limiting distribution of the limit book must be normal.

If you center the distribution of prices around the equilibrium prices and assume the market is approximately in equilibrium, ruling out things such as a liquidity crisis, then you end up with a bivariate normal distribution.  The equilibrium return $r^*=\frac{p_{t+1}^*}{p_t^*}$.  Errors could be thought of as excess return, plus or minus.

Noting that the slope is the same as the tangent of the angle, you quickly get very close to the Cauchy distribution by observing that $$\theta=\tan^{-1}(r).$$  The arctangent is the kernel of the cumulative density function of the Cauchy distribution.  The angle is, minimally, a transform of returns via the Cauchy distribution.

Solving for the ratio of two normal distributions by integrating around the equilibrium rather than (0,0), in other words treating the equilibrium as the location of no error, or conceptually by operating in the error space rather than the price space, give you a Cauchy distribution.  It is well known in statistics that the ratio of two normal distributions centered around zero is a Cauchy.  By treating the equilibrium as zero, it still comes out a Cauchy distribution truncated at -100%.  If there exists just one stock that didn't pay a dividend, then the CAPM is intrinsically falsified.

So the answer to your question is, there isn’t a solution.

If you want to optimize a portfolio, use the Kelly bet instead.

Visually, I took the returns for Carnival Cruises, I didn't subtract one, I just took the ratio.  I performed a kernel density estimate having removed the return from dividends to make it consistent with the above.  I then mapped the truncated Cauchy distribution from the maximum likelihood estimator.  As you can see, even if everything I wrote above is false, there cannot be a $\beta$ as described in the CAPM.



There are actually three ways to improve the fit, but I used the method most quants would use instead.