Why do riskier investments pay more? I'm talking about bonds, stocks, and the sort.

I understand that an individual investor that's planning to invest, say, 50% of his savings, may require a higher expected gain to go for a riskier investment.

However, if risk can be diversified away, wouldn't people with enough money, even if they are risk averse, choose the investment opportunities with highest expected value and diversify among them? Wouldn't then any investment with higher-than-average expected value be flooded with money until they almost don't pay more?

Is it that...


most risky investment options are too correlated to allow for almost all of the excess risk to dissolve?
risk premiums only happen in markets with limitted access or that don't work like the ideal 101 market in a relevant way?
some other reason?
(1) You have already correctly pointed out that correlation makes it difficult to diversify away all the risk.

(2) You have also correctly alluded to the fact that most people are risk averse and hence that risky products tend to have a risk premium.

(3) What you have not mentioned -- and what I think is most important in the real world -- is the uncertainty premium.



The distinction between risk and uncertainty is usually credited to Frank Knight (1921, pp. 43–48).

Where we can quite precisely calculate probabilities and expected values, such as with lottery tickets and roulette wheels, we have risk. Where we cannot, such as with most real-world events, we have uncertainty.



Example. Asset $A$ pays $\$100$ with probability (w.p.) $1$ and so has expected value $\$100$.

Asset $B$ pays $\$300$ w.p. $0.5$ but loses $\$100$ w.p. $0.5$. So, the expected value of $B$ is: $$0.5 \times \$300 + 0.5 \times (-\$100) = \$100.$$

Each asset has the same expected value $\$100$.

(2) Nonetheless, because most people are risk averse, most people would prefer $A$ to $B$. That is, they'd be willing to pay more for $A$ than $B$. Or equivalently, $B$ would command a risk premium.

But any such risk premium should be small in the real world, since in the real world, as you've pointed out, there will probably be (i) ways to diversify away the risk; and also (ii) sufficiently many risk-neutral or risk-seeking individuals who bid away the risk premium.



(3) But the real world is rarely like the above example, which involves risk. The real world usually involves uncertainty:

Example. Asset $C$ is a US government bond that will pay out $\$100$.

Asset $D$ is a stock that -- based on your data, models, and analysis -- you expect will pay out $\$300$ w.p. $0.5$ but lose $\$100$ w.p. $0.5$.

Again, each of assets $C$ and $D$ has expected value $\$100$.

But now, even if you're risk-neutral, you'd probably prefer $C$ to $D$. That is, you'd pay more for asset $C$ than asset $D$. Equivalently, asset $D$ has an uncertainty premium.

The reason is that you do not (and should not) completely trust your data, models, and analysis. There are fundamental limits to your knowledge over and beyond mere risk -- we may call such epistemological limits uncertainty.