Did previous researchers fail to detect the hot hand simply because of a statistical fallacy? Many basketball fans/players believe that having made several shots in a row,  the next shot is more likely to go in. This is sometimes called the hot hand.

Starting (I think) with Gilovich, Mallone, and Tversky (1985), it was "shown"  that this was in fact a fallacy. Even if several shots in a row have gone in, the next shot is no more likely to go in than your average shooting percentage would dictate. 

Miller and Sanjurjo (2015) argue that the hot hand does in fact exist and previous researchers had simply fallen prey to a fairly basic statistical fallacy. Their argument is something like this:

Flip a coin four times. Compute the probability that H follows H. To give a few examples: HHTT would have probability 1/2, HTHT would have probability 0/2, TTHH would have probability 0/1 1/1, and both TTTT and TTTH would be N.A.

Miller and Sanjurjo's punchline is that the expected value of this probability is not 0.5, but ≈0.4. And the error made by previous researchers was to incorrectly assume that the expected value of this probability is 0.5. So if for example these previous researchers conducted the above coin-flipping experiment and found the average probability to be say 0.497, they incorrectly concluded that there was no evidence of a hot hand (not significantly different from 0.5), when in fact there was very strong evidence of a hot hand (significantly different from 0.4).

My question is this: Are Miller and Sanjurjo correct that previous researchers failed to detect the hot hand simply because of this mistake? I have only skimmed one or two papers on this so I wanted to get some confirmation from someone here who might know this literature better. This seems like a surprisingly silly error to have persisted for three decades or more.
(This answer was completely rewritten for greater clarity and readability in July 2017.)
Flip a coin 100 times in a row.
Examine the flip immediately after a streak of three tails. Let $\hat{p}(H|3T)$ be the proportion of coin flips after each streak of three tails in a row that are heads. Similarly, let $\hat{p}(H|3H)$ be the proportion of coin flips after each streak of three heads in a row that are heads. (Example at bottom of this answer.)
Let $x:=\hat{p}(H|3H)-\hat{p}(H|3T)$.
If the coin-flips are i.i.d., then &quot;obviously&quot;, across many sequences of 100 coin-flips,
(1) $x&gt;0$ is expected to happen as often as $x&lt;0$.
(2) $E(X)=0$.
We generate a million sequences of 100 coin-flips and get the following two results:
(I) $x&gt;0$ happens roughly as often as as $x&lt;0$.
(II) $\bar{x} \approx 0$ ($\bar{x}$ is the average of $x$ across the million sequences).
And so we conclude that the coin-flips are indeed i.i.d. and there is no evidence of a hot hand. This is what GVT (1985) did (but with basketball shots in place of coin-flips). And this is how they concluded that the hot hand does not exist.

Punchline: Shockingly, (1) and (2) are incorrect. If the coin-flips are i.i.d., then it should instead be that
(1-corrected) $x&gt;0$ occurs only about 37% of the time, while $x&lt;0$ occurs about 60% of the time. (In the remaining 3% of the time, either $x=0$ or $x$ is undefined — either because there was no streak of 3H or no streak of 3T in the 100 flips.)
(2-corrected) $E(X) \approx -0.08$.
The intuition (or counter-intuition) involved is similar to that in several other famous probability puzzles: the Monty Hall problem, the two-boys problem, and the principle of restricted choice (in the card game bridge). This answer is already long enough and so I'll skip the explanation of this intuition.
And so the very results (I) and (II) obtained by GVT (1985) are actually strong evidence in favor of the hot hand. This is what Miller and Sanjurjo (2015) showed.

Further analysis of GVT's Table 4.
Many (e.g. @scerwin below) have — without bothering to read GVT (1985) — expressed disbelief that any &quot;trained statistician would ever&quot; take an  average of averages in this context.
But that is exactly what GVT (1985) did in their Table 4.
See their Table 4, columns 2-4 and 5-6, bottom row. They find that averaged across the 26 players,

$\hat{p}(H|1M) \approx 0.47$ and $\hat{p}(H|1H) \approx 0.48$,
$\hat{p}(H|2M) \approx 0.47$ and $\hat{p}(H|2H) \approx 0.49$,
$\hat{p}(H|3M) \approx 0.45$ and $\hat{p}(H|3H) \approx 0.49$.

Actually it is the case that for each $k=1,2,3$, the averaged $\hat{p}(H|kH)&gt;\hat{p}(H|kM)$. But GVT's argument seems to be that these are not statistically significant and so these are not evidence in favor of the hot hand. OK fair enough.
But if instead of taking the average of averages (a move considered unbelievably stupid by some), we redo their analysis and aggregate across the 26 players (100 shots for each, with some exceptions), we get the following table of weighted averages.
Any                     1175/2515 = 0.4672

3 misses in a row       161/400 = 0.4025
3 hits in a row         179/313 = 0.5719

2 misses in a row       315/719 = 0.4381
2 hits in a row         316/581 = 0.5439        

1 miss in a row         592/1317 = 0.4495
1 hit in a row          581/1150 = 0.5052

The table says, for example, that a total of 2,515 shots were taken by the 26 players, of which 1,175 or 46.72% were made.
And of the 400 instances where a player missed 3 in a row, 161 or 40.25% were immediately followed by a hit. And of the 313 instances where a player hit 3 in a row, 179 or 57.19% were immediately followed by a hit.
The above weighted averages seem to be strong evidence in favor of the hot hand.
Bear in mind that the shooting experiment was set up so that each player was shooting from where it had been determined he/she could make roughly 50% of his/her shots.
(Note: &quot;Strangely&quot; enough, in Table 1 for a very similar analysis with the Sixers' in-game shooting, GVT instead present the weighted averages. So why didn't they do the same for Table 4? My guess is that they certainly did calculate the weighted averages for Table 4 — the numbers I present above, didn't like what they saw, and chose to suppress them. This sort of behavior is unfortunately par for the course in academia.)


Example: Say we have the sequence $HHHTTTHHHHH…H$ (only flips #4-#6 are tails, the remaining 97 flips are all heads). Then $\hat{p}(H|3T)=1/1=1$ because there is only 1 streak of three tails and the flip immediately after that streak is heads.
And $\hat{p}(H|3H)=91/92 \approx 0.989$ because there are 92 streaks of three heads and for 91 of those 92 streaks, the flip immediately after is heads.


P.S. GVT's (1985) Table 4 contains several errors. I spotted at least two rounding errors. And also for player 10, the parenthetical values in columns 4 and 6 do not add up to one less than that in column 5 (contrary to the note at the bottom). I contacted Gilovich (Tversky is dead and Vallone I am not sure), but unfortunately he no longer has the original sequences of hits and misses. Table 4 is all we have.