## Geometric Distribution

Suppose we have a biased coin with $\mathbb{P}(H) = p$ and we toss it repeatedly, then the sample space will look like 

$$\Omega = \{H, TH, TTH, \ldots \}$$
Then 
$$\mathbb{P}(TH) = \mathbb{P}(T)\cdot \mathbb{P}(H) = (1-p)p$$

Let $X$ is the random variable defined as 

$$X = \#\text{ tosses untill first heads }$$
Then 
$$\begin{align}\mathbb{P}(X=x) &= \mathbb{P}(x-1 \text{ tails followed by heads}) \\
&= (1-p)^{x-1}p \end{align}
$$
This is probability mass function of random variable $X$. If we normalize we will get the probability distribution.

### Practice Problem: The Geometric Distribution

Let $X \sim \text {Geo}(p)$ so that

$$p_ X(x) = (1-p)^{x-1} p\qquad \text {for }x=1, 2, \dots $$
 
**Question:** Show that each of the table entries $p_ X(x)$ is nonnegative for $x = 1, 2, \dots.$

**Solution:** Since $p$ is a probabiltiy hence $p \in [0,1]$ then $1-p \in [0,1]$, also $x-1 \geq 0$. Hence 

$$p_X(x) = \underbrace{(1-p)^{x-1}}_{\geq 0}\cdot \underbrace{p}_{\geq 0} \geq 0.$$ 

**Question:**Show that the sum of all the table entries is 1, i.e., $\sum _{x=1}^\infty p_ X(x) = 1.$

**Hint:** You may find the following result from calculus helpful: For $r\in (-1,1),$

$$\sum _{i=0}^{\infty }r^{i}=\frac{1}{1-r}.$$

Solution: For $p\in (0,1),$

$$\begin{align}
\sum _{x=1}^{\infty }p_{X}(x)&=\sum _{x=1}^{\infty }(1-p)^{x-1}p &\\
&= p \cdot \sum _{i=0}^{\infty }(1-p)^{i}&\text{where } i = x-1\\
&= p \cdot \frac{1}{1-(1-p)}&\\
&= p \cdot \frac{1}{p}=1.&
\end{align}.$$

### Exercise: The Expectation of a Geometric Distribution

In this exercise, we use the law of total expectation to find the expected value of a geometric random variable. The law of total expectation says that for a random variable $X$ (with alphabet $\mathcal{X}$) and a partition $\mathcal{B}_1,\dots ,\mathcal{B}_ n$ of the sample space,

$$\mathbb {E}[X]=\sum _{i=1}^{n}\mathbb {E}[X\mid \mathcal{B}_{i}]\mathbb {P}(\mathcal{B}_{i}),$$
 
where

$$\mathbb {E}[X\mid \mathcal{B}_{i}] = \sum _{x\in \mathcal{X}}xp_{X\mid \mathcal{B}_{i}}(x) = \sum _{x\in \mathcal{X}}x\frac{\mathbb {P}(X=x,\mathcal{B}_{i})}{\mathbb {P}(\mathcal{B}_{i})}.$$
 
Let $X \sim \text {Geo}(p)$ be the number of tosses until we get heads for the first time, where the probability of heads is $p$. Let $\mathcal{B}$ be the event that we get heads in 1 try. Let $\mathcal{B}^c$ be the event that we get heads in more than 1 try. Note that $\mathcal{B}$ and $\mathcal{B}^c$ form a partition of the sample space.

**Question:** What is $\mathbb {P}(\mathcal{B})?$

**Solution:** $\mathcal{B}$ is the event of getting "head" in one coin toss. Hence, $\mathbb {P}(\mathcal{B})=p.$

**Question:** What is $\mathbb {E}[X \mid \mathcal{B}]?$

**Solution:** 

$$\begin{align}\require{cancel}
\mathbb{E}[X\mid \mathcal{B}] &= \sum_{x=1}^{\infty} xp_{X\mid \mathcal{B}}(x)\\
&= \sum_{x=1}^{\infty} x\frac{\mathbb {P}(X=x,\mathcal{B})}{\mathbb{P}(\mathcal{B})}\\
&= 1\times \frac{\cancelto{\mathbb{P}(\mathcal{B})}{\mathbb {P}(X=1,\mathcal{B})}}{\mathbb{P}(\mathcal{B})} + \sum_{x=2}^{\infty} x\frac{\cancelto{0}{\mathbb {P}(X=x,\mathcal{B})}}{\mathbb{P}(\mathcal{B})}\\
&= 1.
\end{align}$$

**Question:** What is $\mathbb{E}[X \mid \mathcal{B}^c]?$ Write your answer in terms of $m \triangleq \mathbb {E}[X].$ Note that we do not know $\mathbb {E}[X]$ for right now, but we can still relate $\mathbb {E}[X \mid \mathcal{B}^ c]$ to $\mathbb {E}[X].$

Hint: If you do not get heads the first time, then starting from the second toss, the distribution for the number of tosses remaining is still geometric!

**Solution:** So we tossed the coin and it was tails, so this took up 1 toss. The number of tosses that remains is just another $\text {Geo}(p)$ random variable (remember: the tosses are all independent so that initial toss doesn't affect any of the future tosses)!

Thus, the expectation of $X$ given that the first toss was tails (i.e., it takes more than 1 try) is

$$\mathbb {E}[X \mid \mathcal{B}^ c] = 1 + \mathbb {E}[X].$$
 
The 1 appears because that's the first toss where we got tails.

Using the law of total expectation,

$$\mathbb {E}[X] = \mathbb {E}[X \mid \mathcal{B}]\mathbb {P}(\mathcal{B}) + \mathbb {E}[X \mid \mathcal{B}^ c](1 - \mathbb {P}(\mathcal{B})).$$
 
Using your answers to the previous part, you should now have a recursive equation, meaning that the unknown quantity $\mathbb {E}[X]$ appears on both sides of the equation, and so you can solve for it.

**Question:** What is $\mathbb {E}[X]?$

In this part, please provide your answer as a mathematical formula (and not as Python code). Use ^ for exponentiation, e.g., x^2 denotes x^2. Explicitly include multiplication using *, e.g. x*y is xy.

**Solution:** Putting together the pieces from the previous parts,

$$\begin{align}\mathbb {E}[X] &= 1 \cdot p + (1 + \mathbb {E}[X]) \cdot (1 - p)\\	 	 
&= p + 1 - p + \mathbb {E}[X] - \mathbb {E}[X] p\\	 	 
&= 1 + \mathbb {E}[X] - \mathbb {E}[X] p.\end{align}$$

Rearranging terms yields

$$\mathbb {E}[X] = \boxed {\frac1p}.$$


```python

```

88c1651fa6af2157e7961f08b728f5ff37b14857
ipynb
Jupyter Notebook
week04/05 Geometric Distribution.ipynb
infimath/Computational-Probability-and-Inference
e48cd52c45ffd9458383ba0f77468d31f781dc77
MIT
2019-04-04T03:07:47.000Z
2019-04-04T03:07:47.000Z
week04/05 Geometric Distribution.ipynb
infimath/Computational-Probability-and-Inference
e48cd52c45ffd9458383ba0f77468d31f781dc77
MIT
week04/05 Geometric Distribution.ipynb
infimath/Computational-Probability-and-Inference
e48cd52c45ffd9458383ba0f77468d31f781dc77
MIT
2021-02-27T05:33:49.000Z
2021-02-27T05:33:49.000Z
Qwen/Qwen-72B
1. YES
2. YES