# Contitional probability

Let us assume that we have two random variables $X$ and $Y$.  Both are generated by rolling fair six-sided dice, so the possible outcomes are:

\begin{align}
Pr(X=1) & = \frac{1}{6} & Pr(Y=1) & = \frac{1}{6} \\
Pr(X=2) & = \frac{1}{6} & Pr(Y=2) & = \frac{1}{6} \\
Pr(X=3) & = \frac{1}{6} & Pr(Y=3) & = \frac{1}{6} \\
Pr(X=4) & = \frac{1}{6} & Pr(Y=4) & = \frac{1}{6} \\
Pr(X=5) & = \frac{1}{6} & Pr(Y=5) & = \frac{1}{6} \\
Pr(X=6) & = \frac{1}{6} & Pr(Y=6) & = \frac{1}{6} 
\end{align}

Further, we assume that $X$ and $Y$ are <i>independent</i>.  

So, if one defines notation $Pr(X=x \cap Y=y)$ to mean the probability that $X=x$ <i>and</i> $Y=y$
then $X$ and $Y$ being independent means that $Pr(X=x \cap Y=y) = Pr(X=x)Pr(Y=y)$.

For example, $Pr(X=1 \cap Y=2) = Pr(X=1)Pr(Y=2) = \frac{1}{6} \frac{1}{6} = \frac{1}{36}$.


With this notation in mind, we define the <i>conditional</i> probality of $X=x$ <i>given</i> $Y=y$, denoted by $Pr(X=x | Y=y)$, as:

$$
Pr(X=x | Y=y) = \frac{Pr(X=x \cap Y=y)}{Pr(Y=y)}
$$

If $X$ and $Y$ are independent then the conditional probability is 

$$
Pr(X=x | Y=y) = \frac{Pr(X=x \cap Y=y)}{Pr(Y=y)} = \frac{Pr(X=x) Pr(Y=y)}{Pr(Y=y)} = Pr(X=x). 
$$

In other words, knowing $Y=y$ does not provide any information about $Pr(X=x)$.

Note, since

$$
Pr(X=x | Y=y) = \frac{Pr(X=x \cap Y=y)}{Pr(Y=y)}
$$

we also have that

$$
Pr(X=x | Y=y)Pr(Y=y) = Pr(X=x \cap Y=y).
$$

So, we have the following identity

$$
Pr(X=x | Y=y)Pr(Y=y) = Pr(X=x \cap Y=y) = Pr(Y=y \cap X=x) = Pr(Y=y | X=x)Pr(X=x).
$$



# Example

Things get more interesting when you have dependent random variables so let us define the random variables $Z$ by setting $Z=X+Y$.  All of the possible values for $Z$ as a function of $Y$ and $Z$ can be seen in the following table.

x\Y  |  1 | 2 | 3 | 4 | 5 | 6  
-----|:---|:--|:--|:--|:--|:--
1    |  2 | 3 | 4 | 5 | 6 | 7 
2    |  3 | 4 | 5 | 6 | 7 | 8 
3    |  4 | 5 | 6 | 7 | 8 | 9 
4    |  5 | 6 | 7 | 8 | 9 | 10 
5    |  6 | 7 | 8 | 9 | 10| 11 
6    |  7 | 8 | 9 | 10| 11| 12    

Let us compute the

$$
Pr(Z=5 \cap X=2).
$$

Note there are two ways to proceed since

$$
Pr(Z=5 \cap X=2) = Pr(Z=5 | X=2) Pr(X=2) = Pr(X=2 | Z=5) Pr(Z=5)
$$

Doing it the first way we have that

\begin{align}
Pr(Z=5 \cap X=2) &= Pr(Z=5 | X=2) Pr(X=2) \\
                 &= Pr(Y=3) Pr(X=2) \\
                 &= \frac{1}{6} \frac{1}{6} \\
                 &= \frac{1}{36} 
\end{align}

We use the face that $Pr(Z=5 | X=2) = Pr(Y=3)$ since, given that $X=2$, $Z=5$ if, and only if, $Y=3$.

Doing it the second way we have that

\begin{align}
Pr(X=2 \cap Z=5) &= Pr(X=2 | Z=5) Pr(Z=5) \\
                 &= \frac{1}{4} \frac{4}{36} \\
                 &= \frac{1}{36} 
\end{align}

We use the face that $Pr(Z=5)$ in 4 of the 36 entries in the table above giving us that
$Pr(Z=5) = \frac{4}{36}$.  We also know that the four places where  $Z=5$ occur when

\begin{align}
X=1 ,\;& Y = 4 \\
X=2 ,\;& Y = 3 \\
X=3 ,\;& Y = 2 \\
X=4 ,\;& Y = 1 
\end{align}

and $X=2$ in exactly one of those places, so $Pr(X=2 | Z=5) = \frac{1}{4}$.

The two ways match!



```python

```

516962c6b036565d05b2817e07a3f3d00817e995
ipynb
Jupyter Notebook
lectures/05 Basic Statistics, Probability, and Linear Algebra/ConditionalProbabilty.ipynb
thomasmeagher/DS-501
b5c697c3bc4f44903af16219f242b5728c9a82d1
MIT
2017-07-27T02:52:06.000Z
2020-04-01T09:25:14.000Z
lectures/05 Basic Statistics, Probability, and Linear Algebra/ConditionalProbabilty.ipynb
thomasmeagher/DS-501
b5c697c3bc4f44903af16219f242b5728c9a82d1
MIT
lectures/05 Basic Statistics, Probability, and Linear Algebra/ConditionalProbabilty.ipynb
thomasmeagher/DS-501
b5c697c3bc4f44903af16219f242b5728c9a82d1
MIT
2017-07-18T21:50:17.000Z
2020-04-01T09:25:18.000Z
Qwen/Qwen-72B
1. YES
2. YES