# Joint Probability Distribution

### Introduction

Before starting with *Joint probabilty distribution* we can have a look at Probability Density Function.

**Probability Density Function of a Discrete Random Variable**
Let X be a discrete rv taking distinct values x1, x2, ... , xn, .... Then the function

 $$ f (x) = P(X = x_{i})   \hspace{1cm}          for \hspace{0.1cm} i = 1, 2, ... , n, ...$$
  
 $$       = 0              \hspace{1cm}          for \hspace{0.1cm} x = x_{i}  $$  
          
               
is called the **discrete probability density function** (PDF) of X, where P(X = $x_{i}$) means the probability that the discrete rv X takes the value of $x_{i}$ .   

**Probability Density Function of a Continuous Random Variable**
Let X be a continuous rv. Then f(x) is said to be the PDF of X if the following conditions
are satisfied:  


$$f(x) ≥ 0 $$
                    
$$ \int_{−\infty}^{\infty} f(x) = 1  $$
                    
$$ \int_a^b f(x) = P(a ≤ x ≤ b) $$


where f (x) dx is known as the probability element (the probability associated with a small interval of a continuous variable) and where P(a ≤ x ≤ b) means the probability that X lies in the interval a to b.
For a continuous rv, in contrast with a discrete rv, the probability that X takes a specific
value is zero;$x^{3}$ probability for such a variable is measurable only over a given range or interval, such as (a, b).


###  Joint Probability Distribution

The joint probability distribution can be expressed either in terms of a **joint cumulative distribution function** or in terms of a **joint probability density function** (in the case of continuous variables) or **joint probability mass function** (in the case of discrete variables). 

These in turn can be used to find two other types of distributions: the **marginal distribution** giving the probabilities for any one of the variables with no reference to any specific ranges of values for the other variables, and the **conditional probability distribution** giving the probabilities for any subset of the variables conditional on particular values of the remaining variables.  

**Joint Probability Density Function**   

Let X and Y be two discrete random variables. Then the function


$$f (x, y) = P(X = x \hspace{0.1cm}and\hspace{0.1cm} Y = y)$$  

$$ \hspace{3.5cm} = 0 \hspace{1cm}when \hspace{0.1cm}X \neq x  \hspace{0.2cm}and \hspace{0.2cm} Y \neq y$$

is known as the discrete joint probability density function and gives the (joint) probability that X takes the value of x and Y takes the value of y.


### Marginal Probability Distribution

In probability theory and statistics, the marginal distribution of a subset of a collection of [random variables](https://en.wikipedia.org/wiki/Random_variable) is the [probability distribution](https://en.wikipedia.org/wiki/Probability_distribution) of the variables contained in the subset. It gives the probabilities of various values of the variables in the subset without reference to the values of the other variables.

The **marginal probability** is the probability of a single event occurring, independent of other events .

**Marginal Probability Density Function**  

In relation to *f (x, y), f (x) and f (y)* are called individual, or marginal, probability density functions. These marginal PDFs are derived as follows: 

$$f (x) = \sum_{y}f (x, y)  \hspace{1cm} marginal\hspace{0.1cm} PDF\hspace{0.2cm} of\hspace{0.2cm} X$$ 

$$f (y) = \sum_{x}f (x, y)  \hspace{1cm} marginal\hspace{0.1cm} PDF \hspace{0.2cm}of\hspace{0.2cm} Y$$

where, for example $\sum_{y}$,  means the sum over all values of Y and $\sum_{x}$ means the sum over all values of X.

### Conditional Probability Distribution

A conditional probability, is the probability that an event occurs given that another specific event has already occurred. This means that the calculation for one variable is dependent on another variable.  

In probability theory and statistics, given two jointly distributed random variables X and Y, the conditional probability distribution of Y given X is the probability distribution of Y when X is known to be a particular value; in some cases the conditional probabilities may be expressed as functions containing the unspecified value x of X as a parameter. When both X and Y are categorical variables, a conditional probability table is typically used to represent the conditional probability.

**Conditional PDF**

The function 
$$f (x | y) = P(X = x | Y = y)$$ 

is known as the conditional PDF of X; it gives the probability that X takes on the value of x given that Y has assumed the value y. Similarly,
$$f (y | x) = P(Y = y | X = x)$$

which gives the conditional PDF of Y.
The conditional PDFs may be obtained as follows: 

$$f(x | y) = \frac{f (x, y)}{f (y)} \hspace{1cm}conditional PDF\ of\ X$$
$$f (y | x) = \frac{f (x, y)}{f (x)} \hspace{1cm} conditional PDF\ of\ Y $$
As the preceding expressions show, the conditional PDF of one variable can be expressed
as the ratio of the joint PDF to the marginal PDF of another (conditioning) variable.

Now we will be looking at some problem examples on Joint Probability, Marginal Probability and Conditional Probability . So, before moving on let us take a simple example.  

**Example 1.** A survey was taken to find the favourite food people . The data is as follows :



```python
import pandas as pd

df=pd.DataFrame({
    'Male' : [80,100,50,230],
    'Female': [120,25,125,270],
    'Total':[200,125,175,500]
},
index= ["Indian","Italian","Other",'Total'],)
df
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Male</th>
      <th>Female</th>
      <th>Total</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>Indian</th>
      <td>80</td>
      <td>120</td>
      <td>200</td>
    </tr>
    <tr>
      <th>Italian</th>
      <td>100</td>
      <td>25</td>
      <td>125</td>
    </tr>
    <tr>
      <th>Other</th>
      <td>50</td>
      <td>125</td>
      <td>175</td>
    </tr>
    <tr>
      <th>Total</th>
      <td>230</td>
      <td>270</td>
      <td>500</td>
    </tr>
  </tbody>
</table>
</div>



Now we find **Joint Probability Distribution** of the above data. This can be done by dividing the overall data with the total i.e. **500**.


```python
# Joint Probability
ndf=pd.DataFrame()
ndf=df.div(500)
ndf
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Male</th>
      <th>Female</th>
      <th>Total</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>Indian</th>
      <td>0.16</td>
      <td>0.24</td>
      <td>0.40</td>
    </tr>
    <tr>
      <th>Italian</th>
      <td>0.20</td>
      <td>0.05</td>
      <td>0.25</td>
    </tr>
    <tr>
      <th>Other</th>
      <td>0.10</td>
      <td>0.25</td>
      <td>0.35</td>
    </tr>
    <tr>
      <th>Total</th>
      <td>0.46</td>
      <td>0.54</td>
      <td>1.00</td>
    </tr>
  </tbody>
</table>
</div>



This is the Joint Probability Distribution of the given data.

|Male||Female||Total|
|----||------||-----|
|**0.16**||**0.24**||0.4|
|**0.20**||**0.05**||0.25|
|**0.10**||**0.25**||0.35|
|0.46||0.54||1|

Now **Marginal probability** distribution of the given data is containted by its two columns is :


```python
# Marginal Probability
m=pd.DataFrame()
m=ndf.loc["Total",['Male','Female']]
m
```




    Male      0.46
    Female    0.54
    Name: Total, dtype: float64




```python
# Marginal Probability
ndf.loc[['Indian','Italian','Other'],"Total"]
```




    Indian     0.40
    Italian    0.25
    Other      0.35
    Name: Total, dtype: float64



|Male||Female||Total|
|----||------||-----|
|0.16||0.24||0.4|
|0.20||0.05||0.25|
|0.10||0.25||0.35|
|**0.46**||**0.54**||1|

|Male||Female||Total|
|----||------||-----|
|0.16||0.24||**0.4**|
|0.20||0.05||**0.25**|
|0.10||0.25||**0.35**|
|0.46||0.54||1|

Now we find the **Conditional Probability**.
Here, like we need to find Conditional Probability of Indian food for Females.This can be found out by dividing probability of Indian Food for Probability of Females by Total Probability for Females i.e. **0.24/0.54**. In the same way we can find for Italian and Other also.


```python
cond=pd.DataFrame()
cond=ndf.loc[:,'Female']
cond
```




    Indian     0.24
    Italian    0.05
    Other      0.25
    Total      0.54
    Name: Female, dtype: float64




```python
# Conditional Probability
cond.div(cond.loc['Total'])
```




    Indian     0.444444
    Italian    0.092593
    Other      0.462963
    Total      1.000000
    Name: Female, dtype: float64



|Female||Female||Total|
|------||------||-----|
|0.24||**0.444**||0.4|
|0.05||**0.093**||0.25|
|0.25||**0.463**||0.35|
|0.54||1||1|

Similarly, we can find for Males also.

**Example 2.** Two balls are selected at random from a box containing three red, two green and four white. If X and Y are the number of red balls and green balls respectively included among two balls drawn from the box , find 

1. Joint Probabilty of X and Y
2. Marginal Probabilty of X and Y
3. Conditional distribution of X given Y=1

As in the above example we were given the table so it was easy for us to find Joint Probability Distribution . In this example we will be making the Joint Probability Distribution.

The Joint Probability Distribution Table looks like this :


|X\Y||0||1||2||Total|
|---||-||-||-||-----|
|0|
|1|
|2|
|3|
|Total|

Since, there are three red balls so we will be taking X=0,1,2,3 and two green balls so we will take Y=0,1,2. Now we take out probabilities . We know that the probability of choosing two balls out of 3 red, 2 green and 4 white balls is given by choosing any two balls out of 9 balls i.e. $9 \choose 2$ .

Now , the block with row and column 0 represents P(0,0) means here we have 0 red and 0 green balls . It means we have only white balls with us so we can choose two balls from 4 white balls by $4\choose2$. 

So, P(0,0)= $\frac{4\choose2}{9 \choose 2} \ =0.17$  

Now,P(0,1) represents we can choose 0 red balls and 1 green ball. We know that we need to choose two balls in total according to the question so we will choose 1 ball from 2 green balls i.e. $2\choose1$ and 1 ball from 4 white balls i.e. $4\choose1$ and total probability of choosing 2 balls is $9\choose2$. Hence,

P(0,1) = ${2\choose1}\frac{4\choose1}{9\choose2} =0.22$

Similarly, if we look at P(0,2) so we can choose 2 green balls and we need to choose 2 balls in total. This can be done by choosing 2 balls from green itself and we need not take white balls here. This is done by$2\choose2$.

P(0,2)=$\frac{2\choose2}{9\choose2} = 0.03$

Now, we look at P(1,2). This means we can choose 1 red and 2 green balls but according to the question we have to choose only 2 balls so this is not possible. Hence, 

P(1,2) = 0  

Similarly, we can take out probabilities for all and this will complete our Joint Probability Distribution Table.


```python
# Joint Probability Distribution
df = pd.DataFrame({
             'X\Y' : [0, 1, 2, 3, 'Total'],
                 '0' : [ 1/6, 1/3, 1/12, 0, 21/36],
                 '1' : [2/9, 1/6, 0, 0, 14/36],
                 '2' :  [1/36, 0, 0, 0, 1/36],
           'Total' : [15/36, 18/36, 3/36, 0, 1],
                  }
    ,)
df=df.round(2)
         
df.set_index('X\Y')
print (df)

```

         X\Y     0     1     2  Total
    0      0  0.17  0.22  0.03   0.42
    1      1  0.33  0.17  0.00   0.50
    2      2  0.08  0.00  0.00   0.08
    3      3  0.00  0.00  0.00   0.00
    4  Total  0.58  0.39  0.03   1.00


|X\Y||0||1||2||Total|
|---||-||-||-||-----|
|0||0.17||0.22||0.03||0.42|
|1||0.33||0.17||0.00||0.50|
|2||0.08||0.00||0.00||0.08|
|3||0.00||0.00||0.00||0.00|
|Total||0.58||0.39||0.03||1.00|

Now, we have Joint Probability Distribution so we can easily find Marginal and Conditional Probability as seen in the previous example. You may find Marginal Probability of X and Y by yourself first and then check your answer here. Please choose an option   :  
1. Marginal Probability  
   To check Marginal Probability of X type x  
   To check Marginal Probability of Y type y  
  
2. Conditional Probability


```python
# Marginal Probability Distribution of X
mrx = pd.DataFrame()
mrx = df.loc[0:3,'Total']
mrx
```




    0    0.42
    1    0.50
    2    0.08
    3    0.00
    Name: Total, dtype: float64




```python
# Marginal Probability Distribution of Y
mry = pd.DataFrame()
mry = df.loc[4,['0','1','2']] 
mry
```




    0    0.58
    1    0.39
    2    0.03
    Name: 4, dtype: object




```python
# Conditional Probability given Y=1
cond = pd.DataFrame()
cond =df.loc[0:3,'1'].div(df.loc[4,'1']).round(2)
cond
```




    0    0.56
    1    0.44
    2    0.00
    3    0.00
    Name: 1, dtype: float64




```python
# Marginal Probability Distribution   
a = int(input("Enter a number: "))
if a == 1:
    b = input("Enter a x\y: ")
    if b == "x":
        print(mrx)
    elif b == 'y':
        print(mry)
elif a ==2:
    print('Conditional Probability when Y=1')
    print(cond)
else:
    print('Invalid input')
```

    Enter a number: 1
    Enter a x\y: x
    0    0.42
    1    0.50
    2    0.08
    3    0.00
    Name: Total, dtype: float64


# Joint Probability Density Function For Continuous Random Variable  

For a **Continuous Random Variable** (X,Y) we define the ***Joint Probability Density Function*** f(x,y) , as
1. $f(x,y)\ge 0$  

2. $ \int_{-\infty}^{\infty}\int_{-\infty}^{\infty} f(x,y)dxdy = 1 \rightarrow$ **This states that total volume under the surface given by f(x,y) is 1.0**
  

 * The Joint pdf f(x,y) is not a probability.
 * For small$ \Delta x,\Delta y(+ve)$ , f(x,y) $\Delta x \Delta y$ is approximately equal to $P[x\leq X \leq x +\Delta x,y\leq Y \leq y +\Delta y] $

**Joint Cumulative Distribution Function (X,Y)**  

The Joint Cumulative Distribution Function F(x,y) of two dimensional random variable x,y is defined as  
$$ F(x,y)\ = P[X\leq x,\ Y\leq y] $$
$$ \\\\\\\\ = \int_{-\infty}^y\int_{-\infty}^x f(x,y)dxdy $$

It follows from the definition ,that  

* F($-\infty,\infty$) = 1.0  

* F($-\infty$,y) = F(x,$-\infty$) = 0

**Example** Flows in two adjacent streams are denoted as random vector (X,Y) with joint pdf 

$$f(x,y) = c\ if\ 5\leq x\leq 10;\ 4\leq y\leq 9 $$ 
$$= 0 , \ elsewhere$$  

1. Obtain c
2. Obtain P[X $\geq$ Y]  

**Ans** To determine **c** we use the definition of joint pdf , $ \int_{-\infty}^\infty\int_{-\infty}^\infty f(x,y)dxdy = 1$ 

$ \int_{4}^9\int_{5}^{10} cf(x,y)dxdy = 1$ 


```python
# Integrating the function

import numpy as np
import matplotlib.pyplot as plt
from sympy import sin, cos, symbols, integrate
x, y = symbols('x y')
integrate(1, (x, 4, 9), (y, 5, 10))
from scipy.integrate import nquad

def f(x, y):
    return 1

def bounds_y():
    return [5, 10]

def bounds_x(y):
    return [4, 9]

y, err = nquad(f, [bounds_x, bounds_y])
y
```




    25.0



So, the value of integration is 25. The equation now becomes *25c = 1* 


```python
# value of c
c = 1/25
c
```




    0.04



**Ans 2** For P[X $\geq$ Y]  
P[X $\geq$ Y] = 1-P[X $\leq$ Y]   


```python
# Plotting graph to visualise the area or region for finding probability

%matplotlib inline
import matplotlib.pyplot as plt
plt.style.use('seaborn-whitegrid')
import numpy as np  

a = plt.axhline(y=4,color='green',linestyle='dashed')
b = plt.axhline(y=9,color='red',linestyle='dashed')
c = plt.axvline(x=5,color='black',linestyle='dashed')
d =plt.axvline(x=10,color='brown',linestyle='dashed')

x = np.linspace(5,10,100)
y = x
y1 = 9

plt.title('Visualising the graph for finding Probability')
plt.plot(x,y,linestyle='solid')
plt.fill_between(x,y,y1,where=y1>y, facecolor='gold')
```

The dashed lines represent the limits or boundaries within which the function is positive. The shaded represents the area where x$\leq$y . So, we can focus on the shaded region and find the probability. We see that x varies from 5 to y and y varies from 5 to 9 and these become the limits for x and y.

We had,  

P[X $\geq$ Y] = 1-P[X $\leq$ Y]   

=1-$ \int_{5}^9\int_{5}^y f(x,y)dxdy $  
=1-$\frac{1}{25} \int_{5}^9\int_{5}^y f(x,y)dxdy $


```python
# Integrating the function

import numpy as np
import matplotlib.pyplot as plt
from sympy import sin, cos, symbols, integrate
x, y = symbols('x y')
integrate(1, (x, 5, 9), (y, 5, y))
from scipy.integrate import nquad

def f(x, y):
    return 1

def bounds_y():
    return [5, 9]

def bounds_x(y):
    return [5, y]

y, err = nquad(f, [bounds_x, bounds_y])
y
```




    8.0



So, we have 1-{$\frac{1}{25}y$} after integration where y is the value of integration


```python
result = 1-1/25*y
result = round(result,2)
result
```




    0.68



Hence, P[X $\geq$ Y] = 0.68

So, before moving onto Marginal Density Functions for Continuous Random Variables we can have a little glance on **Monte Carlo Integration** which is used for fast integration of functions.

# Monte Carlo Integration

Monte Carlo methods are numerical techniques which rely on random sampling to approximate their results . Monte Carlo Integration applies this process to numerical estimation of integrals.Monte Carlo integration is a technique for numerical integration using random numbers. It is a particular Monte Carlo method that numerically computes a definite integral.Monte Carlo randomly chooses points at which the integrand is evaluated. This method is particularly useful for higher-dimensional integrals.  

So, in simple words it can be said that Monte Carlo Integration is used for definite integration .

# How is Monte Carlo Integration Done ?

Here we will look at the method of doing Monte Carlo .  

In order to integrate a function over a complicated domain D, Monte Carlo integration picks random points over some simple domain $D^{'}$ which is a superset of D, checks whether each point is within D, and estimates the area of D (volume, n-dimensional content, etc.) as the area of $D^{'}$ multiplied by the fraction of points falling within D.  

Now, we pick up N randomly distributed points $x_{1},x_{2},x_{3},......x_{N},$ in multidimensional volume V to determine the integral of function f in this volume gives a result  

$\int_ f dv \approx V<f>\pm V \sqrt \frac{<f^{2}>-<f>^{2}}{N}$  

where, 

$<f> \equiv \frac{1}{N}\sum_{i=1}^{N}f(x_{i})$

$<f^{2}>\equiv \frac{1}{N}\sum_{i=1}^{N}f^{2}(x_{i})$  

Now , we will look at the implementation of Monte Carlo . We take $\sin x$ as the function for integration from 0 to $\pi$ .


```python
# Monte Carlo Integration 

from scipy import random 
import numpy as np

#Limits of integration 
a = 0
b = np.pi

# No of random variables
N=1000

# Setting up random variable
xrand = random.uniform(a,b,N)

# Implementing the given function of integration
def func(x):
    return np.sin(x)
integral =0.0 

for i in range(N):
    integral += func(xrand[i])

# Final answer of integration
answer = (b-a)/float(N)*integral
print('The integral from o to pi of sin(x):' ,answer)
```

    The integral from o to pi of sin(x): 2.0178989749080714



```python
# Visualising the answer using Monte Carlo Integration

from scipy import random 
import numpy as np
import matplotlib.pyplot as plt

#Limits of integration 
a = 0
b = np.pi

# No of random variables
N=1000

areas =[]
for i in range(N):
    
# Setting up random variable
    xrand = random.uniform(a,b,N)

# Implementing the given function of integration
    def func(x):
        return np.sin(x)
    integral =0.0 

    for i in range(N):
        integral += func(xrand[i])

# Final answer of integration
    answer = (b-a)/float(N)*integral
    areas.append(answer)

plt.title('Distribution of Areas Calculated')
plt.hist(areas,bins=30, ec = 'black')
plt.xlabel('Areas')
```

So, we can see that we are getting approximately equivalent value using **Monte Carlo Integration**.

# Marginal Density Functions for Continuous Random Variable

* In continuous case, we proceed as follows:  
- Let f(x,y) denote the joint pdf of (X,Y)  
- We define g(x) and h(y) as marginal probability density functions of X and Y respectively as  
$$g(x) = \int_{-\infty}^\infty f(x,y)dy , h(x) =\int_{-\infty}^\infty f(x,y)dx$$  
These marginal pdfs which are infact derived from joint pdf f(x,y) correspond to original pdf of the one dimensional r.v.s X and Y  

This may be seen from   
P[c$\leq X\leq d$] = P[c$\leq X\leq d,-\infty\leq Y\leq \infty$]  
 =$\int_{c}^d\underbrace{\int_{-\infty}^\infty f(x,y)dy}dx $  
 =$\int_{c}^d g(x) dx$  
 From the definitions of pdf it is thus seen that this is infact original pdf of r.v. X  
 Thus ,$$g(x) = \int_{-\infty}^\infty f(x,y)dy\ and \ F(x) =\int_{-\infty}^\infty g(x,y)dx$$  
 Similarly, for r.v. of Y  
 That is, starting with joint pdf f(x,y), we are not able to get pdfs of X and Y respectively.    
 
 Lets take an example.  
**Example** Consider a joint pdf :  
f(x,y) =$\frac{1}{25}\\\\\\\\$  {$5\leq X\leq 10 $; $4 \leq Y\leq 9$}    
= 0, elsewhere  

1. Obtain the marginal density g(x), h(y)
2. Obtain CDF G(x), H(y)  
3. P[$x\geq 7$]  
4. P[$5\leq Y \leq 8$]  

**Ans** To obtain g(x),  
g(x) = $\int_{-\infty}^\infty f(x,y)dy$ , $5\leq X \leq 10$  
$\hspace {0.5cm}$ =$\int_{4}^9 \frac{1}{25}dy$



```python
# Using Monte Carlo for integration  
from scipy import random 
import numpy as np

#Limits of integration 
a = 4
b = 9

# No of random variables
N=1000

# Setting up random variable
xrand = random.uniform(a,b,N)

# Implementing the given function of integration
def func(x):
    return 1/25
integral =0.0 

for i in range(N):
    integral += func(xrand[i])

# Final answer of integration
answer = (b-a)/float(N)*integral
answer=round(answer,2)
answer
```




    0.2



Hence, g(x)= answer
g(x) = 5  

**ii)** To obtain h(y)  
h(y) = $\int_{-\infty}^{\infty} f(x,y)dx \hspace{0.5cm} 4 \leq Y\leq 9$  
= $\int_{5}^{10} \frac{1}{25}dx$


```python
# Using Monte Carlo for integration  
from scipy import random 
import numpy as np

#Limits of integration 
a = 5
b = 10

# No of random variables
N=1000

# Setting up random variable
xrand = random.uniform(a,b,N)

# Implementing the given function of integration
def func(x):
    return 1/25
integral =0.0 

for i in range(N):
    integral += func(xrand[i])

# Final answer of integration
answer = (b-a)/float(N)*integral
answer=round(answer,2)
answer
```




    0.2



Similarly , h(y) = answer  
h(y) = 0.2  $ \\\\$ $4 \leq Y\leq 9$  

**2.i.)** To obtain G(x)  
G(x) = $\int_{-\infty}^x g(x)dx$ $ \hspace{0.5cm} 5\leq x \leq 10$  
= $\int_{5}^x \frac{1}{5}dx$  
G(x) = $\frac{x-5}{5}  \hspace{0.5cm} 5\leq x \leq 10$  

**ii.)** To obtain H(y) ,  
H(y) = $\int_{-\infty}^y h(y)dy$ $ \hspace{0.5cm} 4\leq x \leq 9$   
= $\int_{4}^y \frac{1}{5}dy$  
H(y) = $\frac{y-4}{5}  \hspace{0.5cm} 4\leq x \leq 9$ 

**3.** P[$x\geq 7$] = 1- P[$x\leq 7$]  
$\hspace{2cm} = 1 - G(7) $   
$\hspace{2cm} = 1 -\frac{7-5}{5} = \frac{3}{5}$  

**4.** P[$5\leq Y \leq 8$]  = H(8) - H(5)  
$\hspace{2.7cm} = \frac{4}{5} - \frac {1}{5} = \frac{1}{5}$

# Conditional Distribution for Continuous variables  

Definition : (X,Y) is a continuous two dimensional r.v. with joint pdf of f(x,y)  
* Let g(x) and h(x) are marginal pdfs of X and Y respectively.  
* The conditional pdf of X given Y=y is defined as  
$$ g(x/y) = \frac{f(x,y)}{h(y)} \hspace{1cm}h(y)\geq 0$$  
* The conditional pdf of Y given X=x is defined as  
$$ g(y/x) = \frac{f(x,y)}{g(x)} \hspace{1cm}g(x)\geq 0$$  
* The conditional pdf of X given Y$\in $ R and conditional pdf of Y given X$\in$R is defined as  
$$ g(x/y\in R) =\frac{\int_{R}f(x,y)dy}{\int_{R}h(y)dy} \hspace{0.5cm},h(y/x\in R)= \frac{\int_{R}f(x,y)dx}{\int_{R}g(x)dx}$$  

The conditional pdfs g(x\y) and h(y\x) satisfy all the conditions for a pdf.  
For a given y, g(x/y)>0 as both f(x,y) and h(y) are positive .  
$$ \int_{-\infty}^\infty g(x/y) =\int_{-\infty}^{\infty} \frac{f(x,y)}{h(y)}dx $$  
$$= \frac{1}{h(y)}\int_{-\infty}^{\infty} f(x,y)dx$$  
$$=\frac{h(y)}{h(y)}=1.0$$   

**Cumulative Conditional Distributions**  
$$G(x/y) = \int_{-\infty}^xg(x/y)dx, H(y/x)= \int_{-\infty}^y h(y/x)dy$$  

Let us take a small example to understand this .

**Example** Consider the joint pdf 
f(x,y) = $\frac{x(1+3y^2)}{4} \hspace{1cm} 0\leq x\leq 2 \hspace{1cm}  0\leq y\leq 1$  
$\hspace{5.5cm}= 0,elsewhere$  

* Obtain h(y/x)

**Ans** To obtain h(y/x) , first we obtain g(x)  
g(x) = $\int_{-\infty}^\infty f(x,y)dy  \hspace{0.5cm} 0\leq x \leq 2  $ 

 = $\int_{0}^1\frac{x(1+3y^2)}{4}dy$  
 
 = $\frac {(x+1)}{4}$  
 
 h(y/x) =$\frac{f(x,y)}{g(x)}$
 
 = $\frac{x(1+3y^2)}{x+1}$

# Bivariate Normal Distribution

Suppose that $Z_{1} and Z_{2}$ are independent random variables, each of which has a standard normal distribution. Then the joint p.d.f. g($z_{1},z_{2}$) of $Z_{1} and Z_{2}$ is specified for all values of $z_{1} and z_{2}$ by the equation

g(z1,z2) = $\frac{1}{2\pi}exp[\frac{-1}{2}(z_{1}^2 + z_{2}^2)]$  

where

For constants $\mu_{1}$,$\mu_{2},\sigma_{1},\sigma_{2}$ and $\rho$ such that $-\infty$ < $\mu_{i}$ < $\infty$ (i = 1, 2), $\sigma_{i} > 0$ (i = 1, 2) and -1 < $\rho$ < 1, we shall now define two new random variables $X_{1} and X_{2}$ as follows:

$X_{1}$ = $\sigma_{1}$ $Z_{1}$ + $\mu_{1}$
    
$X_{2} = \sigma_{2}[\rho Z_{1} +\sqrt{1-\rho^2} Z_{2}] + \mu_{2}$

The “regular” normal distribution has one random variable; A bivariate normal distribution is made up of two independent random variables. The two variables in a bivariate normal are both are normally distributed, and they have a normal distribution when both are added together. Visually, the bivariate normal distribution is a three-dimensional bell curve.

The bivariate distribution can be described in many different ways and as such, there isn’t a unified agreement for a succinct definition. Some of the more common ways to characterize it include:

* Random variables X & Y are bivariate normal if aX + bY has a normal distribution for all a,b$\in$R.  

* X and Y are jointly normal if they can be expressed as X = aU + bV, and Y = cU + dV (Bertsekas & Tsitsiklis, 2002)  

* If a and b are non-zero constants, aX + bY has a normal distribution (Johnson & Kotz, 1972).  

* If X - aY and Y are independent and if Y - bx and X are independent for all a,b (such that ab $\neq$ 0 or 1), then (X,Y) has a   normal distribution (Rao, 1975).  

The bivariate normal distribution arises directly and naturally in many practical problems. For example, for many populations the joint distribution of two physical characteristics such as the heights and the weights of the individuals in the population will be approximately a bivariate normal distribution. For other populations, the joint distribution of the scores of the individuals in the population on two related tests will be approximately a bivariate normal distribution.

# PDF of the Bivariate Normal Distribution

The bivariate normal distribution can be defined as the probability density function (PDF) of two variables X and Y that are linear functions of the same independent normal random variables (adapted from Wolfram):

P($x_{1},x_{2}) = \frac{1}{2\pi\sigma_{1}\sigma_{2}\sqrt{1-\rho^2}}exp[-\frac{z}{2(1-\rho^2)}]$

Where:

* μ = mean
* &sigma = standard deviation  

$$ z\equiv \frac{(x_{1}-\mu_{1})^2}{\sigma_{1}^2}-\frac{2\rho(x_{1}-\mu_{1})(x_{2}-\mu_{2})}{\sigma_{1}\sigma_{2}}+\frac{(x_{1}-\mu_{2})^2}{\sigma_{2}^2}$$  

* $$ρ\equiv cor(x_{1}x_{2}) =\frac{V_{12}}{\sigma_{1}\sigma_{2}}$$  
where  
* ρ correlation of x1 and x2.
* $V_{x12} = covariance\ of x1\ and\ x2$

If $\rho$ = 2, this is equal to the bivariate normal distribution.



```python
# Displaying Bell curve (shows Bivariate normal distribution)

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import multivariate_normal
from mpl_toolkits.mplot3d import Axes3D

#Parameters to set
mu_x = 0
variance_x = 3

mu_y = 0
variance_y = 15

#Create grid and multivariate normal
x = np.linspace(-10,10,500)
y = np.linspace(-10,10,500)
X,Y = np.meshgrid(x,y)

pos = np.empty(X.shape +(2,))
pos[:, :, 0] = X; pos[:, :,1] = Y

rv = multivariate_normal([mu_x, mu_y], [[variance_x, 0], [0, variance_y]])

#Make a 3D plot
fig = plt.figure()
ax = fig.gca(projection='3d')
ax.plot_surface(X, Y, rv.pdf(pos),cmap='viridis',linewidth=0)
ax.set_xlabel = 'X-axis'
ax.set_ylabel = 'Y-axis'
ax.set_zlabel = 'Z-axis'
ax.set_title(label='Bell curve')
plt.show()
```


```python

```

79ad435ee37820eddf0bbc4e3fcae5405568a6db
ipynb
Jupyter Notebook
notebooks/JointProbabilityDistribution.ipynb
aswarth123/Statistics-and-Econometrics-for-Data-Science
f6717b6d2a197b0161a4163858fe556abcdddefd
MIT
2020-12-05T20:15:47.000Z
2022-02-07T06:00:01.000Z
notebooks/JointProbabilityDistribution.ipynb
JayantGoel001/Statistics-and-Econometrics-for-Data-Science
1bd54699d6a9f2c94fd612fcfb2a61902d43bcf4
MIT
2020-11-29T18:37:22.000Z
2022-03-06T04:03:08.000Z
notebooks/JointProbabilityDistribution.ipynb
JayantGoel001/Statistics-and-Econometrics-for-Data-Science
1bd54699d6a9f2c94fd612fcfb2a61902d43bcf4
MIT
2020-12-04T12:40:23.000Z
2021-12-16T17:29:15.000Z
Qwen/Qwen-72B
1. YES
2. YES