# Error propagation using Bayesian methods

For a Bayesian, everything is a probability distribution. This means that the way to do "error propagation" is to figure out how to combine two probability distributions to generate a third one.

Specifically, suppose that $Z=f(X,Y)$, and that we already know the pdf ${\rm pr}(X,Y|I)$. We want to compute ${\rm pr}(Z|I)$. Using marginalization introduce the variables $X$ and $Y$.


```python

```

Now use the relationship between $Z$ and $X,Y$ to rewrite ${\rm pr}(Z|X,Y)$. The result should be a formula involving the function $f$ and the joint pdf ${\rm pr}(X,Y|I)$. 


```python

```

We will call this the "master formula" for combining $X$ and $Y$.

## Example 1: $Z=X+Y$, independent Gaussians

Now write down Gaussian pdfs for $X$ and $Y$. Choose the means to be $x_0$ and $y_0$ respectively, and the standard deviations $\sigma_x$ and $\sigma_y$. 


```python

```

Put your results into the master formula, choose $f(X,Y)=X+Y$, and then combine the Gaussians. 


```python

```

Complete the square, and so obtain the result that the pdf for Z is:

\begin{equation}
{\rm pr}(Z|I)=\frac{1}{\sqrt{2 \pi} \sigma_z} \exp\left[-\frac{(Z-z_0)^2}{2 \sigma_z^2}\right].
\end{equation}
with $z_0=x_0 + y_0$ and $\sigma_z^2=\sigma_x^2 + \sigma_y^2$.

## Example 2: $Z=X+Y$, $X$ and $Y$ are correlated Gaussians 

Now re-do the analysis for the case that $X$ and $Y$ are not independent. Specifically, choose:

\begin{equation}
{\rm pr}(X,Y|I)=\frac{1}{\sqrt{2 \pi \, {\rm det} C}} \exp\left[-\frac{1}{2}(X \, \, \,Y)C^{-1}\begin{pmatrix}
X\\
Y
\end{pmatrix}\right],
\end{equation}
with 
\begin{equation}
C \equiv \begin{pmatrix} \sigma_x^2 &  \rho \sigma_x \sigma_y\\
\rho \sigma_x \sigma_y & \sigma_y^2 \end{pmatrix}.
\end{equation}

Before we get too far into combining $X$ and $Y$ you should check what happens if you set $\rho=0$ here.....does the answer make sense?


```python

```

Put the above equation for ${\rm pr}(X,Y|I)$ into the master formula for the case $f(X,Y)=X+Y$ and compute the pdf for $Z$.


```python

```

Check the limit $\rho=0$. Do you get back what you got in Example 1?


```python

```

What happens in thew limit $\rho=1$? What about $\rho=-1$? Do those results make sense?

## Example 3: $\vec{Z}=f(\vec{X},\vec{Y})$, independent multi-variate Gaussians for ${\rm pr}(\vec{X}|I)$ and ${\rm pr}(\vec{Y}|I)$.

Now take 

\begin{eqnarray}
{\rm pr}(\vec{X}|I) \propto \exp\left[-\frac{1}{2} (\vec{X} - \vec{\mu_X})^T \Sigma_X^{-1}  (\vec{X} - \vec{\mu_X})\right];\\
{\rm pr}(\vec{Y}|I) \propto \exp\left[-\frac{1}{2} (\vec{Y} - \vec{\mu_Y})^T \Sigma_Y^{-1}  (\vec{Y} - \vec{\mu_Y})\right].
\end{eqnarray} 
A short way to notate this is 

\begin{equation}
X \sim N(\vec{\mu}_X,\Sigma_X); Y \sim N(\vec{\mu}_Y,\Sigma_Y).
\end{equation}

I.e., we specify that they are given by normal distributions with particular (vector) means and covariance matrices.

Go back to the master formula and generalize it for a vector $\vec{Z}$ that is related to $\vec{X}$ and $\vec{Y}$ as $\vec{Z}=f(\vec{X},\vec{Y})$. What do you get? This should not take longer than 90 seconds. :)


```python

```

Take $\vec{Z}=a \vec{X} + b \vec{Y}$ with $a$ and $b$ real scalars. Complete the square, do the integral and show that:

\begin{equation}
\vec{Z} \sim N(a \vec{\mu}_X + b \vec{\mu}_Y, a^2 \Sigma_X + b^2 \Sigma_Y)
\end{equation}


```python

```

Comment on this result for a=b=1.


```python

```

Comment on this result for a=1, b=-1, i.e. the case $\vec{Z}=\vec{X}-\vec{Y}$.


```python

```

3cbcd503976e2696d746c7ac3d408c15e8395595
ipynb
Jupyter Notebook
topics/basics-of-bayesian-statistics/Error_Propagation.ipynb
asemposki/Bayes2019
bea9dbe5205fbf5939a154b1c3773e6c3baf39a4
CC0-1.0
2019-06-06T17:55:08.000Z
2021-11-16T08:26:26.000Z
topics/basics-of-bayesian-statistics/Error_Propagation.ipynb
asemposki/Bayes2019
bea9dbe5205fbf5939a154b1c3773e6c3baf39a4
CC0-1.0
2019-06-14T16:17:36.000Z
2019-06-15T04:41:39.000Z
topics/basics-of-bayesian-statistics/Error_Propagation.ipynb
asemposki/Bayes2019
bea9dbe5205fbf5939a154b1c3773e6c3baf39a4
CC0-1.0
2019-06-10T18:23:29.000Z
2021-12-22T15:38:30.000Z
Qwen/Qwen-72B
1. YES
2. YES