**Exercise set 4**
==============


>The goal of this exercise is to perform **least-squares regression** and to see how we can estimate errors in the parameters we find.

**Exercise 4.1**

In this exercise we will use least-squares regression to investigate a physical phenomenon: the decay of beer froth with time. The file [erdinger.txt](Data/erdinger.txt) (located at 'Data/erdinger.txt') contains [measured heights](https://doi.org/10.1088/0143-0807/23/1/304) for beer froth as a function of time, along with the errors in the measured heights.

**(a)**  Use least-squares regression to create a linear model that predicts the beer froth height as a function of time. Plot your linear model together with the raw data.


```python
# Your code here
```

**Your answer to question 4.1(a):** *Double click here*

**(b)**  Obtain the [coefficient of determination](https://en.wikipedia.org/wiki/Coefficient_of_determination), $R^2$, for your model.


```python
# Your code here
```

**Your answer to question 4.1(b):** *Double click here*

**(c)**  It is reasonable to assume that the change in the volume of froth is proportional
to the volume present at a given time. One can show that this leads
to exponential decay,

\begin{equation}
h(t) = h(0) \exp \left(-\frac{t}{\tau} \right),
\label{eq:hard}
\tag{1}\end{equation}

where $h(t)$ is the height of the froth as a function of time $t$, and $\tau$ is a parameter.
In the following, consider $h(0)$ as an unknown parameter to be determined. Show
how you can transform the equation above to a linear equation of the form,

\begin{equation}
y = a + b x,
\tag{2}\end{equation}

and give the relation(s) between the variables $h, h(0), t, \tau$ and 
$a, b, x, y$.

**Your answer to question 4.1(c):** *Double click here*

**(d)**  Using the transformation found above, create a new linear model, and estimate $h(0)$ and $\tau$.
Further, calculate the coefficient of determination for this case, and compare the two
linear models you have found so far.


```python
# Your code here
```

**Your answer to question 4.1(d):** *Double click here*

**(e)**  From the analytical model Eq. (1), $h(0)$ is a known constant, equal to the height of
the froth at time zero. Thus, we can reformulate our model and fit it to just obtain
one parameter, $b$. Essentially, we are defining $y^\prime = y - a$ and using the model,

\begin{equation}
y^\prime = y - a = b x,
\tag{3}\end{equation}

that is, we have a linear model *without* the constant term.
Show that the least-squares solution for $b$ when fitting $y^\prime = bx$ is given by,

\begin{equation}
b = \frac{
\sum_{i=1}^n y_i^\prime x_i
}{\sum_{i=1}^n x_i^2
},
\label{eq:bexpr}
\tag{4}\end{equation}

where $n$ is the number of measurements and $x_i$ and $y_i^\prime$ are the
measured values.

**Your answer to question 4.1(e):** *Double click here*

**(f)**  Do the fitting a final time, but use Eq. (4)
to obtain the parameter $b$.
Calculate the coefficient of determination and compare the three linear models you have found.


```python
# Your code here
```

**Your answer to question 4.1(f):** *Double click here*

**Exercise 4.2**

In this exercise, we will consider a linear model where we have one variable:

\begin{equation}
y = a + bx,
\end{equation}

and we have determined $a$ and $b$ using the least-squares equations. We further have
$n$ data points $(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)$ where the $x_i$'s do not have
any uncertainty, while the uncertainty in the $y_i$'s are all equal to $\sigma_y$.


> Our goal here is to find expressions for estimating
> the errors in the parameters $a$ and $b$,
> given the error in our measurements of $y$.

**Background information: Propagation of errors**

To be able to estimate the errors in $a$ and $b$, we will use [propagation of errors](https://en.wikipedia.org/wiki/Propagation_of_uncertainty).
For simplicity, consider a function, $f$, of two variables $u$ and $v$: $f = f(u, v)$.
By doing a Taylor expansion about the average values, $\bar{u}$ and $\bar{v}$, we can
show that the uncertainty (or "error") in the function $f$, $\sigma_f$, due to the uncertainties in $u$
and $v$ ($\sigma_u$ and $\sigma_v$, respectively) is given by:

\begin{equation}
\sigma_f^2 = \left(\frac{\partial f}{\partial u} \right)^2 \sigma_u^2 +
\left(\frac{\partial f}{\partial v} \right)^2 \sigma_v^2 +
2 \frac{\partial f}{\partial u} \frac{\partial f}{\partial v} \sigma_{uv} + \text{higher-order terms},
\end{equation}

where $\sigma_{uv}$ is the *covariance* between $u$ and $v$. Typically, the errors are "small"
and this motivates us to neglect the higher-order terms. Further, we will assume that the
variables $u$ and $v$ are *not* correlated: $\sigma_{uv} = 0$. We then arrive at the
(perhaps well-known) approximate propagation-of-errors-expression for the uncertainty in $f$:

\begin{equation}
\sigma_f^2 \approx \left(\frac{\partial f}{\partial u} \right)^2 \sigma_u^2 +
\left(\frac{\partial f}{\partial v} \right)^2 \sigma_v^2 .
\end{equation}

This can be generalized to $k$ variables, say $f=f(z_1, z_2, \ldots, z_k)$. The approximate
expression for the uncertainty in $f$, $\sigma_f$, due to the uncertainties
in the $z_i$'s, $\sigma_{z_{i}}$, is then:

\begin{equation}
\sigma_f^2 \approx \sum_{i=1}^{k} \left(\frac{\partial f}{\partial z_{i}} \right)^2 \sigma_{z_{i}}^2 .
\label{eq:errorp}
\tag{5}\end{equation}

We will use this expression to estimate the uncertainties in $a$ and $b$.

**Deriving expressions for the uncertainties in $a$ and $b$**

**(a)**  Show that the error in the $b$ parameter, $\sigma_b$,
is given by the following expression:

\begin{equation}
\sigma_b^2 = \frac{\sigma_y^2}{\sum_{i=1}^n \left(x_i - \bar{x}\right)^2},
\end{equation}

where $\bar{x} = \frac{1}{n} \sum_{i=1}^{n} x_i$ is the average of $x$.

***Hint:*** Use the least-squares expression for $b$:

\begin{equation}
b = \frac{
\sum_{i=1}^n (x_i - \bar{x}) (y_i - \bar{y})
}{
\sum_{i=1}^n (x_i - \bar{x})^2
},
\end{equation}

together with the propagation-of-errors expression (Eq. (5)), and consider $b$ as a
function of the $y_i$'s: $b = f(y_1, y_2, \ldots, y_n)$. You might find it helpful to determine
$\frac{\partial b}{\partial y_j}$
as an intermediate step in your derivation. 

**Your answer to question 4.2(a):** *Double click here*

**(b)**  Show that the error in the $a$ parameter, $\sigma_a$, is given by the following expression:

\begin{equation}
\sigma_a^2 = \frac{\sigma_y^2}{n} \times
\frac{
\sum_{i=1}^{n} x_i^2
}{
\sum_{i=1}^{n} (x_i - \bar{x})^2 
} .
\end{equation}

***Hint:*** Use the least-squares expression for $a$:

\begin{equation}
a = \bar{y} - b \bar{x},
\end{equation}

together with the propagation-of-errors expression (Eq. (5)), and consider $a$ as a
function of the $y_i$'s *and* $b$: $a = f(y_1, y_2, \ldots, y_n,b)$. You might find it
helpful to determine
$\frac{\partial a}{\partial y_j}$ and $\frac{\partial a}{\partial b}$ as intermediate steps
in your derivation.

**Your answer to question 4.2(b):** *Double click here*





1ced6fbc11e5278aed2932fa42d1e8ccf255be15
ipynb
Jupyter Notebook
exercises/04_Exercise_Set_4.ipynb
sroet/chemometrics
c797505d07e366319ba1544e8a602be94b88fbb6
CC-BY-4.0
2020-02-04T12:09:19.000Z
2021-08-06T12:28:24.000Z
exercises/04_Exercise_Set_4.ipynb
sroet/chemometrics
c797505d07e366319ba1544e8a602be94b88fbb6
CC-BY-4.0
2020-01-06T10:24:33.000Z
2022-01-21T10:37:46.000Z
exercises/04_Exercise_Set_4.ipynb
sroet/chemometrics
c797505d07e366319ba1544e8a602be94b88fbb6
CC-BY-4.0
2020-01-09T12:04:26.000Z
2021-01-19T10:06:14.000Z
Qwen/Qwen-72B
1. YES
2. YES