# Linear Regression Derived

#### Cost function

$ C = \sum_i (y_i - mx_i - c)^2 $

#### Calculate the partial derivative with respect to $m$

$
\begin{align}
\frac{\partial C}{\partial m} &= \sum_i 2(y_i - m x_i -c)(-x_i) \\
&= -2 \sum_i x_i (y_i - m x_i -c) \\
\end{align}
$

#### Set derivative to zero

$
\begin{align}
& \frac{\partial C}{\partial m} = 0 \\
\Rightarrow & -2 \sum_i x_i (y_i - m x_i -c) = 0 \\
\Rightarrow & \sum_i ( x_i y_i - m x_i x_i - x_i c ) = 0 \\
\Rightarrow & \sum_i x_i y_i - \sum_i m x_i x_i - \sum_i x_i c = 0 \\
\Rightarrow & \sum_i x_i y_i - c \sum_i x_i = m \sum_i x_i x_i \\
\Rightarrow & m = \frac{\sum_i x_i y_i - c \sum_i x_i}{\sum_i x_i x_i}
\end{align}
$

#### Calculate the partial derivative with respect to $c$

$
\begin{align}
\frac{\partial C}{\partial c} &= \sum_i 2(y_i - m x_i -c)(-1) \\
&= -2 \sum_i (y_i - m x_i -c) \\
\end{align}
$

#### Set the derivative to zero

$
\begin{align}
& \frac{\partial C}{\partial c} = 0 \\
\Rightarrow & -2 \sum_i (y_i - m x_i - c) = 0 \\
\Rightarrow & \sum_i (y_i - m x_i - c) = 0 \\
\Rightarrow & \sum_i y_i - \sum_i m x_i - \sum_i c = 0 \\
\Rightarrow & \sum_i y_i - m \sum_i x_i - c \sum_i 1 = 0 \\
\Rightarrow & \sum_i y_i - m \sum_i x_i = c \sum_i 1 \\
\Rightarrow & c = \frac{\sum_i y_i - m \sum_i x_i}{\sum_i 1} \\
\Rightarrow & c = \frac{\sum_i y_i}{\sum_i 1} - m \frac{\sum_i x_i}{\sum_i 1} \\
\Rightarrow & c = \bar{y} - m \bar{x} \\
\end{align}
$

#### Combine the estimates

$
\begin{align}
& m = \frac{\sum_i x_i y_i - c \sum_i x_i}{\sum_i x_i x_i} \\
& c = \bar{y} - m \bar{x} \\
& \Rightarrow m = \frac{\sum_i x_i y_i - (\bar{y} - m \bar{x}) \sum_i x_i}{\sum_i x_i x_i} \\
& \Rightarrow m = \frac{\sum_i x_i y_i - \bar{y} \sum_i x_i  + m \bar{x} \sum_i x_i}{\sum_i x_i x_i} \\
& \Rightarrow m = \frac{\sum_i x_i y_i - \bar{y} \sum_i x_i}{\sum_i x_i x_i}  + m \frac{\bar{x} \sum_i x_i}{\sum_i x_i x_i} \\
& \Rightarrow m - m \frac{\bar{x} \sum_i x_i}{\sum_i x_i x_i} = \frac{\sum_i x_i y_i - \bar{y} \sum_i x_i}{\sum_i x_i x_i} \\
& \Rightarrow m(1 - \frac{\bar{x} \sum_i x_i}{\sum_i x_i x_i}) = \frac{\sum_i x_i y_i - \bar{y} \sum_i x_i}{\sum_i x_i x_i} \\
& \Rightarrow m(\frac{\sum_i x_i x_i - \bar{x} \sum_i x_i}{\sum_i x_i x_i}) = \frac{\sum_i x_i y_i - \bar{y} \sum_i x_i}{\sum_i x_i x_i} \\
& \Rightarrow m(\sum_i x_i x_i - \bar{x} \sum_i x_i) = \sum_i x_i y_i - \bar{y} \sum_i x_i \\
& \Rightarrow m = \frac{\sum_i x_i y_i - \bar{y} \sum_i x_i}{\sum_i x_i x_i - \bar{x} \sum_i x_i} \\
\end{align}
$

#### End

744a3fb2ee56906abcfeef89b2dd4e4f7cfc5d2e
ipynb
Jupyter Notebook
linear-regression-derived.ipynb
ianmcloughlin/jupyter-teaching-notebooks
46fed19115bf2f7e63c7bcb3d78bee92295c33b6
Unlicense
2018-10-23T15:30:48.000Z
2021-10-31T20:30:47.000Z
linear-regression-derived.ipynb
ianmcloughlin/jupyter-teaching-notebooks
46fed19115bf2f7e63c7bcb3d78bee92295c33b6
Unlicense
linear-regression-derived.ipynb
ianmcloughlin/jupyter-teaching-notebooks
46fed19115bf2f7e63c7bcb3d78bee92295c33b6
Unlicense
2018-10-30T00:08:01.000Z
2021-01-08T23:33:52.000Z
Qwen/Qwen-72B
1. YES
2. YES