```python
%%markdown
# Overview
In a linear regression, the outcome, $y$, is approximated by a hidden function of the type $\forall x \in S, f(x) = a + bx$,
and the error function is $J(x) = E_{x\in S} (f(x) - y)^2$

For a $n$ samples of $x$:
\begin{align}
J_{a,b}(x) = \frac{1}{n} \sum_{i=1}^{i=n} (f(x_i) - y_i)^2 = \frac{1}{n} \sum_{i=1}^{i=n} (a + bx_i - y_i)^2
\end{align}

The gradient of $J(x)$ is then:
\begin{align}
\frac{\partial J(x)}{\partial a} &= \frac{2}{n} \sum_{i=1}^{i=n} (a + bx_i - y_i) \\
\frac{\partial J(x)}{\partial b} &= \frac{2}{n} \sum_{i=1}^{i=n} (a + bx_i - y_i)x_i
\end{align}

# References
## Gradient descent
* http://en.wikipedia.org/wiki/Gradient_descent
* http://en.wikipedia.org/wiki/Stochastic_gradient_descent#Example
## Formatting
* [Tex/$\LaTeX$ support in Jupyter Markdown](http://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Typesetting%20Equations.html)
```


# Overview
In a linear regression, the outcome, $y$, is approximated by a hidden function of the type $\forall x \in S, f(x) = a + bx$,
and the error function is $J(x) = E_{x\in S} (f(x) - y)^2$

For a $n$ samples of $x$:
\begin{align}
J_{a,b}(x) = \frac{1}{n} \sum_{i=1}^{i=n} (f(x_i) - y_i)^2 = \frac{1}{n} \sum_{i=1}^{i=n} (a + bx_i - y_i)^2
\end{align}

The gradient of $J(x)$ is then:
\begin{align}
\frac{\partial J(x)}{\partial a} &= \frac{2}{n} \sum_{i=1}^{i=n} (a + bx_i - y_i) \\
\frac{\partial J(x)}{\partial b} &= \frac{2}{n} \sum_{i=1}^{i=n} (a + bx_i - y_i)x_i
\end{align}

# References
## Gradient descent
* http://en.wikipedia.org/wiki/Gradient_descent
* http://en.wikipedia.org/wiki/Stochastic_gradient_descent#Example
## Formatting
* [Tex/$\LaTeX$ support in Jupyter Markdown](http://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Typesetting%20Equations.html)



```python
from mpl_toolkits import mplot3d
import numpy as np
import matplotlib.pyplot as plt

```


```python
# Error function for x = {1, 2}: 1/2 [(a+b-1)^2 + (a+2b-2)^2]
# Derivative:
#  a. dJa = (a+b-1) + (a+2b-2) = 2a + 3b - 3
#  b. dJb = (a+b-1) + 2(a+2b-2) = 3a + 5b - 5
#  a = 0, b = 0 => dJa = -3 ; dJb = -5
def error_function(a, b):
    total_error = 0.0
    n = 2
    for x in range(1, n):
        total_error += (a + b*x - x) ** 2
    
    return total_error / n

x = np.linspace(-5, 5, 30)
y = np.linspace(-5, 5, 30)

X, Y = np.meshgrid(x, y)
Z = error_function(X, Y)
```


```python
fig = plt.figure()
ax = plt.axes(projection='3d')
ax.contour3D(X, Y, Z, 50, cmap='binary')
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
```


```python

```

e71285b47f08afa1842d7407eefcd198652ed601
ipynb
Jupyter Notebook
general/error-function-for-linear-regression.ipynb
machine-learning-helpers/induction-books-python
d26816f92d4f6a64e8c4c2ed6c7c8343c77cd3ad
RSA-MD
2018-02-11T12:34:19.000Z
2021-09-22T18:06:01.000Z
general/error-function-for-linear-regression.ipynb
machine-learning-helpers/induction-books-python
d26816f92d4f6a64e8c4c2ed6c7c8343c77cd3ad
RSA-MD
2019-11-22T00:48:20.000Z
2022-01-16T11:00:50.000Z
general/error-function-for-linear-regression.ipynb
machine-learning-helpers/induction-python
631a735a155f0feb7012472fbca13efbc273dfb0
RSA-MD
Qwen/Qwen-72B
1. YES
2. YES