# The Process 
Not everytime is there a situation where $Ax = b$ has exact solutions , such is the case when we are trying to find a best fit line for a given set of datapoints , there is no direct solution for it , instead we are in search for the best possible solution.So instead of finding solutions for $Ax = b$ , we can project the $b$ vector onto the column space of $A$ so that we can get the best possible solution.The solution to the equation $A^TA\hat{X} = A^Tb$ , gives us the best possible solution that we are looking for , which can be simplified to , $\hat{X} = (A^TA)^{-1}A^Tb$ understanding all this with an example would give us a better idea.

Let us find the best possible line passing/fitting through $(1,1),(2,2) ,(3,2)$ 
taking the equation to be of the form $y = mx + c$ 

\begin{equation}
\begin{pmatrix}
1 & 1 \\
2 & 2 \\
3 & 2 
\end{pmatrix} 
\begin{pmatrix}
m \\
c
\end{pmatrix}
=
\begin{pmatrix}
1 \\
2 \\
2
\end{pmatrix}
\end{equation}
Which has no solution,so our $\hat{X}$ is given by
\begin{equation}
\begin{pmatrix}
1 & 1 \\
2 & 2 \\
3 & 2 
\end{pmatrix}^{T} 
\begin{pmatrix}
1 & 1 \\
2 & 2 \\
3 & 2 
\end{pmatrix} 
\hat{X}
=
\begin{pmatrix}
1 & 1 \\
2 & 2 \\
3 & 2 
\end{pmatrix}^{T} 
\begin{pmatrix}
1 \\
2 \\
2
\end{pmatrix}
\end{equation}
which on solving gives
\begin{equation}
\begin{pmatrix}
\hat{m} \\
\hat{c}
\
\end{pmatrix}
=
\begin{pmatrix}
\frac{1}{2} \\
\frac{2}{3}
\
\end{pmatrix}
\end{equation}
Thus the best fitting line is given by
$ y = \frac{1}{2}x+\frac{2}{3} $

This exact process is done by the below given python fxn.


```python
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
```


```python
def LeastSquares(points):
    A = []
    b = []
    #Extracting the A,b matrix from the given set of points
    for p in points:
        A.append([1,p[0]])
        b.append([p[1]])
    A = np.array(A)
    b = np.array(b)
    points = np.array(points)
    #finding the parameters m and c
    x = np.matmul(np.matmul(np.linalg.inv(np.matmul(A.T,A)),A.T),b)
    #drawing the fitting line
    inp_space = np.linspace(points[:,0].min(),points[:,0].max(),1000)
    out = (inp_space)*x[1]+x[0]
    plt.plot(inp_space,out)
    for i in points:
        plt.scatter(i[0],i[1])
    plt.title(f"Y = {x[1]}x+{x[0]}")
```


```python
A = LeastSquares([(0,0),(-1,1),(2,3),(3,4)]) # takes in list of points to fit line on.
```

# Learn more on
[MIT OCW LECTURE](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/lecture-16-projection-matrices-and-least-squares)  

[Least square Approximations](http://math.mit.edu/~gs/linearalgebra/ila0403.pdf)

5aefb101235bd1f06d103563648bdc0e8a72945d
ipynb
Jupyter Notebook
Linearreg.ipynb
B20204/Professorpy
572c735c360c19c55f7554245c65975505e5e237
MIT
Linearreg.ipynb
B20204/Professorpy
572c735c360c19c55f7554245c65975505e5e237
MIT
Linearreg.ipynb
B20204/Professorpy
572c735c360c19c55f7554245c65975505e5e237
MIT
Qwen/Qwen-72B
1. YES
2. YES