# Regression

## Linear Regression

### Step 1 : Setup the Environment


```python
# Import necessary Libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
sns.set()
```

### Step 2 : Clean the data and Visually analyse the data


```python
x = 10 * np.random.random(50)
y = 2 * x - 1 + np.random.randn(50)
print("x = ",x)
print("y = ",y)
```

    x =  [4.71575962 2.79611957 6.53365325 6.89171715 1.79343983 3.40168302
     2.75966938 6.47687307 4.66321201 7.44681371 8.62711037 4.93215967
     0.55907761 6.34443264 1.39039105 6.18648423 7.48840008 1.55065097
     8.22250359 6.35686719 8.57282616 5.26966386 2.56339095 6.60315119
     3.15469759 9.32347228 9.20683701 2.55341251 5.05858655 5.0380703
     1.17126163 1.98061133 6.10325314 2.00837555 4.8957142  7.17552295
     5.41394894 2.23837019 5.32201578 0.22963054 7.16808765 9.33971103
     9.98976715 5.21045821 5.16935013 7.52269385 9.7892179  2.47902801
     1.63778159 6.15557107]
    y =  [10.03012272  4.49529193 12.74188777 12.72551805  1.43077755  6.65275082
      4.86431874 12.11007062  7.54992829 12.89914534 16.6046464  10.44832372
     -0.85667271 11.8215409   2.96789924 11.30315529 15.32275064  2.45224635
     16.28522892 10.90016452 16.27444692 10.60256859  5.33915333 12.59200911
      5.1195451  18.68928542 18.44462835  4.80234441  9.21489562  9.60413798
      1.95439708  2.08977758 10.94412696  2.84986311  8.16031285 13.31348924
     10.20114927  2.53016296  8.73657176 -1.51279597 13.24798197 18.51094194
     20.27965885 10.72682573 11.09500694 13.20130847 18.60031984  3.51043453
      4.14270882 10.35636608]



```python
x = 10 * np.random.random(50)
y = 2 * x - 1 + np.random.randn(50)
plt.figure(dpi=120)
sns.scatterplot(x,y)
plt.xlabel("X - InDependend Variable")
plt.ylabel("Y - Dependend Variable")
plt.show()
```

### Step 3 : Generate X "Feature matrix" and y "Target vector"


```python
print(x.shape)
print(x.ndim)
print(y.shape)
```

    (50,)
    1
    (50,)



```python
# Convert the x vector to X matrix
X=x[:,np.newaxis]
print(X.shape)
```

    (50, 1)


### Step 4 : Choose a class of model

##### -> Linear Regression


```python
from sklearn.linear_model import LinearRegression
model=LinearRegression()
print(model)
```

    LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,
             normalize=False)


### Step 5 : Fit the model using the labeled data


```python
model.fit(X,y)
```




    LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,
             normalize=False)




```python
print("The slope value is ",model.coef_)
print("The Intercept value is ",model.intercept_)
```

    The slope value is  [1.96255648]
    The Intercept value is  -0.9280522283510404


### Step 6 : Predict the new data using unlabeled data


```python
y_pred=model.predict(X)
print(y_pred)
```

    [ 9.1701522  11.82289582  5.97712395  1.84536583 14.12704812 12.9763063
      2.45114674  5.62634149  3.68247813  3.87402249  8.70465804  3.48268106
     16.97372721  6.34891233  6.70186044 14.64568883  3.70685779 13.52536676
     10.14846648 15.29284059 -0.13688814  1.14230571 12.55199304 -0.76546266
     16.33990974  4.22307493  5.02121503  3.37163395 12.85077063  5.59682773
      2.08999748  7.71392691  9.33150718 11.10757381 18.47969758 18.27654415
     16.02077928 12.20984107  8.87712953  3.3043861  16.72153174 13.60792001
      7.68073427 17.67166163  4.50254112 17.55047799  6.45759014  2.68415955
     -0.55791672 16.55194399]


### Step 7 : Computer the accuracy of the model

#### Visual method of accuracy


```python
plt.figure(dpi=120)
plt.scatter(x,y,c="Red",alpha=0.6,label="Actual Value")
plt.plot(x,y_pred,linewidth=5,c="green",alpha=0.4,label="Predicted Value")
plt.xlabel("X - InDependend Variable",fontsize=18)
plt.ylabel("Y - Dependend Variable",fontsize=18)
plt.legend(loc="best",fontsize=20)
plt.show()
```

## Prediction using metric method

### Mean Absolute Error
Mean of the absolute difference between the actual value and the predicted value. 

\begin{equation}
\frac{1}{N}\sum_{i=1}^N |y_i-\hat{y_i}|
\end{equation}

>Note: 0 -> Best


```python
from sklearn.metrics import mean_absolute_error
mean_absolute_error(y,y_pred)
```




    0.7774688999128481



### Mean Squared Error
Mean of the square of the difference between the actual value and the predicted value. 

\begin{equation}
\frac{1}{N}\sum_{i=1}^N (y_i-\hat{y_i})^2
\end{equation}

>Note: 0 -> Best


```python
from sklearn.metrics import mean_squared_error
mean_squared_error(y, y_pred)
```




    0.8885217699569701



### Root Mean Squared Error
Squareroot of the average of squared difference between the predicted and estimated value

\begin{equation}
\sqrt{\frac{1}{N}\sum_{i=1}^N (y_i-\hat{y_i})^2}
\end{equation}

>Note: 0 -> Best


```python
from sklearn.metrics import mean_squared_error
np.sqrt(mean_squared_error(y,y_pred))
```




    0.9426143272606088



### R Squared
The total sum of squares (proportional to the variance of the data):

\$ SS_{tot}=\sum_{i} (y_i-\hat{y_i})^2 $

The regression sum of squares, also called the explained sum of squares:

\$ SS_{reg}=\sum_{i} (f_i-\hat{y_i})^2 $

The most general definition of the coefficient of determination is

\$ R^2 = \frac{SS_{reg}}{SS_{tot}} $

### R Squared

- Measure of the best estimate
- Coefficient of Determination

> Note: 1 -> Best


```python
from sklearn.metrics import r2_score
r2_score(y, y_pred)
```




    0.9732624820891722



### Models inbuilt score function
Calculates the R^2 values


```python
model.score(X,y)
```




    0.9732624820891722



## Input new data


```python
xnew=np.array([3,4,7])
Xnew=xnew[:,np.newaxis]
model.predict(Xnew)
```




    array([ 4.95961722,  6.92217371, 12.80984316])



# Non-linear equations


```python
x = 10 * np.random.random(100)
y = 2 * np.sin(x) - 1 + np.random.randn(100)
print("x = ",x[:11])
print("y = ",y[:11])
plt.figure(dpi=120)
sns.scatterplot(x,y)
plt.xlabel("X - InDependend Variable")
plt.ylabel("Y - Dependend Variable")
plt.show()
```

### Predict using the normal Linear Regression


```python
X=x[:,np.newaxis] 
from sklearn.linear_model import LinearRegression
model=LinearRegression()
model.fit(X,y)
xnew=np.linspace(0,10,100)
Xnew=xnew[:,np.newaxis]
y_pred=model.predict(Xnew)
```


```python
plt.figure(dpi=120,figsize=(12,7))
plt.scatter(x,y,c="Red",alpha=0.6,label="Actual Value")
plt.plot(xnew,y_pred,linewidth=5,c="green",alpha=0.4,label="Predicted Value")
plt.xlabel("X - InDependend Variable",fontsize=18)
plt.ylabel("Y - Dependend Variable",fontsize=18)
plt.legend(loc="best",fontsize=20)
plt.show()
```

### Calculate the accuracy


```python
model.score(X,y)
```




    0.001899013622799095




```python
model.score(X,y)*100
```




    0.1899013622799095



## Polynomial Regression


```python
from sklearn.preprocessing import PolynomialFeatures
```


```python
dummy=np.array([2,3,4])
poly=PolynomialFeatures(degree=4)
poly.fit_transform(dummy[:,np.newaxis])
```




    array([[  1.,   2.,   4.,   8.,  16.],
           [  1.,   3.,   9.,  27.,  81.],
           [  1.,   4.,  16.,  64., 256.]])



### Transform our actual data


```python
poly=PolynomialFeatures(degree=3)
poly_out=poly.fit_transform(X)
print(poly_out.shape)
print(poly_out[:10])
```

    (100, 4)
    [[1.00000000e+00 4.84034425e+00 2.34289324e+01 1.13404098e+02]
     [1.00000000e+00 8.24433792e+00 6.79691077e+01 5.60360292e+02]
     [1.00000000e+00 5.00138680e+00 2.50138699e+01 1.25104039e+02]
     [1.00000000e+00 2.83545135e+00 8.03978437e+00 2.27964174e+01]
     [1.00000000e+00 9.35763097e+00 8.75652573e+01 8.19403364e+02]
     [1.00000000e+00 3.61606590e+00 1.30759326e+01 4.72834340e+01]
     [1.00000000e+00 7.03069465e+00 4.94306673e+01 3.47531928e+02]
     [1.00000000e+00 2.94759886e+00 8.68833902e+00 2.56097382e+01]
     [1.00000000e+00 3.38868882e+00 1.14832119e+01 3.89130319e+01]
     [1.00000000e+00 2.90474480e-01 8.43754234e-02 2.45089072e-02]]


### Ouput of Polynomial features are fitted to normal Linear Regression


```python
model=LinearRegression()
model.fit(poly_out,y)
xnew=np.linspace(0,10,100)
Xnew=xnew[:,np.newaxis]
poly=PolynomialFeatures(degree=3)
poly_out_new=poly.fit_transform(Xnew)
y_pred=model.predict(poly_out_new)
```


```python
print("The slope value is ",model.coef_)
print("The Intercept value is ",model.intercept_)
```

    The slope value is  [ 0.         -3.04072636  0.66144071 -0.03995414]
    The Intercept value is  2.3936755999800496



```python
plt.figure(dpi=120,figsize=(15,7))
plt.scatter(x,y,c="Red",alpha=0.6,label="Actual Value")
plt.plot(xnew,y_pred,linewidth=5,c="green",alpha=0.4,label="Predicted Value")
plt.xlabel("X - InDependend Variable",fontsize=18)
plt.ylabel("Y - Dependend Variable",fontsize=18)
plt.legend(loc="best",fontsize=20)
plt.show()
```

## Accuracy score


```python
model.score(poly_out,y)*100
```




    29.76026958218042



# Automate the polynomial degree


```python
deg,acc=[],[]
for i in np.arange(2,20):
    poly=PolynomialFeatures(degree=i)
    poly_out=poly.fit_transform(X)
    model=LinearRegression()
    model.fit(poly_out,y)
    deg.append(i)
    acc.append(model.score(poly_out,y)*100)
    print("Degree :",deg[-1],"Accuracy score ->",acc[-1])
bha=pd.DataFrame({"Degree":deg,"Accuracy":acc})
```

    Degree : 2 Accuracy score -> 7.846218787887116
    Degree : 3 Accuracy score -> 29.76026958218042
    Degree : 4 Accuracy score -> 63.81832646336568
    Degree : 5 Accuracy score -> 67.1770768798496
    Degree : 6 Accuracy score -> 71.38923906135227
    Degree : 7 Accuracy score -> 71.94237240276576
    Degree : 8 Accuracy score -> 72.80588501511075
    Degree : 9 Accuracy score -> 73.10447113345143
    Degree : 10 Accuracy score -> 73.45112588076599
    Degree : 11 Accuracy score -> 73.45123595642593
    Degree : 12 Accuracy score -> 73.64777733910813
    Degree : 13 Accuracy score -> 73.73233739729207
    Degree : 14 Accuracy score -> 73.84875220899892
    Degree : 15 Accuracy score -> 73.3347529364348
    Degree : 16 Accuracy score -> 72.61308273270183
    Degree : 17 Accuracy score -> 72.44276162854572
    Degree : 18 Accuracy score -> 72.35501554325751
    Degree : 19 Accuracy score -> 71.69718613551315



```python
plt.figure(dpi=120)
sns.lineplot(data=bha,x="Degree",y="Accuracy")
plt.show()
```


```python
degr=6
poly=PolynomialFeatures(degree=degr)
poly_out=poly.fit_transform(X)
model=LinearRegression()
model.fit(poly_out,y)
xnew=np.linspace(0,10,100)
Xnew=xnew[:,np.newaxis]
poly=PolynomialFeatures(degree=degr)
poly_out_new=poly.fit_transform(Xnew)
y_pred=model.predict(poly_out_new)
print(model.score(poly_out,y)*100)
```

    71.38923906135227



```python
plt.figure(dpi=120,figsize=(15,7))
plt.scatter(x,y,c="Red",alpha=0.6,label="Actual Value")
plt.plot(xnew,y_pred,linewidth=5,c="green",alpha=0.4,label="Predicted Value")
plt.xlabel("X - InDependend Variable",fontsize=18)
plt.ylabel("Y - Dependend Variable",fontsize=18)
plt.legend(loc="best",fontsize=20)
plt.show()
```


```python
print("Model Coeffients",model.coef_)
print("Model Intercept",model.intercept_)
```

    Model Coeffients [ 0.00000000e+00  5.49235132e-01  1.35614195e+00 -1.22862598e+00
      3.12573504e-01 -3.14694607e-02  1.10639586e-03]
    Model Intercept -0.2794770369857815


# Questions ??

aca73995dcf3599260dfe94bdd06efe06309a886
ipynb
Jupyter Notebook
python/MachineLearning/Basic Linear regression.ipynb
BharathC15/NielitChennai
c817aaf63b741eb7a8e4c1df16b5038a0b4f0df7
MIT
python/MachineLearning/Basic Linear regression.ipynb
BharathC15/NielitChennai
c817aaf63b741eb7a8e4c1df16b5038a0b4f0df7
MIT
python/MachineLearning/Basic Linear regression.ipynb
BharathC15/NielitChennai
c817aaf63b741eb7a8e4c1df16b5038a0b4f0df7
MIT
2020-06-11T08:04:43.000Z
2020-06-11T08:04:43.000Z
Qwen/Qwen-72B
1. YES
2. YES