### Suppose we have `n` features and `m` observations

| Index        | $X_{1}$       | $X_{2}$       | $X_{3}$       | .... | .... | $X_{n}$        | y        |
|--------------|---------------|---------------|---------------|------|------|----------------|----------|
| 1            | $x_{1}^{1} $  | $x_{2}^{1}$   | $x_{3}^{1}$   | ...  | ...  | $x_{n}^{1}$    | $y^{1}$  |
| 2            | $x_{1}^{2}$   | $x_{2}^{2}$   | $x_{3}^{2}$   | ...  | ...  | $x_{n}^{2}$    | $y^{2}$  |
| 3            | $x_{1}^{3}$   | $x_{2}^{3}$   | $x_{3}^{3}$   | ...  | ...  | $x_{n}^{3}$    | $y^{3}$  |
| .            | .             | .             | .             | ...  | ...  | .              |          |
| .            | .             | .             | .             | ...  | ...  | .              |          |
| .            | .             | .             | .             | ...  | ...  | .              |          |
| m            | $x_{1}^{m}$   | $x_{2}^{m}$   | $x_{3}^{m}$   | ...  | ...  | $x_{n}^{m}$    | $y^{m}$  |

Here `subscript` denotes feature and `superscript` denote observation number

Suppose the weights of matrix for n features is denoted by column vectors of shape [1, n]
<br/>
$$ \beta = \begin{bmatrix} \beta_{1} & \beta_{2} & \beta_{3} & .... &  \beta_{n} \end{bmatrix} $$

### we need to calculate prediction for each observation

\begin{equation}
\hat{y^1} = \beta_{0} + \beta_{1}x_{1}^{1} +  \beta_{2}x_{2}^{1}  +  \beta_{3}x_{3}^{1}  +  \beta_{3}x_{3}^{1} + .... + \beta_{n}x_{n}^{1}
\end{equation}

\begin{equation}
\hat{y^2} = \beta_{0} + \beta_{1}x_{1}^{2} +  \beta_{2}x_{2}^{2}  +  \beta_{3}x_{3}^{2}  +  \beta_{3}x_{3}^{2} + .... + \beta_{n}x_{n}^{2}
\end{equation}


\begin{equation}
\hat{y^3} =  \beta_{0} +\beta_{1}x_{1}^{3} +  \beta_{2}x_{2}^{3}  +  \beta_{3}x_{3}^{3}  +  \beta_{3}x_{3}^{3} + .... + \beta_{n}x_{n}^{3}
\end{equation}

\begin{equation}
 ..................
\end{equation}

\begin{equation}
 ..................
\end{equation}

\begin{equation}
\hat{y^m} =  \beta_{0} +\beta_{1}x_{1}^{m} +  \beta_{2}x_{2}^{m}  +  \beta_{3}x_{3}^{m}  +  \beta_{3}x_{3}^{m} + .... + \beta_{n}x_{n}^{m}
\end{equation}

### In matrix form 

\begin{equation}
\begin{bmatrix} \hat{y}^{1} \\ \hat{y}^{2} \\ \hat{y}^{3} \\ .. \\.. \\  \hat{y}^{m} \end{bmatrix} = 
\begin{bmatrix} x_{1}^{1} & x_{2}^{1} & x_{3}^{1} & .... &  x_{n}^{1}
           \\ x_{1}^{2} & x_{2}^{2} & x_{3}^{2} & .... &  x_{n}^{2}
           \\ x_{1}^{3} & x_{2}^{3} & x_{3}^{3} & .... &  x_{n}^{3}
           \\.... &.... & ... &.... &....
           \\.... &.... & ... &.... &....
            \\ x_{1}^{m} & x_{2}^{m} & x_{3}^{m} & .... &  x_{n}^{m}
\end{bmatrix}
*
\begin{bmatrix} \beta_{1} \\ \beta_{2} \\ \beta_{3} \\ .. \\.. \\  \beta_{n} \end{bmatrix}
+ \beta_{0}
\end{equation}

\begin{equation} \hat{y} = X.\beta^{T} + \beta_{0} \end{equation}

#### Equivalent numpy implementation is:

 `y_hat = np.dot(X, B.T) + b`  
Where 
 $$ B = \beta $$
 $$ b = \beta_{0}$$

### The Mean Squared Error cost function is:

$$ J(\beta, \beta_{0}) = \frac{1}{2m}\sum_{n=1}^{m}   (y^{(i)} - \hat{y}^{(i)})^{2} $$

#### Equivalent numpy implementation is:

`cost = (1/(2*m))*np.sum((y-y_hat)**2)`

### Just for confirmation let us take an example

| y | $\hat{y}$ | (y - $\hat{y}$)^2 |
|---|---------|-----------------|
| 2 | 1       | 1               |
| 4 | 2       | 4               |
| 6 | 3       | 9               |
|  |        | 14               |

\begin{equation} 14/3 = 4.67 \end{equation}


```python
import numpy as np

y=np.array([[2],
           [4],
           [6]])
y_hat=np.array([[1],
           [2],
           [3]])
```


```python
np.sum((y-y_hat)**2)/3
```




    4.666666666666667



### Now lets calculate the gradient descent

### We have the loss function  defined as

$$ J(\beta, \beta_{0}) = \frac{1}{2m}\sum_{n=1}^{m}   (y - \hat{y})^{2}   $$

where: $$ \hat{y} = \beta_{0} + X*\beta^{T} $$

So:
$$ \frac {\partial J(\beta, \beta_{0})}{\partial \beta_{0}} = 
\frac{-1}{m}\sum_{n=1}^{m}(y-\hat{y}) $$

Again: $$ J(\beta, \beta_{0}) = \frac{1}{2m}\sum_{n=1}^{m}   (y - \hat{y})^{2}   $$

$$ \frac {\partial J(\beta, \beta_{0})}{\partial \beta} = \frac{-1}{m}(y - \hat{y}) * \frac{\partial (\beta_{0} + X*\beta^{T})}{\partial \beta} $$

$$ \frac {\partial J(\beta, \beta_{0})}{\partial \beta} = 
\frac{-1}{m}(y-\hat{y})*X * \frac{\partial \beta^{T}}{\partial \beta} $$

where $$ \beta = \begin{bmatrix} \beta_{1} & \beta_{2} & \beta_{3} &.... & \beta_{n} \end{bmatrix}$$ 

and

$$ \beta^{T} = \begin{bmatrix} \beta_{1} \\ \beta_{2} \\ \beta_{3} \\.... \\ \beta_{n} \end{bmatrix}$$ 


Since: $$ \frac{\partial \beta^{T}}{\partial \beta} = I $$

### Just a side note for calculation of $$ \frac{\partial \beta^{T}}{\partial \beta} $$
where $$ \beta = \begin{bmatrix} \beta_{1} & \beta_{2} & \beta_{3} &.... & \beta_{n} \end{bmatrix}$$ 

and

$$ \beta^{T} = \begin{bmatrix} \beta_{1} \\ \beta_{2} \\ \beta_{3} \\.... \\ \beta_{n} \end{bmatrix}$$ 

$$ \frac{\partial \beta^{T}}{\partial \beta} = \begin{bmatrix} \frac{\partial \beta^{T}}{\partial \beta_{1}} & \frac{\partial \beta^{T}}{\partial \beta_{2}} & \frac{\partial \beta^{T}}{\partial \beta_{3}} & ....  & \frac{\partial \beta^{T}}{\partial \beta_{n}}\end{bmatrix} $$



\begin{equation} 
\frac{\partial \beta^{T}}{\partial \beta} = 
\begin{bmatrix} 
\frac{\partial \beta_{1}}{\partial \beta_{1}} & \frac{\partial \beta_{1}}{\partial \beta_{2}} & \frac{\partial \beta_{1}}{\partial \beta_{3}} & ..&..& \frac{\partial \beta_{1}}{\partial \beta_{n}}  
\\  
\frac{\partial \beta_{2}}{\partial \beta_{1}} & \frac{\partial \beta_{2}}{\partial \beta_{2}} & \frac{\partial \beta_{2}}{\partial \beta_{3}} &..&..& \frac{\partial \beta_{2}}{\partial \beta_{n}} 
\\ 
\frac{\partial \beta_{2}}{\partial \beta_{1}} & \frac{\partial \beta_{3}}{\partial \beta_{2}} & \frac{\partial \beta_{3}}{\partial \beta_{3}} &..&..& \frac{\partial \beta_{3}}{\partial \beta_{n}} 
\\ 
.. & .. & .. & ..  & .. &..
\\
.. & .. & .. & ..  & .. &..
\\
\frac{\partial \beta_{n}}{\partial \beta_{1}} & \frac{\partial \beta_{n}}{\partial \beta_{2}} & \frac{\partial \beta_{n}}{\partial \beta_{3}} & ..&..& \frac{\partial \beta_{n}}{\partial \beta_{n}}  
\end{bmatrix}
\end{equation}


\begin{equation} 
\frac{\partial \beta^{T}}{\partial \beta} = 
\begin{bmatrix} 
1 & 0 & 0 & ..&..& 0  
\\  
0 & 1 & 0 &..&..& 0 
\\ 
0 & 0 & 1 &..&..& 0 
\\ 
.. & .. & .. & ..  & .. &..
\\
.. & .. & .. & ..  & .. &..
\\
0 & 0 & 0 & ..&..& 1  
\end{bmatrix} 
\end{equation}


i.e $$  \frac{\partial \beta^{T}}{\partial \beta} = I_{n*n} $$

Since: $$ X_{m*n} * I_{n*n} = X $$

So: 
 $$ \frac {\partial J(\beta, \beta_{0})}{\partial \beta} = 
\frac{-1}{m}(y-\hat{y})*X  $$

### Shape of $$ (y-\hat{y}) $$  `m rows and 1 cols [m, 1]`

### Shape of `X is [m, n] ie m observations and n features`

### Required shape of `dB is [1, n] `

### [1, m] * [m, n] == [1, n]

## Equivalent Numpy Implementation is:

 `dB = (-1/m)* np.dot((y-y_hat).T, X)`
 <br/>
  `db = (-1/m)*np.sum(y-y_hat)`


```python
def propagate(B, b, X, Y):
    """
    params:
    B: weights of size [1, X.shape[1]]
    b: bias
    X: matrix of observations and features size [X.shape[0], X.shape[1]]
    Y: matrix of actual observation size [Y.shape[0], 1]
    
    returns:
    grads: dict of gradients, dB of shape same as B and db of shape [1, 1].
    cost: MSE cost
    """
    
    ## m is no of observations ie rows of X
    m = X.shape[0]
    
    #Calculate hypothesis
    y_hat = np.dot(X, B.T) + b
    
    y = Y.values.reshape(Y.shape[0],1)
    
    #Compute Cost
    cost = (1/(2*m))*np.sum((y-y_hat)**2)
    
    # BACKWARD PROPAGATION (TO FIND GRAD)
    dB = (-1/m)* np.dot((y-y_hat).T, X)
    
    db = -np.sum(y-y_hat)/m
    
    grads = {"dB": dB,
             "db": db}
    
    return grads, cost
```


```python
def optimize(B, b, X, Y, num_iterations, learning_rate):
    """
    params:
    B: weights of size [1, X.shape[1]]
    b: bias
    X: matrix of observations and features size [X.shape[0], X.shape[1]]
    Y: matrix of actual observation size [Y.shape[0], 1]
    num_iterations: number of iterations
    learning_rate: learning rate
    returns:
    params: parameters B of shape [1, X.shape[1]] and bias
    grads: dict of gradients, dB of shape same as B and db
    costs:  MSE cost 
    """
    costs = []
    
    for i in range(num_iterations):
        
        
        # Cost and gradient calculation call function propagate
        grads, cost = propagate(B,b,X,Y)
        
        # Retrieve derivatives from grads
        dB = grads["dB"]
        db = grads["db"]
        
        # update parameters
        B = B - learning_rate * dB
        b = b - learning_rate * db
        
        costs.append(cost)
    
    params = {"B": B,
              "b": b}
    
    grads = {"dB": dB,
             "db": db}
    
    return params, grads, costs
```


```python
def predict(B, b, X):
    """:param
    B: weights
    b: bias
    X: matrix of observations and features
    """
  # Compute predictions for X
    Y_prediction = np.dot(X, B.T) + b
    return Y_prediction
```


```python
def model(X_train, Y_train, X_test, Y_test, num_iterations = 2000, learning_rate = 0.5):
    """
    params: 
    X_train: X_train
    Y_train: Y_train
    X_test: X_test
    Y_test: Y_test
    
    returns:
    d: dictionary
    """
    
    
    # initialize parameters with zeros 
    B = np.zeros(shape=(1, X_train.shape[1]))
    b = 0
    
    # Gradient descent
    parameters, grads, costs = optimize(B, b, X_train, Y_train, num_iterations, learning_rate)
    
    # Retrieve parameters w and b from dictionary "parameters"
    B = parameters["B"]
    b = parameters["b"]
    
    # Predict test/train set examples
    Y_prediction_test = predict(B, b, X_test)
    Y_prediction_train = predict(B, b, X_train)
    
    Y_train = Y_train.values.reshape(Y_train.shape[0], 1)
    Y_test = Y_test.values.reshape(Y_test.shape[0], 1)

   # Print train/test Errors
    print("train accuracy: {} %".format(100 - np.mean(np.abs(Y_prediction_train - Y_train)) * 100))
    print("test accuracy: {} %".format(100 - np.mean(np.abs(Y_prediction_test - Y_test)) * 100))

    
    d = {"costs": costs,
         "Y_prediction_test": Y_prediction_test, 
         "Y_prediction_train" : Y_prediction_train, 
         "B" : B, 
         "b" : b,
         "learning_rate" : learning_rate,
         "num_iterations": num_iterations}
    
    return d
```


```python
import pandas as pd
df = pd.read_csv('../datasets/USA_Housing.csv')
```


```python
df
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Avg. Area Income</th>
      <th>Avg. Area House Age</th>
      <th>Avg. Area Number of Rooms</th>
      <th>Avg. Area Number of Bedrooms</th>
      <th>Area Population</th>
      <th>Price</th>
      <th>Address</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>79545.458574</td>
      <td>5.682861</td>
      <td>7.009188</td>
      <td>4.09</td>
      <td>23086.800503</td>
      <td>1.059034e+06</td>
      <td>208 Michael Ferry Apt. 674\nLaurabury, NE 3701...</td>
    </tr>
    <tr>
      <th>1</th>
      <td>79248.642455</td>
      <td>6.002900</td>
      <td>6.730821</td>
      <td>3.09</td>
      <td>40173.072174</td>
      <td>1.505891e+06</td>
      <td>188 Johnson Views Suite 079\nLake Kathleen, CA...</td>
    </tr>
    <tr>
      <th>2</th>
      <td>61287.067179</td>
      <td>5.865890</td>
      <td>8.512727</td>
      <td>5.13</td>
      <td>36882.159400</td>
      <td>1.058988e+06</td>
      <td>9127 Elizabeth Stravenue\nDanieltown, WI 06482...</td>
    </tr>
    <tr>
      <th>3</th>
      <td>63345.240046</td>
      <td>7.188236</td>
      <td>5.586729</td>
      <td>3.26</td>
      <td>34310.242831</td>
      <td>1.260617e+06</td>
      <td>USS Barnett\nFPO AP 44820</td>
    </tr>
    <tr>
      <th>4</th>
      <td>59982.197226</td>
      <td>5.040555</td>
      <td>7.839388</td>
      <td>4.23</td>
      <td>26354.109472</td>
      <td>6.309435e+05</td>
      <td>USNS Raymond\nFPO AE 09386</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>4995</th>
      <td>60567.944140</td>
      <td>7.830362</td>
      <td>6.137356</td>
      <td>3.46</td>
      <td>22837.361035</td>
      <td>1.060194e+06</td>
      <td>USNS Williams\nFPO AP 30153-7653</td>
    </tr>
    <tr>
      <th>4996</th>
      <td>78491.275435</td>
      <td>6.999135</td>
      <td>6.576763</td>
      <td>4.02</td>
      <td>25616.115489</td>
      <td>1.482618e+06</td>
      <td>PSC 9258, Box 8489\nAPO AA 42991-3352</td>
    </tr>
    <tr>
      <th>4997</th>
      <td>63390.686886</td>
      <td>7.250591</td>
      <td>4.805081</td>
      <td>2.13</td>
      <td>33266.145490</td>
      <td>1.030730e+06</td>
      <td>4215 Tracy Garden Suite 076\nJoshualand, VA 01...</td>
    </tr>
    <tr>
      <th>4998</th>
      <td>68001.331235</td>
      <td>5.534388</td>
      <td>7.130144</td>
      <td>5.44</td>
      <td>42625.620156</td>
      <td>1.198657e+06</td>
      <td>USS Wallace\nFPO AE 73316</td>
    </tr>
    <tr>
      <th>4999</th>
      <td>65510.581804</td>
      <td>5.992305</td>
      <td>6.792336</td>
      <td>4.07</td>
      <td>46501.283803</td>
      <td>1.298950e+06</td>
      <td>37778 George Ridges Apt. 509\nEast Holly, NV 2...</td>
    </tr>
  </tbody>
</table>
<p>5000 rows × 7 columns</p>
</div>




```python
df.drop(['Address'],axis=1,inplace=True)
```


```python
df.head()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Avg. Area Income</th>
      <th>Avg. Area House Age</th>
      <th>Avg. Area Number of Rooms</th>
      <th>Avg. Area Number of Bedrooms</th>
      <th>Area Population</th>
      <th>Price</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>79545.458574</td>
      <td>5.682861</td>
      <td>7.009188</td>
      <td>4.09</td>
      <td>23086.800503</td>
      <td>1.059034e+06</td>
    </tr>
    <tr>
      <th>1</th>
      <td>79248.642455</td>
      <td>6.002900</td>
      <td>6.730821</td>
      <td>3.09</td>
      <td>40173.072174</td>
      <td>1.505891e+06</td>
    </tr>
    <tr>
      <th>2</th>
      <td>61287.067179</td>
      <td>5.865890</td>
      <td>8.512727</td>
      <td>5.13</td>
      <td>36882.159400</td>
      <td>1.058988e+06</td>
    </tr>
    <tr>
      <th>3</th>
      <td>63345.240046</td>
      <td>7.188236</td>
      <td>5.586729</td>
      <td>3.26</td>
      <td>34310.242831</td>
      <td>1.260617e+06</td>
    </tr>
    <tr>
      <th>4</th>
      <td>59982.197226</td>
      <td>5.040555</td>
      <td>7.839388</td>
      <td>4.23</td>
      <td>26354.109472</td>
      <td>6.309435e+05</td>
    </tr>
  </tbody>
</table>
</div>




```python
df.info()
```

    <class 'pandas.core.frame.DataFrame'>
    RangeIndex: 5000 entries, 0 to 4999
    Data columns (total 6 columns):
    Avg. Area Income                5000 non-null float64
    Avg. Area House Age             5000 non-null float64
    Avg. Area Number of Rooms       5000 non-null float64
    Avg. Area Number of Bedrooms    5000 non-null float64
    Area Population                 5000 non-null float64
    Price                           5000 non-null float64
    dtypes: float64(6)
    memory usage: 234.5 KB



```python
#normalization of column values
df_norm = (df - df.mean()) / (df.max() - df.min())

# Putting feature variable to X
X = df_norm[['Avg. Area Income','Avg. Area House Age','Avg. Area Number of Rooms','Avg. Area Number of Bedrooms','Area Population']]

# Putting response variable to y
y = df_norm['Price']

#random_state is the seed used by the random number generator, it can be any integer.
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.7 ,test_size = 0.3, random_state=2)


```


```python
df_norm.head()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Avg. Area Income</th>
      <th>Avg. Area House Age</th>
      <th>Avg. Area Number of Rooms</th>
      <th>Avg. Area Number of Bedrooms</th>
      <th>Area Population</th>
      <th>Price</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0.121932</td>
      <td>-0.042817</td>
      <td>0.002844</td>
      <td>0.024149</td>
      <td>-0.188292</td>
      <td>-0.070538</td>
    </tr>
    <tr>
      <th>1</th>
      <td>0.118631</td>
      <td>0.003735</td>
      <td>-0.034156</td>
      <td>-0.198073</td>
      <td>0.057734</td>
      <td>0.111620</td>
    </tr>
    <tr>
      <th>2</th>
      <td>-0.081153</td>
      <td>-0.016194</td>
      <td>0.202692</td>
      <td>0.255260</td>
      <td>0.010348</td>
      <td>-0.070557</td>
    </tr>
    <tr>
      <th>3</th>
      <td>-0.058260</td>
      <td>0.176153</td>
      <td>-0.186228</td>
      <td>-0.160296</td>
      <td>-0.026685</td>
      <td>0.011636</td>
    </tr>
    <tr>
      <th>4</th>
      <td>-0.095667</td>
      <td>-0.136247</td>
      <td>0.113193</td>
      <td>0.055260</td>
      <td>-0.141246</td>
      <td>-0.245046</td>
    </tr>
  </tbody>
</table>
</div>




```python
X_train.shape
```




    (3500, 5)




```python
X_train.head()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Avg. Area Income</th>
      <th>Avg. Area House Age</th>
      <th>Avg. Area Number of Rooms</th>
      <th>Avg. Area Number of Bedrooms</th>
      <th>Area Population</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>2416</th>
      <td>0.129642</td>
      <td>-0.143456</td>
      <td>0.003923</td>
      <td>-0.169184</td>
      <td>-0.027249</td>
    </tr>
    <tr>
      <th>2417</th>
      <td>-0.094771</td>
      <td>-0.263002</td>
      <td>0.052597</td>
      <td>-0.164740</td>
      <td>0.132247</td>
    </tr>
    <tr>
      <th>2513</th>
      <td>-0.019134</td>
      <td>0.219268</td>
      <td>-0.100217</td>
      <td>-0.109184</td>
      <td>-0.300045</td>
    </tr>
    <tr>
      <th>1698</th>
      <td>-0.033811</td>
      <td>-0.295470</td>
      <td>0.058020</td>
      <td>0.533038</td>
      <td>-0.104025</td>
    </tr>
    <tr>
      <th>3322</th>
      <td>0.030541</td>
      <td>-0.015484</td>
      <td>-0.129778</td>
      <td>-0.329184</td>
      <td>-0.135710</td>
    </tr>
  </tbody>
</table>
</div>




```python
y_train.head()
```




    2416   -0.041265
    2417   -0.182448
    2513   -0.082988
    1698   -0.196576
    3322   -0.069286
    Name: Price, dtype: float64




```python
y_test.shape[0]
```




    1500




```python
model1 = model(X_train=X_train, Y_train=y_train, X_test=X_test, Y_test=y_test, num_iterations = 1000, learning_rate = 0.001)

model2 = model(X_train=X_train, Y_train=y_train, X_test=X_test, Y_test=y_test, num_iterations = 1000, learning_rate = 0.01)

model3 = model(X_train=X_train, Y_train=y_train, X_test=X_test, Y_test=y_test, num_iterations = 1000, learning_rate = 0.1)

model4 = model(X_train=X_train, Y_train=y_train, X_test=X_test, Y_test=y_test, num_iterations = 1000, learning_rate = 0.3)
```

    train accuracy: 88.62860037424024 %
    test accuracy: 88.8357850464578 %
    train accuracy: 90.16271542913721 %
    test accuracy: 90.33382459987604 %
    train accuracy: 95.97241738688837 %
    test accuracy: 96.00886188507282 %
    train accuracy: 96.6522103927996 %
    test accuracy: 96.73232357323806 %



```python
import matplotlib.pyplot as plt
plt.plot([i for i in range(1000)], model1['costs'])
plt.plot([i for i in range(1000)], model2['costs'])
plt.plot([i for i in range(1000)], model3['costs'])
plt.plot([i for i in range(1000)], model4['costs'])

plt.gca().legend(('alpha 0.001','alpha 0.01', 'alpha 0.1', 'alpha 0.3'))
plt.show()
```


```python
# model4 = model(X_train=X_train, Y_train=y_train, X_test=X_test, Y_test=y_test, num_iterations = 10, learning_rate = 0.01)

# model5 = model(X_train=X_train, Y_train=y_train, X_test=X_test, Y_test=y_test, num_iterations = 100, learning_rate = 0.01)

# model6 = model(X_train=X_train, Y_train=y_train, X_test=X_test, Y_test=y_test, num_iterations = 1000, learning_rate = 0.01)
```

c07d77e74bcc483bb8b6170e68fd0e7519daef90
ipynb
Jupyter Notebook
notebooks/Multivariate-Linear-Regression.ipynb
silencedsre/Multivariate-Linear-Regression
9c17dd88d077ee58d3213d6b786dab1cf01b9325
MIT
notebooks/Multivariate-Linear-Regression.ipynb
silencedsre/Multivariate-Linear-Regression
9c17dd88d077ee58d3213d6b786dab1cf01b9325
MIT
notebooks/Multivariate-Linear-Regression.ipynb
silencedsre/Multivariate-Linear-Regression
9c17dd88d077ee58d3213d6b786dab1cf01b9325
MIT
Qwen/Qwen-72B
1. YES
2. YES