## Homework-4
### CSC-722: Machine Learning Fundamentals
### Md Hafizur Rahman


###  1) Write a code to find norm 0, norm 1, norm 2, and norm infinity of the vectore x

### Theory:
In mathematics, a norm is a function from a vector space over the real or complex numbers to the nonnegative real numbers, that satisfies certain properties pertaining to scalability and additivity and takes the value zero only if the input vector is zero [1].

On $R^n$ and $p≥1$, where R is real number and n=0,1,2... the p-norm is defined as
\begin{align}
l_p=(\sum_{j=1}^n|x_j|^p)^{1/p}
\end{align}

In our case, we have to calculate $l_0$, $l_1$, $l_2$ and $l_\inf$. X is a list of 10 random numbers between -10 and 10. So, j= 1,2,3...,10


```python
#importing the necessary libraries
import numpy as np
from scipy.sparse import csr_matrix
```


```python
#generating 10 random numbers of a list between -10 and 10.
x = np.random.randint(-10, 10, 10)
print(x)
```

    [  2   9  -9 -10   6  -5  -8  -9   5  -8]




#### Norm-0:

The $l_0$ norm corresponds to the total number of nonzero elements in a given vector.

We have used count_nonzero method of numpy to find the norm-0.


```python
norm_0=np.count_nonzero(x)
print("Norm-0 of the X is", norm_0)
```

    Norm-0 of the X is 10


#### Norm-1:
Norm-1 is also known as Manhattan Distance or Taxicab norm. L1 norm is the sum of the magnitudes of the vectors in a space. 

Mathematically,

\begin{align}
l_1=(\sum_{j=1}^n|x_j|)
\end{align}

We used sum method of numpy to calculate the norm-1. abs() has to use for getting the absolute values of x. 


```python
norm_1=sum(abs(x))
print("Norm-1 of the X is", norm_1)
```

    Norm-1 of the X is 71


#### Norm-2:
Norm-2 is also known as the Euclidean norm. It is the shortest distance to go from one point to another. 

Mathematically, we can write the $l_2$ norm as

\begin{align}
l_2=(\sum_{j=1}^n|x_j|^2)^{1/2}
\end{align}

We used sum method of numpy to calculate the norm-1. abs() has to use for getting the absolute values of x. 


```python
norm_2=(sum(x**2))**(1/2)
print("Norm-2 of the X is",norm_2) 
```

    Norm-2 of the X is 23.68543856465402


#### Norm-inf:
Norm-inf gives the largest magnitude among each element of a vector. 

Mathematically, we can write $l_\inf$ norm as

\begin{align}
l_\inf=max|x_j|
\end{align}

We used max method of numpy to calculate the norm-inf. abs() has to use for getting the absolute values of x. 


```python
norm_inf=max(abs(x))
print("Norm-inf of the X is",norm_inf)
```

    Norm-inf of the X is 10


###  2) Explain the following cells 
###  3) Explain what they do and why

If most of the elements in the matrix are zero then the matrix is called a sparse matrix. It is wasteful to store the zero elements in the matrix since they do not affect the results of our computation. This is why we implement these matrices in more efficient representations than the standard 2D Array. Using more efficient representations we can cut down space and time complexities of operations significantly without changing the main matrix[2]. We are disscusing the following example step by step.

At first, we generate a 3 $\times$ 6 matrix and store in A. We are seeing that most of the elements of A are zero. We can create CSR matrix for A.


```python
A = np.array([[1, 0, 0, 1, 0, 0], [0, 0, 2, 0, 0, 1], [0, 0, 0, 2, 0, 0]])
print(A)
```

    [[1 0 0 1 0 0]
     [0 0 2 0 0 1]
     [0 0 0 2 0 0]]


To create CSR matrix, we used csr_matrix function from numpy.


```python
# convert to sparse matrix (CSR method)
S = csr_matrix(A)
print(S)
```

      (0, 0)	1
      (0, 3)	1
      (1, 2)	2
      (1, 5)	1
      (2, 3)	2


From the result we can see that there are 5 items [consider in row] with value.
The row and position start from 0.

The 1. item is in row 0 position 0 and has the value 1.

The 2. item is in row 0 position 3 and has the value 1.

The 3. item is in row 1 position 2 and has the value 2.

The 4. item is in row 1 position 5 and has the value 1.

The 5. item is in row 2 position 3 and has the value 2.

Therefore, this matrix S will take less space and reduce the time of computation.

We can reconstruct the orginal matrix A from S. For that we have used todense() function.


```python
# reconstruct dense matrix
B = S.todense()
print(B)
```

    [[1 0 0 1 0 0]
     [0 0 2 0 0 1]
     [0 0 0 2 0 0]]


#### Refs:
[1] Bourbaki, Nicolas (1987) [1981]. Topological Vector Spaces: Chapters 1–5 [Sur certains espaces vectoriels topologiques]. Annales de l'Institut Fourier. Elements of mathematics. 2. Translated by Eggleston, H.G.; Madan, S. Berlin New York: Springer-Verlag. ISBN 978-3-540-42338-6. 

[2] Pissanetzky, Sergio (1984). Sparse Matrix Technology. Academic Press.

For the code please click the following link:

https://github.com/hafizurr/Machine_Learnig_course/blob/master/home_work/Homework_4.ipynb

38864fd980ff713f645a5b75e95a7c2affbddb60
ipynb
Jupyter Notebook
home_work/Homework_4.ipynb
hafizurr/Machine_Learnig_course
2a1fd51fd69a3d62730be500f794f30165a7cf24
Unlicense
home_work/Homework_4.ipynb
hafizurr/Machine_Learnig_course
2a1fd51fd69a3d62730be500f794f30165a7cf24
Unlicense
home_work/Homework_4.ipynb
hafizurr/Machine_Learnig_course
2a1fd51fd69a3d62730be500f794f30165a7cf24
Unlicense
Qwen/Qwen-72B
1. YES
2. YES