## Neural Networks
Mathematically this looks like:

$$
\begin{align}
y &amp;= f(w_1 x_1 + w_2 x_2 + b) \\
y &amp;= f\left(\sum_i w_i x_i +b \right)
\end{align}
$$
With vectors this is the dot/inner product of two vectors:

$$
h = \begin{bmatrix}
x_1 \, x_2 \cdots  x_n
\end{bmatrix}
\cdot 
\begin{bmatrix}
           w_1 \\
           w_2 \\
           \vdots \\
           w_n
\end{bmatrix}
$$


## Tensors
It turns out neural network computations are just a bunch of linear algebra operations on tensors, a generalization of matrices. A vector is a 1-dimensional tensor, a matrix is a 2-dimensional tensor, an array with three indices is a 3-dimensional tensor (RGB color images for example). The fundamental data structure for neural networks are tensors and PyTorch (as well as pretty much every other deep learning framework) is built around tensors.

### Reshaping Tensors
There are a few options here: weights.reshape(), weights.resize_(), and weights.view().

- weights.reshape(a, b) will return a new tensor with the same data as weights with size (a, b) sometimes, and sometimes a clone, as in it copies the data to another part of memory.

- weights.resize_(a, b) returns the same tensor with a different shape. However, if the new shape results in fewer elements than the original tensor, some elements will be removed from the tensor (but not from memory). If the new shape results in more elements than the original tensor, new elements will be uninitialized in memory. Here I should note that the underscore at the end of the method denotes that this method is performed in-place. Here is a great forum thread to read more about in-place operations in PyTorch.

- weights.view(a, b) will return a new tensor with the same data as weights with size (a, b).



```python
#import pyTorch
import torch
```


```python
def activation(x):
    return 1/(1+ torch.exp(-x))
```


```python
#Generate some data
torch.manual_seed(7) #set the random seeds so things are predictible

#Features are 5 random normal varables , 1 row 5 columns
features = torch.randn((1,5))
#True weights for our data , random normal variables again , weights created of same shape as features
weights = torch.rand_like(features)
#Bias term
bias = torch.randn((1,1))
```


```python
#Output of network
y = activation(torch.sum(weights*features) + bias)
print(y)
# y = activation((features*weights).sum + bias)
```

    tensor([[0.6140]])



```python
#Methods for matrix multiplication
    # torch.mm() - strict about shape of tensors 
    # torch.matmul() - provides output even if shape not appropriate
'''
torch.mm(weights,features) 
provides output---  size mismatch, m1: [1 x 5], m2: [1 x 5] 
'''
#To find shape of a tensor - tensor.shape
print(weights.shape)


```

    torch.Size([1, 5])



```python
#Reshaping Tensors
# tensor.reshape(), tensor.view() , tensor.resize_()- underscore means it is inplace operation
# features.view(5,1) - creates a new tensor have to be saved

```


```python
#Output using matrix multiplication 
y = activation(torch.mm(weights,features.view(5,1)) + bias)
print(y)
```

    tensor([[0.6140]])


1d4fb4c959b55bd9d3a1ad4d9bf07a4ca5ecad11
ipynb
Jupyter Notebook
2_Simple Neural Network and Random Functions.ipynb
AkankshaShrimal/Deep-Learning-Pytorch-Step-Wise-Learning
6216027d57fad0aa00c3bc9074d9dab94cf91a39
MIT
2019-02-21T14:38:06.000Z
2019-07-16T11:50:55.000Z
2_Simple Neural Network and Random Functions.ipynb
AkankshaShrimal/Deep-Learning-Pytorch-Step-Wise-Learning
6216027d57fad0aa00c3bc9074d9dab94cf91a39
MIT
2018-12-30T15:49:52.000Z
2019-01-12T04:34:52.000Z
2_Simple Neural Network and Random Functions.ipynb
AkankshaShrimal/Deep-Learning-Pytorch-Step-Wise-Learning
6216027d57fad0aa00c3bc9074d9dab94cf91a39
MIT
Qwen/Qwen-72B
1. YES
2. YES