# For today's code challenge you will be reviewing yesterdays lecture material. Have fun!

### if you get done early check out [these videos](https://www.3blue1brown.com/neural-networks).

# The Perceptron

The first and simplest kind of neural network that we could talk about is the perceptron. A perceptron is just a single node or neuron of a neural network with nothing else. It can take any number of inputs and spit out an output. What a neuron does is it takes each of the input values, multplies each of them by a weight, sums all of these products up, and then passes the sum through what is called an "activation function" the result of which is the final value.

I really like figure 2.1 found in this [pdf](http://www.uta.fi/sis/tie/neuro/index/Neurocomputing2.pdf) even though it doesn't have bias term represented there.



If we were to write what is happening in some verbose mathematical notation, it might look something like this:

\begin{align}
 y = sigmoid(\sum(weight_{1}input_{1} + weight_{2}input_{2} + weight_{3}input_{3}) + bias)
\end{align}

Understanding what happens with a single neuron is important because this is the same pattern that will take place for all of our networks. 

When imagining a neural network I like to think about the arrows as representing the weights, like a wire that has a certain amount of resistance and only lets a certain amount of current through. And I like to think about the node itselef as containing the prescribed activation function that neuron will use to decide how much signal to pass onto the next layer.

# Activation Functions (transfer functions)

In Neural Networks, each node has an activation function. Each node in a given layer typically has the same activation function. These activation functions are the biggest piece of neural networks that have been inspired by actual biology. The activation function decides whether a cell "fires" or not. Sometimes it is said that the cell is "activated" or not. In Artificial Neural Networks activation functions decide how much signal to pass onto the next layer. This is why they are sometimes referred to as transfer functions because they determine how much signal is transferred to the next layer.

## Common Activation Functions:



# Implementing a Perceptron from scratch in Python

### Establish training data


```python
import numpy as np

np.random.seed(812)

inputs = np.array([
    [0, 0, 1],
    [1, 1, 1],
    [1, 0, 1],
    [0, 1, 1]
])

correct_outputs = [[0], [1], [1], [0]]
```

### Sigmoid activation function and its derivative for updating weights


```python
def sigmoid(x):
  return 1 / (1 + np.exp(-x))

def sigmoid_derivative(x):
  sx = sigmoid(x)
  return sx * (1 - sx)
```

## Updating weights with derivative of sigmoid function:



### Initialize random weights for our three inputs


```python
weights = np.random.rand(3,1)
weights
```




    array([[0.32659171],
           [0.59345002],
           [0.25569456]])



### Calculate weighted sum of inputs and weights


```python
weighted_sum = np.dot(inputs,weights)
```

### Output the activated value for the end of 1 training epoch


```python
activated_value = sigmoid(weighted_sum)
activated_value
```




    array([[0.56357763],
           [0.76418031],
           [0.64159331],
           [0.70038767]])



### take difference of output and true values to calculate error


```python
error = correct_outputs - activated_value
error
```




    array([[-0.56357763],
           [ 0.23581969],
           [ 0.35840669],
           [-0.70038767]])




```python
derivative = sigmoid_derivative(error)
derivative
```




    array([[0.23115422],
           [0.24655628],
           [0.24214035],
           [0.22168396]])



### Put it all together


```python
weights +=  np.dot(inputs,derivative)
```


```python

```

b40330e34b1d3ceb758b83164aa7563deab03613
ipynb
Jupyter Notebook
Tuesday's_Challenge.ipynb
vishnuyar/Neural_network_foundations_code_challenges
5c628b2634eb25de443faff703b60ffb42d59dc7
MIT
Tuesday's_Challenge.ipynb
vishnuyar/Neural_network_foundations_code_challenges
5c628b2634eb25de443faff703b60ffb42d59dc7
MIT
Tuesday's_Challenge.ipynb
vishnuyar/Neural_network_foundations_code_challenges
5c628b2634eb25de443faff703b60ffb42d59dc7
MIT
Qwen/Qwen-72B
1. YES
2. YES