## What is a Percepton

A percepton is a type of binary classifier. A binary classfier is a function
which can be decide weather or not an input, represented by a vector of numbers,
belong to a specific class. 

The perceptron maps its input $x$ ( a real value vector ) to an output value $f(x)$ ( a single binary value ):

\begin{equation}
    f(x) = \chi(\langle w, x \rangle + b)
\end{equation}

where $w$ is the vector of the weights with real value, $\langle \cdot , \cdot \rangle$ is the scalar product,
$b$ is the bias,  a constant term that doesn't dipend on any input value and $\chi(x)$ is the output fuction, also called 
activation function. 
The most common choices for $\chi(x)$ are:

- $\chi(x)$ = $sing(x)$
- $\chi(x)$ = $Θ(x)$
- $\chi(x)$ = $x$

where $Θ(x)$ is the Heavside Function.



The perceptron works weel when the learning set is linearly separable, while when the learning isn't linearly separable
its learning algorithm doesn't terminate. If the vector are not linearly separable will never reach a point where all vectors are 
classified properly. The most famous example of perceptron inability to solve problems with linearly separable vector is the 
boolean **XOR** problem.



## How to train the Perceptron

There are various way to train a perceptron, one of the most effienct ( the one that used here )
is the **Delta Rule**

The Delta Rule is a gradient descend learning rule for updating the weights of the inputs of an artifical 
neuron in a single-layer neural network. It is a special case of the more general backpropagation algorithm. For
a neuron $j$ with activation function $g(x)$, the delta rule for $j$'s weight $w_{ji}$ is given by:

\begin{equation}
\Delta w_{ji} = \eta (t_j - y_j)g'(h_i)x_i
\end{equation}

where:

- $\eta$ is a small constant, called learning rate
- $g(x)$ is the neuron activation function
- $g'$ is the derivative of $g$
- $t_j$ us the target output
- $h_i$ is the weighted sum of the neuron's inputs
- $y_j$ is the actual output
- $x_i$ is the ith input

It holds that $h_j = \sum x_i w_{ji}$ and $ y_j = g(h_j)$. 
The delta rule is commonly stated is a simplified forn for a neuron with 
a linear activation function as 

\begin{equation}
\Delta w_{ji} = \eta (t_j - y_j)x_i
\end{equation}

While the delta rule is similar to the percetron's training rule, the derivation is different.
If the percepton uses the Heaviside step function as the activation function , it turn out that 
$g(h)$ is no differantiable in zero, which make the direct application of the delta rule impossible.

c11a0afea853f65bff9ed51e8b36f01f0bdf3bcd
ipynb
Jupyter Notebook
demo/What_is_a_perceptron.ipynb
paolodelia99/Python-Perceptron
9a4e9ab47634fd7a81d54c5792658b77a715b4b4
MIT
2021-03-16T11:18:01.000Z
2021-03-16T11:18:01.000Z
demo/What_is_a_perceptron.ipynb
paolodelia99/Python-Perceptron
9a4e9ab47634fd7a81d54c5792658b77a715b4b4
MIT
demo/What_is_a_perceptron.ipynb
paolodelia99/Python-Perceptron
9a4e9ab47634fd7a81d54c5792658b77a715b4b4
MIT
Qwen/Qwen-72B
1. YES
2. YES