# Logistic Regression
[source](https://www.youtube.com/watch?v=hSXFuypLukA)
### Step 1: Function Set  
We want to find $P_{w,b}(C1|x)$  
if $P_{w,b}(C_1|x) \geq 0.5$, output $C_1$, otherwise, output $C_2$  

$P_{w,b}(C_1|x) = \sigma(z)$  
$$z = w \cdot x + b = \sum_{i}wi xi + b \\  
\sigma(z) = \frac{1}{1+exp(-z)}$$  

$\begin{equation}
\large{f_{w,b}(x) = \sigma(\sum_{i}w_i x_i + b)}  
\end{equation}$  
Output should between 0 and 1


### Step 2: Goodness of a Function
Training Data$
\ x^1 \ x^2 \ x^3 \ ... \ x^N \\
\ C_1 \ C_2 \ C_1 \ ... \ C_1 
$  
Assume the data is generated based on $f_{w,b}(x) = P_{w,b}(C_1|x)$  
Given a set of w and b, what is its probability of generating the data?  
$\begin{equation}
\large L(w,b) = f_{w,b}(x^)(1 - f_{w,b}(x^2))f_{w,b}(x^3)...f_{w,b}(x^N)
\end{equation}$  

The most likely $w^*$ and $b^*$ is the one with the largest $L(w,b)$.  

$$w^*,b^* = arg\ max_{w,b}L(w,b) = 
w^*, b^* = arg\min_{w,b}-lnL(w,b)$$  
$$
x^1 \qquad x^2 \qquad x^3 \qquad ... \\
\hat{y}^1=1 \quad \hat{y}^2=0 \quad \hat{y}^3=1 ... \\
\hat{y}^n : 1 \text{ for class 1, 0 for class 2}
$$


右側省略w,b

左右兩側相等, 故可寫成下列公式
$$
\large{\sum_{n}-[\hat{y}^n ln f_{w,b}(x^n) + (1-\hat{y}^n) ln (1- f_{w,b}(x^n))]} \\
\text{Cross entropy between two Bernoulli distribution}
$$  


### Step 3: Find the best function









f45b72617adfed2b01dc8751d64e5d79aee9d0bc
ipynb
Jupyter Notebook
Machine learning/Logistic Regression/notebook.ipynb
Sean2525/notebooks
10afea47674f437441426ba32c42f2bb5d25c19f
MIT
Machine learning/Logistic Regression/notebook.ipynb
Sean2525/notebooks
10afea47674f437441426ba32c42f2bb5d25c19f
MIT
Machine learning/Logistic Regression/notebook.ipynb
Sean2525/notebooks
10afea47674f437441426ba32c42f2bb5d25c19f
MIT
Qwen/Qwen-72B
1. YES
2. YES