# differentiation for optimization

### basic formular



- constant

$$ \dfrac{d}{dx}(c) = 0 $$ 

- exponentiation

$$ \dfrac{d}{dx}(x^n) = nx^{n-1} $$

- log

$$ \dfrac{d}{dx}(\log x) = \dfrac{1}{x} $$

- Euler

$$ \dfrac{d}{dx}(e^x) = e^x $$

### linear combination


$$ \dfrac{d}{dx}(c_1f_1 + c_2f_2) = c1\dfrac{df_1}{dx} + c2\dfrac{df_2}{dx} $$

### multiplication


$$ \dfrac{d}{dx}(f \cdot g) = \dfrac{df}{dx} \cdot g + f \cdot \dfrac{dg}{dx}$$

### chain rule

$$ f(x) = h(g(x)) $$


$$ -> \dfrac{df}{dx} = \dfrac{dh}{dg} \cdot \dfrac{dg}{dx} $$

- ex: log function

$$ \dfrac{d}{dx}\log f(x) = \dfrac{f'(x)}{f(x)} $$

---

## second derivative
- differentiate derivative

$$ f'' = \dfrac{d^2}{dx^2}f = \dfrac{d^2f}{dx^2} = \dfrac{d^2}{dx^2}y $$

- second derivative means the slope of derivative
    - when the slope(derivative) goes up, second derivative value is positive
    - and we call it **convex**
    - the other way, second derivative is negative, it is called **concave**

### partial differentiation
- when indipendant variables are more than one, differentiate one variable at once. which is partial differentiation (for convenient) 

$$ f_x(x, y) = \dfrac{\partial f}{\partial x} $$
$$ f_y(x, y) = \dfrac{\partial f}{\partial y} $$



---

## Sympy
for symbolic operation



```python
import sympy
sympy.init_printing(use_latex='mathjax')
```


```python
x, mu, sigma = sympy.symbols('x mu sigma')
f = sympy.exp((x-mu)**2 / sigma **2)
f
```




$$e^{\frac{1}{\sigma^{2}} \left(- \mu + x\right)^{2}}$$



- in case partial, need to nominate the variable 


```python
sympy.diff(f, x) #differentiation
```




$$\frac{1}{\sigma^{2}} \left(- 2 \mu + 2 x\right) e^{\frac{1}{\sigma^{2}} \left(- \mu + x\right)^{2}}$$




```python
sympy.simplify(sympy.diff(f, x))
```




$$\frac{2}{\sigma^{2}} \left(- \mu + x\right) e^{\frac{1}{\sigma^{2}} \left(\mu - x\right)^{2}}$$



443c258563e593c8d78800bda30885d983876e25
ipynb
Jupyter Notebook
Past/DSS/Math/180127_differentiation.ipynb
Moons08/TIL
e257854e1f7b9af5a6e349f38037f3010c07310f
MIT
Past/DSS/Math/180127_differentiation.ipynb
Moons08/TIL
e257854e1f7b9af5a6e349f38037f3010c07310f
MIT
Past/DSS/Math/180127_differentiation.ipynb
Moons08/TIL
e257854e1f7b9af5a6e349f38037f3010c07310f
MIT
Qwen/Qwen-72B
1. YES
2. YES