# Tag Info

In terms of the Heaviside step function $$H(x)=\frac{1+sign(x)}{2}$$ the leaky ReLU can be written as $$(\alpha + (1-\alpha)H(x))x$$ so its derivative is $$\alpha + (1-\alpha)H(x)$$ In your ...