# Relu - $ReLU(x) = max(0,x)$ - $\frac{d}{d_x}ReLU(X) = \begin{cases}0 & x \geq 0 \\ 1 & otherwise \end{cases}$ - He init - MLP, CNN : Hidden -  - [Leaky Relu](Leaky%20Relu.md) - [PRelu](PRelu.md) - [Noisy Relu](Noisy%20Relu.md)