>[!success] Theorem >Minimizing the [[Kullback-Leibler Divergence|KL-divergence]] between two distributions $q$ and $p$ is equivalent to maximizing the [[Likelihood and Score Function|log-likelihood]], because$\mathbb{E}_{x \sim p(x)}[\log(q(x))]=- H(p(x))-KL(p(x)||q(x)).$