Pointwise mutual information indicates how surprising it is to see two events co-occur. Mutual information is the ratio of the [[joint probability]] with the probability if we assumed both events were independent. $I(x,y) = \log_2 \frac{P(x,y)}{P(x)P(y)} $