Covariance

For each outcome, take the deviation of \(X\) from its mean and that of \(Y\) from its own, and multiply them. If both usually have the same sign the product is mostly positive and the covariance is positive; if they usually have opposite signs it is negative; if there is no tendency it stays near 0.

To compute it, use the equivalent form \(\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]\). With \(E[X]=2\), \(E[Y]=3\) and \(E[XY]=7\) it equals 1. A typical mistake is forgetting to subtract the product of the means and reporting \(E[XY]\) as the covariance.

It is symmetric, satisfies \(\operatorname{Cov}(X,X)=\operatorname{Var}(X)\) and is linear in each argument, so \(\operatorname{Cov}(aX+b,cY+d)=ac\operatorname{Cov}(X,Y)\). Also, \(\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y)\): the cross term separates the variance of a sum from the sum of variances.

If \(X\) and \(Y\) are independent the covariance is 0, but the converse is false: with \(X\) uniform on \(\{-1,0,1\}\) and \(Y=X^2\) the covariance is 0 and yet \(Y\) depends on \(X\). Covariance only detects linear dependence; its size is bounded by \(\sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)}\) and its units are the product of those of \(X\) and \(Y\), so it is usually normalized to obtain the correlation.