The natural way to determine how dependent the distributions of ® andā re is to calculate their mutual information (Church and Hanks, 1991): If the occurrences of ® and¯were completely independent of each other, we would expect the maximum likelihood probability .
mutual information is zero when ® and¯are independent, and positive otherwise.