Journal Article

Mutual information is critically dependent on prior assumptions: would the <i>correct</i> estimate of mutual information please identify itself?

Andrew D. Fernandes and Gregory B. Gloor

in Bioinformatics

Volume 26, issue 9, pages 1135-1139
Published in print May 2010 | ISSN: 1367-4803
Published online March 2010 | e-ISSN: 1460-2059 | DOI:
Mutual information is critically dependent on prior assumptions: would the correct estimate of mutual information please identify itself?

More Like This

Show all results sharing this subject:

  • Bioinformatics and Computational Biology


Show Summary Details


Motivation: Mutual information (MI) is a quantity that measures the dependence between two arbitrary random variables and has been repeatedly used to solve a wide variety of bioinformatic problems. Recently, when attempting to quantify the effects of sampling variance on computed values of MI in proteins, we encountered striking differences among various novel estimates of MI. These differences revealed that estimating the ‘true’ value of MI is not a straightforward procedure, and minor variations of assumptions yielded remarkably different estimates.

Results: We describe four formally equivalent estimates of MI, three of which explicitly account for sampling variance, that yield non-equal values of MI given exact frequencies. These MI estimates are essentially non-predictive of each other, converging only in the limit of implausibly large datasets. Lastly, we show that all four estimates are biologically reasonable estimates of MI, despite their disparity, since each is actually the Kullback–Leibler divergence between random variables conditioned on equally plausible hypotheses.

Conclusions: For sparse contingency tables of the type universally observed in protein coevolution studies, our results show that estimates of MI, and hence inferences about physical phenomena such as coevolution, are critically dependent on at least three prior assumptions. These assumptions are: (i) how observation counts relate to expected frequencies; (ii) the relationship between joint and marginal frequencies; and (iii) how non-observed categories are interpreted. In any biologically relevant data, these assumptions will affect the MI estimate as much or more-so than observed data, and are independent of uncertainty in frequency parameters.


Supplementary information: Supplementary data are available at Bioinformatics online.

Journal Article.  3804 words.  Illustrated.

Subjects: Bioinformatics and Computational Biology

Full text: subscription required

How to subscribe Recommend to my Librarian

Users without a subscription are not able to see the full content. Please, subscribe or login to access all content.