Journal Article

Positive and negative forms of replicability in gene network analysis

W. Verleyen, S. Ballouz and J. Gillis

in Bioinformatics

Volume 32, issue 7, pages 1065-1073
Published in print April 2016 | ISSN: 1367-4803
Published online December 2015 | e-ISSN: 1460-2059 | DOI:
Positive and negative forms of replicability in gene network analysis

More Like This

Show all results sharing this subject:

  • Bioinformatics and Computational Biology


Show Summary Details


Motivation: Gene networks have become a central tool in the analysis of genomic data but are widely regarded as hard to interpret. This has motivated a great deal of comparative evaluation and research into best practices. We explore the possibility that this may lead to overfitting in the field as a whole.

Results: We construct a model of ‘research communities’ sampling from real gene network data and machine learning methods to characterize performance trends. Our analysis reveals an important principle limiting the value of replication, namely that targeting it directly causes ‘easy’ or uninformative replication to dominate analyses. We find that when sampling across network data and algorithms with similar variability, the relationship between replicability and accuracy is positive (Spearman’s correlation, rs ∼0.33) but where no such constraint is imposed, the relationship becomes negative for a given gene function (rs ∼ −0.13). We predict factors driving replicability in some prior analyses of gene networks and show that they are unconnected with the correctness of the original result, instead reflecting replicable biases. Without these biases, the original results also vanish replicably. We show these effects can occur quite far upstream in network data and that there is a strong tendency within protein–protein interaction data for highly replicable interactions to be associated with poor quality control.

Availability and implementation: Algorithms, network data and a guide to the code available at:


Supplementary information: Supplementary data are available at Bioinformatics online.

Journal Article.  8197 words.  Illustrated.

Subjects: Bioinformatics and Computational Biology

Full text: subscription required

How to subscribe Recommend to my Librarian

Users without a subscription are not able to see the full content. Please, subscribe or login to access all content. subscribe or login to access all content.