Reflecting on our experiences writing letters to editors and authors to try to improve the literature through post-publication peer review, we were invited to write a comment in Nature, which we titled “Reproducibility: A tragedy of errors”.

Over the course of at least 18 months, we wrote numerous letters to editors and authors, including direct contacts, contacts through submission systems, and PubMed Commons. The effort took much time and was often frustratingly slow, including sometimes ending without a resolution.

However, the efforts resulted in multiple retractions and corrections, which is a win for the purported self-correcting nature of science (though admittedly, not a win for science’s current PR crisis). The Nature commentary also provided a nice form of ‘academic-credit’ for our efforts. If time permits, we would like to to write a more verbose reflection on our experiences, with hopes that it will help authors avoid mistakes, peer-reviewers catch them, and inspire post-publication review.

In the interim, check out Compare Trials to see an ongoing effort of comparing clinical trials with their registration documents.