Worth a Read: Sloan Foundation President Dr. Paul Joskow on “Research Integrity and Reproducibility”

Screen Shot 2016-05-06 at 10.19.13 AM

I just got pointed to this letter written by the President of the Alfred P. Sloan Foundation Paul Joskow on “Research Integrity and Reproducibility” and it is definitely worth a read.

It is available as part of the Sloan Foundation 2014 Annual report. It starts on pXiii.  It is also available via Research Gate.  And I have extracted the letter from the Annual Report and posted it here.

The letter provides a bit of background (including discussing the claims that Science is Broken and refutations of that claim).  Joskow goes on to state ”

“I agree that the recent flurry of studies that raise questions about the credibility of results reported in important scientific papers should not lead to the conclusion that “science is broken.” However, this is not to say that there is no room for improvement”

I generally agree with that statement.  He then outlines some questions related to this topic and to the issue of reproducibility:

  • Why are there seemingly so many published research papers whose empirical results cannot be readily reproduced by other scientists?
  • Why do there seem to be so many high profile retractions of published research papers?
  • How can we change scientific practice, institutions, incentives, or norms in ways that lead to more reproducible, more reliable research?

And then he declares, “The Alfred P. Sloan Foundation has a keen interest in these questions.” which is good to see (although I did kind of know this already).

The rest of the latter then focuses on two particular topics

In what follows I will discuss how the structure of scientific careers and the incentives facing scientists and journals create barriers to conducting fully reproducible research. I will then discuss the opportunities grantmakers have to reduce or remove those barriers and what the Sloan Foundation is doing to make the published scientific literature more reproducible, reliable, and credible.

Among the topics he covers in the 1st topic:

  • What Is Reproducibility?
  • Failures of Reproducibility, Replicability, and Generalizability
  • Incentives (with a funny footnote: Come on, I’m an economist. You think there wasn’t going to be a section on incentives?).

And then what I found most interesting is the part on “Opportunities for Grantmakers”

  • Insist on high methodological standards
  • Support the development of new tools that lower the costs of making data and methods available
  • Fund replication projects
  • Foster the development and adoption of norms and institutions devoted to good replication practices
  • Train scientists directly
  • Support data science professionals
  • Explore ways to facilitate the repeated scientific analysis of private and proprietary data

Overall I think this is definitely worth a read for those working in any area of science or science funding or related activities.


  • I receive funding from the Alfred P. Sloan Foundation for the microBEnet project
  • I really like the Alfred P. Sloan Foundation.
  • The program officer for my microBEnet project Paula Olsiewski is the one who pointed me to this letter.

Leave a Reply

Jonathan Eisen

I am an evolutionary biologist and a Professor at U. C. Davis. My lab is in the UC Davis Genome Center and I hold appointments in the Department of Medical Microbiology and Immunology in the School of Medicine and the Department of Evolution and Ecology in the College of Biological Sciences. My research focuses on the origin of novelty (how new processes and functions originate). To study this I focus on sequencing and analyzing genomes of organisms, especially microbes and using phylogenomic analysis (see my lab site here which has more information on lab activities).  In addition to research, I am heavily involved in the Open Access publishing and Open Science movements.