Chapter 3 Open Science & Reproducible Research

What do we mean when we say open science and reproducibility? Well, a lot..

Open *

When we talk about openness, we run the gamut:

  • Open Code
  • Open Data
  • Open Access Papers
  • Open Educational Resources
  • Open methods
  • Open lab notebooks

and many more!

In this session, we’ll cover open publishing of an executable paper!

Reproducibility

Reproducibility exists on a spectrum!

Reviewable Research: Sufficient detail for peer review & assessment.

Replicable Research: Tools are available to duplicate the author’s results using their data.

Confirmable Research: Main conclusions can be attained independently without author’s software.

Auditable Research: Process & tools archived such that it can be defended later if necessary.

Open/Reproducible Research: Auditable research made openly available.

Stodden et al ICERM report (2013)

Levels of reproducibility

Levels of reproducibility

Even if runnable, results may differ

The Effects of FreeSurfer Version, Workstation Type, and Macintosh Operating System Version on Anatomical Volume and Cortical Thickness Measurements | June 1, 2012

We investigated the effects of data processing variables such as FreeSurfer version (v4.3.1, v4.5.0, and v5.0.0), workstation (Macintosh and Hewlett-Packard), and Macintosh operating system version (OSX 10.5 and OSX 10.6). Significant differences were revealed between FreeSurfer version v5.0.0 and the two earlier versions. […] About a factor two smaller differences were detected between Macintosh and Hewlett-Packard workstations and between OSX 10.5 and OSX 10.6.

Sharing Your Research with the Open Science Framework The Open Science Framework (OSF) lets you integrate your normal workflow into one management platform!

OSF Connections

OSF Connections

  • Wiki: document your lab procedures, standards, etc.
  • Collaborators: add collaborators of all levels, on different parts of your project
  • Components: sub-projects to organize your research
  • Version Control: upload files of the same name & OSF will track your versions!
  • Add-Ons: use OSF to bring together tools you use | GitHub
  • Registrations: when you have an unchanging version of your project, register it & get a DOI!
  • Preprint Integration: it only takes 2 clicks to integrate your OSF project

** Preprints/Postprints **

Preprints: a version of your paper before it has undergone peer review Postprints: a version of your paper after peer review without the publisher’s formatting

Resources for Learning More