Chapter 3 Open Science & Reproducible Research
What do we mean when we say open science and reproducibility? Well, a lot..
Open *
When we talk about openness, we run the gamut:
- Open Code
- Open Data
- Open Access Papers
- Open Educational Resources
- Open methods
- Open lab notebooks
and many more!
In this session, we’ll cover open publishing of an executable paper!
Reproducibility
Reproducibility exists on a spectrum!
Reviewable Research: Sufficient detail for peer review & assessment.
Replicable Research: Tools are available to duplicate the author’s results using their data.
Confirmable Research: Main conclusions can be attained independently without author’s software.
Auditable Research: Process & tools archived such that it can be defended later if necessary.
Open/Reproducible Research: Auditable research made openly available.
Stodden et al ICERM report (2013)
Even if runnable, results may differ
We investigated the effects of data processing variables such as FreeSurfer version (v4.3.1, v4.5.0, and v5.0.0), workstation (Macintosh and Hewlett-Packard), and Macintosh operating system version (OSX 10.5 and OSX 10.6). Significant differences were revealed between FreeSurfer version v5.0.0 and the two earlier versions. […] About a factor two smaller differences were detected between Macintosh and Hewlett-Packard workstations and between OSX 10.5 and OSX 10.6.
Sharing Your Research with the Open Science Framework The Open Science Framework (OSF) lets you integrate your normal workflow into one management platform!
- Wiki: document your lab procedures, standards, etc.
- Collaborators: add collaborators of all levels, on different parts of your project
- Components: sub-projects to organize your research
- Version Control: upload files of the same name & OSF will track your versions!
- Add-Ons: use OSF to bring together tools you use | GitHub
- Registrations: when you have an unchanging version of your project, register it & get a DOI!
- Preprint Integration: it only takes 2 clicks to integrate your OSF project
** Preprints/Postprints **
Preprints: a version of your paper before it has undergone peer review Postprints: a version of your paper after peer review without the publisher’s formatting
Resources for Learning More
- Open Science MOOC
- Open Research Glossary
- Why Open Research
- Sherpa ROMEO - publisher copyright policies & self-archiving
- OSF Preprints
- Do you speak open science? Resources and tips to learn the language
- Making Science Transparent By Default; Introducing the TOP Statement
- How Computers Broke Science
- An introduction to Docker for reproducible research, with examples from the R environment
- Ten Simple Rules for Reproducible Computational Research
- The Practice of Reproducible Research
- Terminologies for Reproducible Research
- Good enough practices in scientific computing
- Data reuse and the open data citation advantage
- The data paper: a mechanism to incentivize data publishing in biodiversity science