Reproducibility - when?

Julia Romanowska

BIOS, UiB

2025-10-28

Project walkthrough

When we publish a manuscript, we typically describe the analysis process as a straight path, from receiving the data to presentation of the results.

In reality, it’s more complicated than that. Usually, it’s at least one cycle:

we get the data
start looking at the data, exploring, this takes time and we never write about this in our publications!
we create a cleaned version of the data, prepared for the analysis
we run one model, perhaps some other models for sensitivity analyses
we plot some figures with results or look a tables with results
usually, we present these findings to some smaller group, perhaps on a local seminar or group meeting
we get some feedback or questions, and, bam, we’re back to looking at the data and creating a different dataset for another set of analyses

Since this is a cycle, it’s useful to keep the notes and codes of what has been done and tried, when we come to the presentation and get tons of suggestions.

At which stage of this cycle should we think about reproducibility?

Task

What would help make these steps reproducible?

receiving data (first look)
data exploration and cleaning
analysis
creating visualizations
preparing paper
submitting paper and data

05:30

Task

receiving data (first look)
- copy and store original data untouched!
- create folder structure
data exploration and cleaning
- tidy data
- keep track of tries
analysis
- make scripts
- keep notes and/or document scripts

Task

creating visualizations
- use scripts, don’t change the graph by hand!
- meaningful naming of figure files
preparing paper
- reproducible reports: Quarto / Rmarkdown
submitting paper and data
- save results in open file formats
- tidy data!
- create a repository, share, also meta-data

Reproducible research

analysis steps
inputs, outputs
code used to generate each figure
notes
versions of software, packages, etc.

Share all that can be shared!

Information is gold

data, data, data …
meta-information is equally important!
- logs of changes
- failed trials (not publishable, but very useful!)
- documentation of code and datasets

Task

Which of these three tables is best to:

look at
re-use in research

Why?

Find the tables in data/table{1,2,3}

05:30

Task

Open a recent project, find some table with results and amend if needed:

create a table that will be easy to re-use: no captions, no explanations, data starts in the very first row and column, numbers are numbers, no merged columns, etc.
is it in a proprietary format? Change it to .csv!
does it have punctuation marks in column names? Fix the naming!
no accompanying text document (not Word!) explaining the meaning of the column names? Create it!

05:30

Book

Building reproducible analytical pipelines with R by Bruno Rodrigues

Quarto

Quarto publishing system

alternative to Rmarkdown,
can execude code in R, python, Stata, JuliaLang