Checking Preregistrations on OSF

Methodology

Abstracts are effectively press releases. They highlight the most statistically significant result, regardless of whether that result was the actual point of the study. To avoid being misled, check the Open Science Framework (OSF) for a preregistration before you commit to the conclusions. The workflow is straightforward. Search the OSF for the study title or lead author. Locate the preregistration document. Compare the listed primary endpoints and hypotheses to the results section of the published paper. If the paper claims a breakthrough on a variable that was not mentioned in the preregistration, you are looking at HARKing (Hypothesizing After the Results are Known). It is easy to find a pattern in a large enough dataset if you are allowed to change the question after the data is collected. Moving from passive reader to auditor requires more effort, but it prevents you from citing noise.

6 comments

Comments

ProfActuallyPhD·1 hour ago

This mirrors the evolution of ClinicalTrials.gov in medical research. The shift toward mandatory outcome reporting was designed specifically to prevent the silent disappearance of failed endpoints in drug trials.

LurkingLorraine·1 hour ago

what about journals that use proprietary registration systems instead of osf?

SkepticalMike·1 hour ago

The platform matters less than the timestamp. The real red flag is a preregistration updated after the data collection phase began.

MemoryHoleMarcus·1 hour ago

We saw this play out during the social psychology replication crashes of the last decade. Now we have registered reports where the methodology is peer reviewed before the data is even collected.

DevilsAdvocate_Dan·1 hour ago

If a researcher tests twenty different variables but only reports the one with a p-value under 0.05, the result is mathematically likely to be a fluke. Preregistration creates a paper trail that makes this selective reporting transparent.

CuriousMarie·1 hour ago

But what happens if they find something totally unexpected and amazing... can they still publish that as an exploratory finding without it being HARKing?