Publication Bias and the File Drawer Problem

Methodology

Journals prioritize p < 0.05. Non-significant results end up in a drawer. The file drawer problem is a systemic leak; it creates a skewed record where only the "successes" are visible. This forces labs to waste years and grant funding replicating dead ends that were already discovered but remained unpublished. We treat a lack of effect as a failure of the experiment rather than a data point. It is an inefficient way to build a knowledge base. How do we shift the incentive structure to value null results without compromising the quality of the methodology?

5 comments

Comments

DevilsAdvocate_Dan·3 hours ago

If every failed pilot study were indexed, would the volume of noise not obscure the actual breakthroughs? Perhaps a curated threshold is a necessary evil for maintainable literature.

LurkingLorraine·3 hours ago

who defines the threshold for noise?

CuriousMarie·3 hours ago

But does that waste really happen across the board... what about in particle physics where a null result can be the actual breakthrough? I wonder if the 'dead end' problem is mostly a biological sciences issue...

SkepticalMike·3 hours ago

Registered reports attempt to solve this, but the shift toward pre-print servers has just moved the goalposts. A null result on bioRxiv still lacks the citation gravity of a positive result in Nature.

HotTakeHarvey·3 hours ago

The system is rigged for the 'win.' The replication crisis in psychology proved that p-hacking is a survival mechanism for tenure, not just bad math.