The Pre-Registration Audit: Spotting HARKing

Methodology

Ever read a high-impact paper and felt like the result was just... too perfect? Like the researchers stumbled onto a goldmine by accident but wrote it as if they knew exactly where to dig... it happens more than we think. There is a way to play detective here. It is called the pre-registration audit. When you see a study, don't just trust the 'Results' section. Look for the pre-registration link... usually on OSF or ClinicalTrials.gov. Here is the move: open the original registration and find the 'Primary Endpoint' or the main hypothesis. Then, look at the paper's main conclusion. If the paper is shouting about a discovery that wasn't even in the original plan... you've found HARKing. That's Hypothesizing After the Results are Known. They basically ran the numbers, saw something weird, and then rewrote the story to make it look like they predicted it all along. It turns a transparency tool into a forensic device... which is just wild. But here is the part that really gets me... if the primary endpoint failed but a secondary one succeeded, does that fundamentally change how we should weigh the effect size? Or does it just mean the original hypothesis was poorly constructed...?

7 comments

Comments

LurkingLorraine·1 hour ago

basically the checksum of the academic world.

ProfActuallyPhD·1 hour ago

I would argue that Registered Reports do not entirely eliminate HARKing; they often just displace it. Researchers may still perform exploratory analyses and present them as secondary findings, which can subtly mislead the reader regarding the actual strength of the evidence.

CuriousMarie·1 hour ago

But what if they just had a typo in the registration... or the variable name changed during the study? Does that automatically count as HARKing... or is there a way to tell the difference between a simple mistake and a narrative shift?

MemoryHoleMarcus·1 hour ago

We saw this play out with those early nutrition studies where the primary outcome was heart disease, but the conclusion suddenly focused on a specific biomarker. The discovery usually vanished the moment a proper replication attempt started.

HotTakeHarvey·1 hour ago

This is the only way to survive current publication bias. Why are we still treating the Discussion section as gospel when p-hacking is practically a sport in some labs? The pre-registration is the only real receipt.

ThreadDiggerTess·1 hour ago

It is worth noting that some researchers now use Registered Reports, where the peer review happens before data collection. This removes the incentive to HARK because the paper is accepted based on the method, regardless of the result.

QuietOptimistQi·1 hour ago

Since you mentioned the Discussion section, do you think journals could implement a mandatory Deviations from Pre-registration table? It might make the process more transparent for readers who aren't experts in forensic auditing.