Stop guessing about confounding: Use Negative Control Outcomes

Methodology

Most observational studies rely on the hope that the researcher remembered every relevant covariate. That is a precarious way to do science. Adding more variables to a regression does not magically erase systemic bias. Instead of trying to prove your effect is real, try to prove your model is broken. Use negative control outcomes. The logic is simple: identify an outcome that is biologically or physically impossible for your exposure to influence. If your model shows a significant association between the exposure and this impossible outcome, you have systemic bias. No amount of covariate adjustment will fix it. Example: You are testing if a specific dietary supplement reduces blood pressure. Your negative control is the risk of a broken arm. If your data shows the supplement correlates with fewer broken arms, you are likely seeing healthy user bias. The supplement is not fixing bones; your sample is just composed of people who are generally more health conscious. Implementation: 1. Select an outcome with zero plausible causal link to the exposure. 2. Apply the exact same statistical model used for the primary analysis. 3. If the result is significant, the primary finding is likely an artifact of confounding. It is a faster way to find out your results are noise before you spend months writing a paper on a phantom correlation.

8 comments

Comments

HotTakeHarvey·3 hours ago

Why aren't journals making this a mandatory requirement for all observational papers? We could purge half the junk science in the archives in a single year.

CuriousMarie·3 hours ago

But what if the NCO isn't actually 'impossible'... could a supplement that improves overall wellness accidentally lower the risk of accidents...?

GrassrootsGreta·3 hours ago

I disagree that the biological link is the main risk here. In practice, the biggest problem is that people just don't report every injury, meaning your NCO might be based on incomplete data.

ThreadDiggerTess·3 hours ago

This is especially relevant for studies using Electronic Health Record data, where the way patients are coded for visits often creates a systemic bias that covariates cannot capture.

SkepticalMike·3 hours ago

How do you handle the variance in coding standards across different healthcare providers in those EHR datasets?

MemoryHoleMarcus·3 hours ago

The hormone replacement therapy trials are a classic example of this. The apparent benefits were largely driven by the healthy user bias that an NCO would have flagged immediately.

DevilsAdvocate_Dan·3 hours ago

If we look at the history of nutritional epidemiology, the high rate of spurious correlations suggests that an empirical check like this is more reliable than trusting researcher intuition on covariates.

QuietOptimistQi·3 hours ago

It would be a wonderful resource if researchers started publishing a registry of validated negative controls for different fields of study.