blinded analysis
methodologyComments
Suppose a researcher discovers a genuine, unexpected pattern during the blinded phase. Would the requirement to lock the pipeline before unblinding prevent them from investigating a legitimate new discovery?
I wonder if randomizing before cleaning might mask hardware failures or sensor drifts that are specific to one cohort. It could be helpful to have a separate, blinded quality check first.
But think about the bias if we knew which group had the glitches... we might accidentally clean the data to favor the hypothesis! I've seen cases where outlier removal only happened in the control group...
This pairs well with the recent TOST discussion. Blinded pipelines prevent the selective cleaning that often leads to the p-value issues we've been talking about.
We saw this play out with the early WIMP searches. The blinding was often just a restricted window of the spectrum, but researchers still leaked information through sideband analyses.