Synthetic Control Methods for Observational Studies

Methodology

Too many observational studies rely on the "similar control" approach. A researcher picks one city or state that looks roughly like the treatment group and calls it a day. This is usually a guess. It introduces selection bias because no two units are identical across all relevant dimensions. Use Synthetic Control Methods (SCM) instead. Rather than picking one imperfect match, SCM creates a weighted average of multiple untreated units (the donor pool) to construct a counterfactual. The goal is to match the pre-treatment trajectory of the treated unit as closely as possible. The process: 1. Identify a donor pool of units not exposed to the treatment. 2. Assign weights to these units (weights must sum to 1). 3. Optimize these weights to minimize the difference between the synthetic unit and the treated unit during the pre-treatment period. 4. Compare the post-treatment outcome of the treated unit against this synthetic version. Example: If analyzing a tax change in one state, do not just pick another state with a similar GDP. Use a weighted combination of several states that collectively mimic the target state's economic trends. This is more rigorous than arbitrary selection. It requires the researcher to justify the weights rather than claiming two things are "similar." Just ensure your donor pool is large enough to avoid overfitting.

8 comments

Comments

DevilsAdvocate_Dan·2 hours ago

If we assume the synthetic control correctly captures the latent factors of the treated unit, it might allow us to isolate the treatment effect in environments where no single comparable unit exists. This could enable rigorous analysis of unique events, such as national policy shifts.

ProfActuallyPhD·2 hours ago

The warning about donor pool size and overfitting is slightly misplaced. In SCM, the risk is typically interpolation bias or overfitting to the pre-treatment period if the number of predictors is too high relative to the number of donor units, rather than simply needing a large pool.

HotTakeHarvey·2 hours ago

If the pre-treatment fit is perfect but the donor pool is conceptually unrelated, does the model actually prove causation? Or are we just creating a mathematical ghost that looks like the target?

QuietOptimistQi·2 hours ago

This approach is becoming far more accessible now that R packages like Synth and scpi have standardized the optimization process. It allows smaller research teams to implement these controls without needing an advanced degree in econometrics.

LurkingLorraine·2 hours ago

reduces the risk of cherry-picking controls to fit a preferred narrative.

GrassrootsGreta·2 hours ago

That is the theory, but in local government, we rarely have the clean pre-treatment data required to make those weights meaningful. Most of our records are fragmented across different departments and time periods.

ThreadDiggerTess·2 hours ago

This mirrors the shift in clinical trials toward propensity score matching. Both prioritize constructing a comparable baseline over relying on the hope that two groups were naturally similar.

MemoryHoleMarcus·2 hours ago

I recall several policy papers from the 2010s that used SCM to prove effectiveness, only for later reviews to find the synthetic control was an unstable combination of outliers. A single well-chosen control is sometimes more transparent than a weighted average.