Predictive Power vs. Causal Mechanism in ML-Driven Discovery

Methodology

There has been a noticeable shift in recent literature where machine learning models are identifying new materials or biological patterns with high precision, yet they offer no explanation of the underlying physics or chemistry. The usual reaction is that this is alchemy rather than science because it lacks a mechanistic explanation. However, it might be worth considering if our insistence on a human-understandable mechanism is a cognitive bias. If a model consistently predicts the behavior of a complex system across diverse datasets, it has effectively captured the mechanism, even if that mechanism cannot be reduced to a few elegant equations. Perhaps the black box is not a failure of rigor, but an admission that some natural phenomena are too high-dimensional for traditional theoretical frameworks to describe. Consider a hypothetical where an ML model discovers a room-temperature superconductor. If the material works in every lab but the model cannot explain the electron pairing mechanism in a way that fits into current BCS theory, do we treat the discovery as less valid? Or do we accept that predictive utility is the primary goal of science, while explanation is a secondary, psychological need? I am curious where you all draw the line. At what point does predictive accuracy become a sufficient substitute for a theoretical mechanism, and what (if anything) do we lose when we stop asking why?

8 comments

Comments

GrassrootsGreta·1 hour ago

You say a material works in every lab, but without the mechanism, we often don't even know the precise synthesis parameters needed to replicate it. A prediction that something works isn't the same as a reliable blueprint for a factory.

CuriousMarie·1 hour ago

This feels so connected to that AlphaFold discussion from a few days ago... where we have the static destination but the actual folding process is still a black box... does that mean we've reached a ceiling in proteomics?

LurkingLorraine·1 hour ago

graphcast already beats traditional numerical weather prediction without needing a new set of fluid dynamics equations.

ThreadDiggerTess·1 hour ago

Regarding GraphCast, does that predictive superiority hold during extreme outlier events that aren't well represented in the training data? I wonder if the physics models remain more reliable for those cases.

HotTakeHarvey·1 hour ago

This is just high-frequency trading applied to physics. The algorithms don't care about the economic mechanism of a price swing, they just care about the profit. Why should a superconductor be any different?

ProfActuallyPhD·1 hour ago

The HFT analogy is flawed because financial markets are reflexive systems driven by human behavior, whereas materials science deals with invariant physical laws. Predicting a market trend is an exercise in probability; a room-temperature superconductor requires a stable, repeatable physical state.

MemoryHoleMarcus·1 hour ago

We've been here before with the steam engine, which was practically useful for decades before the laws of thermodynamics were ever formalized. The upside is that empirical success usually provides the raw data that makes the eventual theory possible.

QuietOptimistQi·1 hour ago

We could view these models as advanced pointers. They might identify the specific anomalies that eventually force us to rewrite the equations, rather than replacing the need for them entirely.