… to test between good and bad science
… all models only say what they are told to say.
Models are lists of statements of the form “If this, then that”. No matter how large they grow, or how sophisticated, or how mathematical, or how computerized, or how much data that is put into them, or from what sources, their natures are not altered. They are always lists of “If this, then that.”
… Here is a simple, common, and most useful model, used by casinos the world over: “If this die has six sides, then the probability any side is up in a throw is one in six.”
That model says exactly what we want it to say, and only what we told it to say. It is an accurate model, too. It matches reality well; indeed, it makes beautiful predictions…
The model says nothing, not one thing, about what causes any side to be up on any toss. Efficient cause cannot be inferred from examining the model. No cause was built into the model. That is, none of the “If this, then that” statements (and there is only one) mentioned cause (except part of the formal cause, that the object has six sides). But the model is still good.
We conclude that models can be good and useful yet be silent on cause. The opposite is also true: a model can perform well in practice, but we cannot from that good performance conclude it has identified the cause of things. Ensuring cause has been identified is a much more difficult task…
Since all models only say what they are told to say, we can always create a model to say anything we want…
We have the freedom to specify the “If this” parts of the model, from which we sometimes can deduce, and sometimes must guess, the “Then that” parts. Or we can work backward, starting from desirable “Then that”, and picking compatible “If this” parts.
We have the freedom to say which and how much and from where the “data” goes into the model, and what “If this, then that” they are married to. We have the freedom to embrace any simplifications or approximations we want. We can, and an increasing minority even do, cheat…
This freedom comes with a cost. Since any model can be made to say anything at all, it means models can’t really be trusted until they are tested against reality.
Models certainly cannot be trusted because of the authority of who builds, or rather creates, them. That is a fallacy. And they can’t be trusted because “We need to do something and there is nothing else.” That is also a fallacy: there are always other options… [full article]
William M Briggs
Related: COVID: Lockdown Mistakes