The water in the animation moves beautifully. It curls around the bend, touches the bank and reaches the bridge just where the model says it should. The screen is convincing. But what, exactly, has been tested?
A simulation produces consequences from a representation and its rules. Whether those consequences describe a particular river is another question. To answer it, we need more than a successful run.
It is not a contest between computers and observation. It is a question about the bridge between them.
First ask what the program was meant to do
Imagine a deliberately simple model of water moving through a narrow channel. Its rules may be useful for exploring one condition without representing every eddy, obstacle or change in the riverbed. Simplification is not automatically a defect. It becomes a problem when a reader is never told what was left out.
Before accepting the animation as an answer, ask what decision it is intended to support. Comparing two possible channel shapes is not the same task as forecasting a flood at a particular home. Different uses can demand different evidence.
A correct calculation can answer the wrong question
NASA’s Standard for Models and Simulations, revision B, approved in March 2024, distinguishes verification from validation. In plain language, verification concerns whether the model has been implemented correctly and its numerical solutions checked; validation concerns how adequately it represents the real world for its intended use.
These questions support one another. Checking that the code follows its equations matters. It does not, by itself, establish that those equations adequately represent the river you intend to discuss. Nor does one attractive comparison excuse a programming error.
A model needs both a working engine and a reason to trust its destination.
Matching yesterday is not the same as predicting tomorrow
Suppose you adjust a channel model until its output closely resembles measurements from one afternoon. That agreement is worth investigating. But if the same observations helped choose the settings, they are not a fully independent challenge to those choices.
Now hold those settings fixed and compare the model with another suitable set of observations. Explain what counts as an acceptable difference before celebrating the match. Describe the measurement uncertainty and whether the new conditions remain within the range the model was built to address. A mismatch may reveal a bad assumption, an implementation problem, poor inputs or a limit of the chosen simplification. It does not identify its own cause.
More runs do not add missing observations
Running the model under different inputs can reveal how sensitive its answer is to a particular assumption. Repeating a stochastic simulation can help characterize variation within that simulation. These are useful questions about the model’s behavior.
But increasing the run count does not automatically supply evidence about the outside world. If the connection between an input and its real-world counterpart is unsupported, a thousand extra runs can explore that unsupported choice very precisely.
The opposite mistake is to dismiss simulation altogether. Models can clarify consequences that are difficult to reason through unaided. Their value grows when the claim stays proportionate to the test.
Read the claim beside the picture
When a result catches your attention, look for the intended use, the assumptions, the observations used for comparison and the conditions where the authors advise caution. Our guide to facts and interpretations offers a related distinction; the question here is specifically how a computational result reaches beyond its own rules.
Return to the moving water on the screen. You need not choose between admiration and skepticism. You can appreciate the work and still ask which parts were checked against something the model did not create. A simulation earns a bounded claim through the evidence connecting it to its use, not through the confidence of its presentation.
Keep the river larger than the screen.
Original AI-generated conceptual illustration of a channel model; not a record of a physical experiment or a simulation result.
M.R. Orion
