RSSScientific/Statistical Interpretation 101
April 23rd, 2008 | View Comments
A rant.
Please, for my sake, stop using the phrase “scientific proof.”
Science is about probabilities, not proof.
Absolute proof, the determination that something must be true in every situation with no exceptions, is only possible if you can construct your own universe. You prove things in mathematics (which is the language of perfection and does let you define your own universe), but we cannot prove things in science because we are stuck with the highly noisy and imperfect universe that we were born into.
The phrase “scientific proof” is an oxymoron. You will never see an actual scientist use this phrase in scientific writing. I cross this phrase out every single time I see it in my undergrads’ papers. Using this phrase tells me immediately that you have not been trained in scientific reasoning and that your scientific commentary should be accompanied by a salt lick.
Science relies on statistics, and statistics are about probabilities. A lot of people seem to equate the phrase “statistically significant” with “proof” and…WRONG. Statistically significant just means that there is only a small chance that this particular experimental result is spurious.
We usually use a threshold (alpha value) of .05 in science. If your statistical test turns up a p-value of .05 , it is statistically significant. It means that there is a 5% chance that you could have gotten this size effect, or a bigger effect, through random chance. There is a 5% chance that this is a false positive. Five percent. One in twenty.
If p is smaller, than there is less of a chance that the finding is random, but p is never zero.
So for every 100 experiments that are statistically significant at a value of p = .05, that means 5 of those findings are likely spurious.
This is why replication and corroborating evidence are the hallmarks of good science. The more times you can successfully repeat an experiment (or related/similar experiments), the more likely that you are onto something real.
The kinds of things that have been accepted as scientific fact are things that have been replicated so many times, that have been corroborated in so many ways, that the odds that these things are happening randomly are extremely low.
And the corollary is also true: if people have tried to demonstrate something experimentally and failed repeatedly, the odds that they are hunting a phantom are extremely high. The “absence of evidence is not evidence of absence” argument has been used to perpetrate many a nonexistent controversy, but this argument is only true within the bounds of formal logic (which is a branch of mathematics). In science, a series of failed studies (assuming said studies are well-designed and well-executed) is evidence that what you’re looking for just isn’t there.
End rant.
Yvonne posted this on April 23rd, 2008 @ 4:16pm in Science | Permalink to "Scientific/Statistical Interpretation 101"
Discussion
1 Comment
Keep track of the discussion! Subscribe to this post's RSS feed.
Have something to say? Leave your own comment!
| Trackback |
1. Fermi » May 5th, 2008 at 9:41 pm
Awesome post. One of my Profs went on a rant about how scientists don’t prove anything. Only Mathematicians prove things. I raised my hand and said “What about Philosophers, don’t they prove things?” To which my Prof said: “No, Philosophers don’t do anything.”