In Search of a Better Reporting of Scientific Results: A Data Probability Language

Abstract

[Abstract] A recent paper published in Trends in Ecology and Evolution suggested a new alternative for the reporting of statistical results, using a language based on evidence against the null hypothesis. I agree that the reporting of null hypothesis statistical testing clearly needs improvement, but the proposal of an evidence-based language has several drawbacks: a) it goes back to the original Fisherian continuous interpretation of p-values, b) at the same time uses some loose categorizations and, c) most importantly, it may provide a wrong idea of what p-values actually are. By saying that there is very strong, strong, moderate, weak or little evidence of an effect, the reader gets the idea that p-values are providing Bayesian-type information on the probability of the null hypothesis given our data. However, p-values are only providing information on the probability of having obtained our data (or more extreme data), under the trueness of the null hypothesis. That is why I suggest reporting results using a data probability-based language, together with a previous and separate specification of the magnitude of the effects.