The use and abuse of Significance Testing ... - Significance tests (part 3)

Описание: Intermingled in today’s statistical controversies are some long-standing, but unresolved, disagreements on the nature and principles of statistical methods and the roles for probability in statistical inference and modelling. In reaction to the so-called “replication crisis” in the sciences, some reformers suggest significance tests as a major culprit. To understand the ramifications of the proposed reforms, there is a pressing need for a deeper understanding of the source of the problems in the sciences and a balanced critique of the alternative methods being proposed to supplant significance tests. In this session speakers offer perspectives on significance tests from statistical science, econometrics, experimental psychology and philosophy of science. There will be also be panel discussion.

Speaker: Aris Spanos, Virginia Tech

Fisher’s significance testing is placed in a broader frequentist testing context with a view to shed light on the merits and demerits of its application in several different contexts. It is argued that the current discussions of the use and abuse of the p-value and the Neyman-Pearson (N-P) accept/reject rules as a basis for providing evidence for or against particular hypotheses need to be clarified by drawing three crucial distinctions: (i) the modeling vs. the inference facet of statistical analysis, (ii) testing within and testing outside the boundaries of a statistical model, and (iii) statistical vs. substantive model validation. Focusing on the use of significance testing in statistical model validation, it is argued that it can be modified to provide a sound basis for statistical misspecification testing [testing outside] with a view to establish the adequacy of a statistical model: the validity of its probabilistic assumptions vis-à-vis the particular data. Indeed, a modified form of significance testing can suggest ways to respecify a misspecified statistical model with a view to account for the overlooked systematic statistical information in the data.