Explaining shrinkage and heterogeneity in large-scale replication projects - Rachel Heyard

Описание: Abstract: Recent large-scale replication projects (RPs) have estimated concerningly low reproducibility rates. Further, all reported substantial degrees of shrinkage of effect size, where the replication effect size was found to be, on average, much smaller than the original effect size. Within these RPs, the original-replication study pairs can vary substantially with respect to aspects of study design, outcome measures, and descriptive features of both the original and replication study population and study team. When broader claims about the reproducibility of an entire field are based on such heterogeneous data, it becomes imperative to conduct a rigorous analysis of the amount of shrinkage and heterogeneity within and between original-replication study pairs included in the RP. Methodology from the meta-analysis literature provides an approach for quantifying the heterogeneity present in RPs, as additive or multiplicative parameter. Meta-analysis methodology further allows for an investigation of the sources of shrinkage and heterogeneity through meta-regression. We propose the use of location-scale meta-regressions as a means to directly relate the identified characteristics with shrinkage (represented by the location) and the heterogeneity variance (represented by the scale).

Bio: Rachel Heyard holds a PhD in Biostatistics and is currently a postdoctoral fellow in meta-research at the Center for Reproducibile Sciene, University of Zurich. Her research interests include developing and testing new approaches for allocating research funding, statistical methods in research, and improving reproducibility and the uptake of Open Science.