Replication studies are essential for assessing the credibility of claims from original studies. A critical aspect of designing replication studies is determining their sample size; a too small sample size may lead to inconclusive studies whereas a too large sample size may waste resources that could be allocated better in other studies. Here we show how Bayesian approaches can be used for tackling this problem. The Bayesian framework allows researchers to combine the original data and external knowledge in a design prior distribution for the underlying parameters. Based on a design prior, predictions about the replication data can be made, and the replication sample size can be chosen to ensure a sufficiently high probability of replication success. Replication success may be defined through Bayesian or non-Bayesian criteria, and different criteria may also be combined to meet distinct stakeholders and allow conclusive inferences based on multiple analysis approaches. We investigate sample size determination in the normal-normal hierarchical model where analytical results are available and traditional sample size determination is a special case where the uncertainty on parameter values is not accounted for. An application to data from a multisite replication project of social-behavioral experiments illustrates how Bayesian approaches help to design informative and cost-effective replication studies. Our methods can be used through the R package BayesRepDesign.
翻译:复制研究对于评估原始研究索赔的可信度至关重要。设计复制研究的一个关键方面是确定其抽样规模;过小的样本规模可能导致无结果的研究,而过大的样本规模可能会浪费资源,而其他研究可以更好地分配这些资源。这里我们展示了如何利用巴伊西亚办法解决这一问题。巴伊西亚框架允许研究人员将原始数据和外部知识结合到原始数据的设计中,并事先分发基本参数的外部知识。根据事先设计,可以对复制数据作出预测,并且可以选择复制样本规模,以确保复制成功的可能性足够高。通过巴伊西亚或非巴耶斯标准来界定复制成功,而不同的标准也可以结合起来,以满足不同的利益攸关方,并允许根据多重分析方法作出结论性推论。我们在有分析结果和传统的样本规模确定是没有考虑到参数值不确定性的特殊案例的情况下,在正常的等级模型中进行抽样规模的确定。对多点复制项目的社会损害实验数据的应用可以说明Bayesian方法如何有助于设计信息化和成本效益强的复制一揽子研究。我们的方法可以通过采用的方法。