In this paper, we investigate the effects of releasing arXiv preprints of papers that are undergoing a double-blind review process. In particular, we ask the following research question: What is the relation between de-anonymization of authors through arXiv preprints and acceptance of a research paper at a (nominally) double-blind venue? Under two conditions: papers that are released on arXiv before the review phase and papers that are not, we examine the correlation between the reputation of their authors with the review scores and acceptance decisions. By analyzing a dataset of ICLR 2020 and ICLR 2019 submissions (n=5050), we find statistically significant evidence of positive correlation between percentage acceptance and papers with high reputation released on arXiv. In order to understand this observed association better, we perform additional analyses based on self-specified confidence scores of reviewers and observe that less confident reviewers are more likely to assign high review scores to papers with well known authors and low review scores to papers with less known authors, where reputation is quantified in terms of number of Google Scholar citations. We emphasize upfront that our results are purely correlational and we neither can nor intend to make any causal claims. A blog post accompanying the paper and our scraping code will be linked in the project website https://sites.google.com/view/deanon-arxiv/home
翻译:在本文中,我们调查了正在接受双盲审查的论文的ArXiv预印本的影响。我们特别询问以下研究问题:通过ArXiv预印本和在(名义上)双盲地点接受研究论文,作者的去匿名文件之间有何关系?在两个条件下:在审查阶段前在ArXiv发布的论文和没有的论文,我们研究其作者的声誉与审查评分和接受决定之间的关系。通过分析2020年ICLR和2019年ICLR提交材料(n=5050)的数据集,我们发现在ArXiv上公布的百分比接受率和高声望论文之间具有积极相关性的统计重要证据。为了更好地了解所观察到的这种关联性,我们根据自我指定的审查者信心评分和不那么自信的评审者更有可能给知名作者的论文分配高评分,而低评分则给不那么知名的作者的论文,在Google学者引用的次数中可以量化声誉。我们强调,我们的结果与在A/com网站上没有任何因果关系,我们也不打算与A号博客的代码挂钩。