Proof-of-Learning (PoL) proposes that a model owner logs training checkpoints to establish a proof of having expended the computation necessary for training. The authors of PoL forego cryptographic approaches and trade rigorous security guarantees for scalability to deep learning. They empirically argued the benefit of this approach by showing how spoofing--computing a proof for a stolen model--is as expensive as obtaining the proof honestly by training the model. However, recent work has provided a counter-example and thus has invalidated this observation. In this work we demonstrate, first, that while it is true that current PoL verification is not robust to adversaries, recent work has largely underestimated this lack of robustness. This is because existing spoofing strategies are either unreproducible or target weakened instantiations of PoL--meaning they are easily thwarted by changing hyperparameters of the verification. Instead, we introduce the first spoofing strategies that can be reproduced across different configurations of the PoL verification and can be done for a fraction of the cost of previous spoofing strategies. This is possible because we identify key vulnerabilities of PoL and systematically analyze the underlying assumptions needed for robust verification of a proof. On the theoretical side, we show how realizing these assumptions reduces to open problems in learning theory.We conclude that one cannot develop a provably robust PoL verification mechanism without further understanding of optimization in deep learning.
翻译:Proof-of-Learning(PoL)提出了这样的构思,即模型拥有者记录训练检查点,以建立经过训练所需计算的证明。 PoL的作者放弃了加密方法,以便将深度学习的规模扩展到严格的安全保证。通过展示伪造的成本(即为获取模型进行了训练的证明)与诚实地训练模型以获得证明的成本相同,他们经验性地论证了这种方法的好处。然而,最近的一项工作提供了对这一观察的反例,从而使其失效。在这项工作中,我们首先证明,虽然当前的PoL验证对于对手缺乏强大的保护措施是正确的,但最近的工作主要低估了这种缺乏鲁棒性。这是因为现有的欺骗策略要么是不可重复的,要么是针对PoL的削弱实例,这意味着它们很容易通过更改验证的超参数来防范。相反,我们介绍了首批可以在PoL验证的不同配置下重现的欺骗策略,并且只需完成比以前少得多的欺骗策略的一小部分即可完成。这是可能的,因为我们确定了PoL的关键漏洞,并系统地分析了验证证明所需的基本假设。在理论方面,我们展示了如何实现这些假设,这减少了学习理论中的开放性问题。我们得出结论:除非进一步了解深度学习中的优化,否则不能开发出可靠的PoL验证机制。