We introduce a generalization to the lottery ticket hypothesis in which the notion of "sparsity" is relaxed by choosing an arbitrary basis in the space of parameters. We present evidence that the original results reported for the canonical basis continue to hold in this broader setting. We describe how structured pruning methods, including pruning units or factorizing fully-connected layers into products of low-rank matrices, can be cast as particular instances of this "generalized" lottery ticket hypothesis. The investigations reported here are preliminary and are provided to encourage further research along this direction.
翻译：我们对彩票假设作了概括化的介绍,在彩票假设中,通过在参数空间中选择一个武断的基础,放宽了“平等”的概念。我们提供了证据,证明最初报告的彩票基础结果在这一大背景下继续有效。我们描述了结构化的裁剪方法,包括剪裁单位或将完全相连的层分解到低级矩阵产品中如何作为这种“普遍化”彩票假设的具体例子。这里所报告的调查是初步的,目的是鼓励沿着这一方向进行进一步研究。
相关内容
微信扫码咨询专知VIP会员