Convolutional Neural Networks (CNNs) have been proven to be extremely successful at solving computer vision tasks. State-of-the-art methods favor such deep network architectures for its accuracy performance, with the cost of having massive number of parameters and high weights redundancy. Previous works have studied how to prune such CNNs weights. In this paper, we go to another extreme and analyze the performance of a network stacked with a single convolution kernel across layers, as well as other weights sharing techniques. We name it Deep Anchored Convolutional Neural Network (DACNN). Sharing the same kernel weights across layers allows to reduce the model size tremendously, more precisely, the network is compressed in memory by a factor of L, where L is the desired depth of the network, disregarding the fully connected layer for prediction. The number of parameters in DACNN barely increases as the network grows deeper, which allows us to build deep DACNNs without any concern about memory costs. We also introduce a partial shared weights network (DACNN-mix) as well as an easy-plug-in module, coined regulators, to boost the performance of our architecture. We validated our idea on 3 datasets: CIFAR-10, CIFAR-100 and SVHN. Our results show that we can save massive amounts of memory with our model, while maintaining a high accuracy performance.
翻译:事实证明,革命神经网络(CNNs)在解决计算机愿景任务方面非常成功。 最先进的方法有利于这种深网络结构的精确性能,其成本是大量的参数和高重量冗余。 以前的工程研究如何淡化这类CNN的重量。 在本文中,我们进入另一个极端,分析一个网络的性能,这个网络堆积在一层层之间,以及其它重力共享技术。 我们命名它为深层反动神经网络(DACNNNN)。 在不同层次共享相同的内核重量可以大大、更准确地减少模型的精确性能。 网络在记忆中压缩了一个要素L,L是网络的理想深度,无视完全相连的预测层。 随着网络越深,DACNNNN的参数数量几乎没有增加,这使我们能够在记忆成本方面建立更深的CDCNNN(DNN-Mix)网络(DACNN-Mix)。 我们还引入一个部分共享的重量网络,以及一个容易插入的模块, 更精确地将网络压缩成型的内积缩缩缩缩缩缩,L是网络的深度, L是网络的精度,, 无视网络的深度的网络的深度,我们10RRRRRAFAR的大规模结构可以展示我们的数据。