The ever-increasing computational demand of Deep Learning has propelled research in special-purpose inference accelerators based on emerging non-volatile memory (NVM) technologies. Such NVM crossbars promise fast and energy-efficient in-situ matrix vector multiplications (MVM) thus alleviating the long-standing von Neuman bottleneck in today's digital hardware. However the analog nature of computing in these NVM crossbars introduces approximations in the MVM operations. In this paper, we study the impact of these non-idealities on the performance of DNNs under adversarial attacks. The non-ideal behavior interferes with the computation of the exact gradient of the model, which is required for adversarial image generation. In a non-adaptive attack, where the attacker is unaware of the analog hardware, we show that analog computing offers a varying degree of intrinsic robustness, with a peak adversarial accuracy improvement of 35.34%, 22.69%, and 31.70% for white box PGD ($\epsilon$=1/255, iter=30) for CIFAR-10, CIFAR-100, and ImageNet(top-5) respectively. We also demonstrate "hardware-in-loop" adaptive attacks that circumvent this robustness by utilizing the knowledge of the NVM model. To the best of our knowledge, this is the first work that explores the non-idealities of analog computing for adversarial robustness at the time of submission to NeurIPS 2020.
翻译:深学习的计算需求不断增加,这推动了基于新兴的非挥发性内存(NVM)技术的特殊目的推断加速器的研究。 NVM交叉条承诺快速和节能的现场矩阵矢量倍增(MVM),从而缓解了当今数字硬件中长期存在的冯纽曼瓶颈。然而,NVM交叉条中的模拟计算性质在MVM操作中引入了近似值。在本文中,我们研究了这些非理想性加速器对DNN在对抗性攻击下的表现的影响。这种非理想行为干扰了模型精确的梯度的计算,而这种模型是生成对抗性图像所需的。在非适应性攻击者不知道模拟硬件的情况下,我们表明模拟计算提供了不同程度的内在稳健性,白框 PGD (=1/255) 的顶峰值精确度改进了35.34 % 22.69% 和31.70% 白框 PGDD (=1/255) 。 非理想行为干扰行为干扰了计算模型精确度的精确度的精确度, 也就是在CIFAR-10号中, CIFAR-N-N-RO-O-RO-O-O-OAS-OAS-O 最佳知识中, 10号, 也分别展示了我们最强的模型-ROFAR-O-O-S-I-O-O-O-O-O-I-O-O-I-O-O-OFAS-O-O-O-I-I-I-I-I-I-I-I-IFAR-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I