While deep neural networks have achieved state-of-the-art performance across a large number of complex tasks, it remains a big challenge to deploy such networks for practical, on-device edge scenarios such as on mobile devices, consumer devices, drones, and vehicles. In this study, we take a deeper exploration into a human-machine collaborative design approach for creating highly efficient deep neural networks through a synergy between principled network design prototyping and machine-driven design exploration. The efficacy of human-machine collaborative design is demonstrated through the creation of AttoNets, a family of highly efficient deep neural networks for on-device edge deep learning. Each AttoNet possesses a human-specified network-level macro-architecture comprising of custom modules with unique machine-designed module-level macro-architecture and micro-architecture designs, all driven by human-specified design requirements. Experimental results for the task of object recognition showed that the AttoNets created via human-machine collaborative design has significantly fewer parameters and computational costs than state-of-the-art networks designed for efficiency while achieving noticeably higher accuracy (with the smallest AttoNet achieving ~1.8% higher accuracy while requiring ~10x fewer multiply-add operations and parameters than MobileNet-V1). Furthermore, the efficacy of the AttoNets is demonstrated for the task of instance-level object segmentation and object detection, where an AttoNet-based Mask R-CNN network was constructed with significantly fewer parameters and computational costs (~5x fewer multiply-add operations and ~2x fewer parameters) than a ResNet-50 based Mask R-CNN network.
翻译:虽然深心神经网络在大量复杂任务中取得了最先进的性能,但部署这种网络以实际的、现成的边缘情景,如移动设备、消费者装置、无人驾驶飞机和车辆,仍然是一项巨大的挑战。在本研究中,我们更深入地探索了一种人体机器协作设计方法,通过有原则的网络设计设计原型和机器驱动的设计探索之间的协同作用,建立高效的深神经网络。通过创建AttoNets,显示人体机器合作设计的效率。AttoNets是一套高高效的深心神经网络,用于在目标边缘进行深思熟虑的深思熟虑学习。每个AttoNet都拥有一个人为的网络级参数,网络级宏观结构,包括由独特的机器设计的模块级宏观建筑和微型结构设计构成的定制模块模块设计,从而创建出高效的深心神经网络网络。 物体识别任务的实验结果显示,通过人体机器协作设计创建的AttoNet的参数和计算成本比设计用于效率的状态网络的参数和计算成本要小得多,而用于显著的NCN-50级的网络级的网络级网络运行,其精确度要达到最精度。