Deep neural networks (DNNs) are known for their inability to utilize underlying hardware resources due to hardware susceptibility to sparse activations and weights. Even in finer granularities, many of the non-zero values hold a portion of zero-valued bits that may cause inefficiencies when executed on hardware. Inspired by conventional CPU simultaneous multithreading (SMT) that increases computer resource utilization by sharing them across several threads, we propose non-blocking SMT (NB-SMT) designated for DNN accelerators. Like conventional SMT, NB-SMT shares hardware resources among several execution flows. Yet, unlike SMT, NB-SMT is non-blocking, as it handles structural hazards by exploiting the algorithmic resiliency of DNNs. Instead of opportunistically dispatching instructions while they wait in a reservation station for available hardware, NB-SMT temporarily reduces the computation precision to accommodate all threads at once, enabling a non-blocking operation. We demonstrate NB-SMT applicability using SySMT, an NB-SMT-enabled output-stationary systolic array (OS-SA). Compared with a conventional OS-SA, a 2-threaded SySMT consumes 1.4x the area and delivers 2x speedup with 33% energy savings and less than 1% accuracy degradation of state-of-the-art CNNs with ImageNet. A 4-threaded SySMT consumes 2.5x the area and delivers, for example, 3.4x speedup and 39% energy savings with 1% accuracy degradation of 40%-pruned ResNet-18.
翻译:深心神经网络(DNNS)因无法利用基本硬件资源而闻名于其无法使用,因为硬件容易受到稀疏的激活和重量。即便在细微的颗粒中,许多非零值也持有零价位,在硬件执行时可能导致效率低下。受常规的CPU同步同时多读(SMT)的启发,通过将计算机资源共享到多个线索来增加计算机资源的利用,我们建议为DNN的加速器指定不阻塞SMT(NB-SMT)。像常规SMT一样,40NB-SMT在一些执行流中共享硬件资源。然而,与SMT不同的是,NB-SMT是无阻的,NB-SMT是非阻塞的,而NB-SMT是非阻塞的,因为它通过利用DNPNS-S-S Riality 2 SDMT,而不是常规的SA-SBS-S-SBS-MT IMS-S-S-MT IMD IMD IML IMD IMD IMD IMD IMS-R IMD 和S-MS-MS-MS-MS-MS-MS-MS-R IMD IMD IMS-MS-S-MS-S-R IMF-MS-MT-MT-MS-MT-R IMS-MT-MT-MT-MT-R IMS-MT-MT-S-MT IMS-S-MT-MT IMS-R IMS-S-S-R-R-R-S-S-S-R-R-R-R IMS-MT-MT-MT-R IMS-MT-MT-MT-MT IMS-MT-S-S-S-S-S-R IMS-R IMS-S-S-S-S-S-S-S-S-S-S-S-R IMS-R-R-R-R-R-R-R-R-R-R