With the success of deep learning, object recognition systems that can be deployed for real-world applications are becoming commonplace. However, inference that needs to largely take place on the `edge' (not processed on servers), is a highly computational and memory intensive workload, making it intractable for low-power mobile nodes and remote security applications. To address this challenge, this paper proposes a low-power (5W) end-to-end neuromorphic framework for object tracking and classification using event-based cameras that possess desirable properties such as low power consumption (5-14 mW) and high dynamic range (120 dB). Nonetheless, unlike traditional approaches of using event-by-event processing, this work uses a mixed frame and event approach to get energy savings with high performance. Using a frame-based region proposal method based on the density of foreground events, a hardware-friendly object tracking is implemented using the apparent object velocity while tackling occlusion scenarios. For low-power classification of the tracked objects, the event camera is interfaced to IBM TrueNorth, which is time-multiplexed to tackle up to eight instances for a traffic monitoring application. The frame-based object track input is converted back to spikes for Truenorth classification via the energy efficient deep network (EEDN) pipeline. Using originally collected datasets, we train the TrueNorth model on the hardware track outputs, instead of using ground truth object locations as commonly done, and demonstrate the efficacy of our system to handle practical surveillance scenarios. Finally, we compare the proposed methodologies to state-of-the-art event-based systems for object tracking and classification, and demonstrate the use case of our neuromorphic approach for low-power applications without sacrificing on performance.
翻译:随着深层学习的成功,可用于现实世界应用的物体识别系统正在变得司空见惯。然而,与使用逐项事件处理的传统方法不同,这项工作采用混合的框架和事件方法来节省高性能,使低功率移动节点和远程安全应用难以处理。为了应对这一挑战,本文件建议采用低功率(5W)端对端神经变形框架来进行物体跟踪和分类,使用基于事件的相机,这些相机具有理想的特性,如低电耗(5-14mW)和高动态范围(120dB)。然而,与使用逐项事件处理的传统方法不同,这项工作使用混合的框架和事件方法来节省能源,高性能。使用基于框架的区域提议方法来降低地面事件的密度,使用硬件友好的物体跟踪方法,同时处理隐蔽情景。对于跟踪对象的低功率分类,事件相机与IBM TrueNorth(IBM Reforlational Flationality State)连接起来,用于处理最多八项的物体情况,通过原始的轨迹监测方法,我们使用直径追踪系统,最后使用直径跟踪跟踪跟踪数据,以显示我们所收集的轨道数据。