Event cameras are biologically-inspired sensors that gather the temporal evolution of the scene. They capture pixel-wise brightness variations and output a corresponding stream of asynchronous events. Despite having multiple advantages with respect to traditional cameras, their use is partially prevented by the limited applicability of traditional data processing and vision algorithms. To this aim, we present a framework which exploits the output stream of event cameras to synthesize RGB frames, relying on an initial or a periodic set of color key-frames and the sequence of intermediate events. Differently from existing work, we propose a deep learning-based frame synthesis method, consisting of an adversarial architecture combined with a recurrent module. Qualitative results and quantitative per-pixel, perceptual, and semantic evaluation on four public datasets confirm the quality of the synthesized images.
翻译:事件相机是收集场景时间演变过程的生物感应器,它们捕捉像素的亮度变化,并产生相应的不同步事件流。尽管传统相机具有多种优势,但传统数据处理和视觉算法的有限适用性部分阻止了这些相机的使用。为此,我们提出了一个框架,利用事件相机的输出流来合成 RGB 框架,依靠一组初步或定期的彩色键框架和中间事件序列。与现有工作不同,我们提出了一种深层次的基于学习的框架合成方法,其中包括一个对抗性结构,与一个经常性模块相结合。四个公共数据集的定性结果和定量半像、感知和语义评估证实了合成图像的质量。