以深神经网络为基础的图像和视频流系统深神经网络增强:调查和未来方向 (Deep Neural Network-based Enhancement for Image and Video Streaming Systems: A Survey and Future Directions)

Internet-enabled smartphones and ultra-wide displays are transforming a variety of visual apps spanning from on-demand movies and 360{\deg} videos to video-conferencing and live streaming. However, robustly delivering visual content under fluctuating networking conditions on devices of diverse capabilities remains an open problem. In recent years, advances in the field of deep learning on tasks such as super-resolution and image enhancement have led to unprecedented performance in generating high-quality images from low-quality ones, a process we refer to as neural enhancement. In this paper, we survey state-of-the-art content delivery systems that employ neural enhancement as a key component in achieving both fast response time and high visual quality. We first present the components and architecture of existing content delivery systems, highlighting their challenges and motivating the use of neural enhancement models as a countermeasure. We then cover the deployment challenges of these models and analyze existing systems and their design decisions in efficiently overcoming these technical challenges. Additionally, we underline the key trends and common approaches across systems that target diverse use-cases. Finally, we present promising future directions based on the latest insights from deep learning research to further boost the quality of experience of content delivery systems.

翻译：互联网智能手机和超广范围的显示正在将各种视觉应用程序从点播电影和360=deg}视频和360=deg}视频转换为视频会议和现场直播。然而,在各种能力装置的网络条件波动的情况下,强有力地提供视觉内容仍是一个尚未解决的问题。近年来,在对诸如超分辨率和图像增强等任务进行深入学习方面取得的进步,导致在从低质量产品生成高质量图像方面取得了前所未有的业绩,我们称之为神经增强过程。在本文中,我们调查利用神经增强作为实现快速反应时间和高视觉质量的关键组成部分的先进内容提供系统。我们首先介绍现有内容提供系统的组成部分和结构,突出其挑战,并激励使用神经增强模型作为反制措施。然后我们介绍这些模型的部署挑战,分析现有系统及其设计决定,以有效克服这些技术挑战。此外,我们强调针对不同使用案例的系统的主要趋势和共同方法。最后,我们根据从深入研究研究到进一步提升交付系统的质量的最新见解,提出了有希望的未来方向。