Video frame interpolation, which aims to synthesize non-exist intermediate frames in a video sequence, is an important research topic in computer vision. Existing video frame interpolation methods have achieved remarkable results under specific assumptions, such as instant or known exposure time. However, in complicated real-world situations, the temporal priors of videos, i.e. frames per second (FPS) and frame exposure time, may vary from different camera sensors. When test videos are taken under different exposure settings from training ones, the interpolated frames will suffer significant misalignment problems. In this work, we solve the video frame interpolation problem in a general situation, where input frames can be acquired under uncertain exposure (and interval) time. Unlike previous methods that can only be applied to a specific temporal prior, we derive a general curvilinear motion trajectory formula from four consecutive sharp frames or two consecutive blurry frames without temporal priors. Moreover, utilizing constraints within adjacent motion trajectories, we devise a novel optical flow refinement strategy for better interpolation results. Finally, experiments demonstrate that one well-trained model is enough for synthesizing high-quality slow-motion videos under complicated real-world situations. Codes are available on https://github.com/yjzhang96/UTI-VFI.
翻译:视频框架内插,目的是在视频序列中合成无性别歧视的中间框,这是计算机视觉中的一个重要研究课题。现有的视频框架内插方法在特定假设下取得了显著成果,例如即时或已知的暴露时间。然而,在复杂的现实环境中,视频的时序前科,即每秒框架(FPS)和接触时间框架,可能与不同的摄像传感器不同。当测试视频在来自培训的不同接触环境中拍摄时,内插框架将遭遇严重的错位问题。在这项工作中,我们解决了一般情况下的视频框架内插问题,即输入框架可在不确定的暴露(和间隔)时间下获得。与以前只能应用于特定时间的方法不同,我们从四个连续的锐度框架或两个连续的模糊框架中得出一个一般的曲线运动轨迹公式,而没有时间间隔。此外,在临近的运动轨迹中,我们设计了一个新型的光流改进战略,以取得更好的内插结果。最后,实验表明,在一般情况下,一个经过良好训练的模型足以使高品质的慢压/慢压状态在真实的法规下同步状态上。