This paper deals with the problem of 3D tracking, i.e., to find dense correspondences in a sequence of time-varying 3D shapes. Despite deep learning approaches have achieved promising performance for pairwise dense 3D shapes matching, it is a great challenge to generalize those approaches for the tracking of 3D time-varying geometries. In this paper, we aim at handling the problem of 3D tracking, which provides the tracking of the consecutive frames of 3D shapes. We propose a novel unsupervised 3D shape registration framework named DeepTracking-Net, which uses the deep neural networks (DNNs) as auxiliary functions to produce spatially and temporally continuous displacement fields for 3D tracking of objects in a temporal order. Our key novelty is that we present a novel temporal-aware correspondence descriptor (TCD) that captures spatio-temporal essence from consecutive 3D point cloud frames. Specifically, our DeepTracking-Net starts with optimizing a randomly initialized latent TCD. The TCD is then decoded to regress a continuous flow (i.e. a displacement vector field) which assigns a motion vector to every point of time-varying 3D shapes. Our DeepTracking-Net jointly optimizes TCDs and DNNs' weights towards the minimization of an unsupervised alignment loss. Experiments on both simulated and real data sets demonstrate that our unsupervised DeepTracking-Net outperforms the current supervised state-of-the-art method. In addition, we prepare a new synthetic 3D data, named SynMotions, to the 3D tracking and recognition community.
翻译:本文涉及 3D 跟踪问题, 即 3D 跟踪问题, 以在时间变化 3D 形状的序列中找到密密密的 3D 形状。 尽管深深层次的学习方法已经为双向密度 3D 形状匹配取得了有希望的性能, 但这些方法对于跟踪 3D 时间变化的映射来说是一个巨大的挑战。 在本文中, 我们的目标是处理 3D 跟踪问题, 提供 3D 形状连续框架的跟踪。 我们提议一个新型的 3D 形状注册框架, 名为 DeepTracking- Net, 使用深层的 神经网络( DNNS) 作为辅助功能, 以生成空间和时间持续迁移的 3D 形状匹配功能。 我们的主要新新新新版本的 3D 隐藏网络, 然后解码到当前 3D 移动方向的不连续的轨迹, 向每个方向的 D- droad 方向显示一个不连续的磁带 。