In this paper, we present a new perspective towards image-based shape generation. Most existing deep learning based shape reconstruction methods employ a single-view deterministic model which is sometimes insufficient to determine a single groundtruth shape because the back part is occluded. In this work, we first introduce a conditional generative network to model the uncertainty for single-view reconstruction. Then, we formulate the task of multi-view reconstruction as taking the intersection of the predicted shape spaces on each single image. We design new differentiable guidance including the front constraint, the diversity constraint, and the consistency loss to enable effective single-view conditional generation and multi-view synthesis. Experimental results and ablation studies show that our proposed approach outperforms state-of-the-art methods on 3D reconstruction test error and demonstrate its generalization ability on real world data.
翻译:在本文中,我们展示了对基于图像的形状生成的新视角。大多数现有的基于深层学习的形状重建方法都采用了单一视角的确定型模型,该模型有时不足以确定单一的地貌真实性,因为后半部被隐蔽。在这项工作中,我们首先引入一个有条件的基因化网络,以模拟单一视角重建的不确定性。然后,我们将多视角重建的任务设计成每个图像上预测的形状空间的交叉点。我们设计了新的差异性指南,包括前端制约、多样性制约和一致性损失,以便能够实现有效的单一视角有条件的生成和多视图合成。实验结果和模拟研究表明,我们拟议的方法在3D重建测试错误方面优于最先进的方法,并展示其在真实世界数据上的通用能力。