The aim of this work is to introduce MaRF, a novel framework able to synthesize the Martian environment using several collections of images from rover cameras. The idea is to generate a 3D scene of Mars' surface to address key challenges in planetary surface exploration such as: planetary geology, simulated navigation and shape analysis. Although there exist different methods to enable a 3D reconstruction of Mars' surface, they rely on classical computer graphics techniques that incur high amounts of computational resources during the reconstruction process, and have limitations with generalizing reconstructions to unseen scenes and adapting to new images coming from rover cameras. The proposed framework solves the aforementioned limitations by exploiting Neural Radiance Fields (NeRFs), a method that synthesize complex scenes by optimizing a continuous volumetric scene function using a sparse set of images. To speed up the learning process, we replaced the sparse set of rover images with their neural graphics primitives (NGPs), a set of vectors of fixed length that are learned to preserve the information of the original images in a significantly smaller size. In the experimental section, we demonstrate the environments created from actual Mars datasets captured by Curiosity rover, Perseverance rover and Ingenuity helicopter, all of which are available on the Planetary Data System (PDS).
翻译:这项工作的目的是推出一个能够利用来自漫游摄影机的一些图像收集集合成火星环境的新框架,即MARF。其设想是生成火星表面的三维场景,以应对行星表面探索中的关键挑战,例如:行星地质学、模拟导航和形状分析。虽然有不同的方法可以进行火星表面的三维重建,但它们依赖经典计算机图形技术,这些技术在重建过程中产生大量计算资源,在将重建推广到看不见的场景和适应来自漫游摄像头的新图像方面受到限制。拟议框架通过利用神经辐射场(NERFs)解决上述局限性,这种方法通过利用分散的图像优化连续的体积场功能来合成复杂场景。为了加快学习进程,我们用其神经图形原始(NGPs)取代了稀薄的轮廓图像集,这是一套固定长度的矢量,用来保存原始图像在大大小的尺寸上的信息。在实验部分中,我们展示了从实际火星数据数据集中创造出来的环境,这是由直观和直视系统所采集的。