In this paper, we address the problem of makeup transfer, which aims at transplanting the makeup from the reference face to the source face while preserving the identity of the source. Existing makeup transfer methods have made notable progress in generating realistic makeup faces, but do not perform well in terms of color fidelity and spatial transformation. To tackle these issues, we propose a novel Facial Attribute Transformer (FAT) and its variant Spatial FAT for high-quality makeup transfer. Drawing inspirations from the Transformer in NLP, FAT is able to model the semantic correspondences and interactions between the source face and reference face, and then precisely estimate and transfer the facial attributes. To further facilitate shape deformation and transformation of facial parts, we also integrate thin plate splines (TPS) into FAT, thus creating Spatial FAT, which is the first method that can transfer geometric attributes in addition to color and texture. Extensive qualitative and quantitative experiments demonstrate the effectiveness and superiority of our proposed FATs in the following aspects: (1) ensuring high-fidelity color transfer; (2) allowing for geometric transformation of facial parts; (3) handling facial variations (such as poses and shadows) and (4) supporting high-resolution face generation.
翻译:在本文中,我们处理化妆转移问题,目的是将化妆面的化妆品从参考面向源面表面移植,同时保存源面的身份; 现有的化妆转移方法在产生现实化的化妆面容方面取得了显著进展,但在颜色忠度和空间转换方面表现不佳; 为了解决这些问题,我们提议了一部新书《复合属性变异器》(FAT)及其用于高质量化妆转移的变异空间FAT; 从NLP的变换器中提取灵感, FAT能够模拟源面和参考面容之间的语义对应和互动,然后精确地估计和转移面部属性; 为了进一步促进面部的形状变形和变形,我们还将薄板样(TPS)纳入FAT,从而创建空间式FAT,这是除了颜色和纹理外能够转移几何属性的第一种方法。 广泛的定性和定量实验表明我们拟议的FAT在以下几个方面的有效性和优越性:(1) 确保高纤维色转移;(2) 允许支持面部面部面部的几何变形转换;(3) 处理高影变形(例如成影)。