Despite remarkable recent progress on both unconditional and conditional image synthesis, it remains a long-standing problem to learn generative models that are capable of synthesizing realistic and sharp images from reconfigurable spatial layout (i.e., bounding boxes + class labels in an image lattice) and style (i.e., structural and appearance variations encoded by latent vectors), especially at high resolution. By reconfigurable, it means that a model can preserve the intrinsic one-to-many mapping from a given layout to multiple plausible images with different styles, and is adaptive with respect to perturbations of a layout and style latent code. In this paper, we present a layout- and style-based architecture for generative adversarial networks (termed LostGANs) that can be trained end-to-end to generate images from reconfigurable layout and style. Inspired by the vanilla StyleGAN, the proposed LostGAN consists of two new components: (i) learning fine-grained mask maps in a weakly-supervised manner to bridge the gap between layouts and images, and (ii) learning object instance-specific layout-aware feature normalization (ISLA-Norm) in the generator to realize multi-object style generation. In experiments, the proposed method is tested on the COCO-Stuff dataset and the Visual Genome dataset with state-of-the-art performance obtained. The code and pretrained models are available at \url{https://github.com/iVMCL/LostGANs}.
翻译:尽管在无条件和有条件图像合成方面最近取得了显著进展,但学习能够从可重新配置的空间布局(即,用图像胶合盒+类标签)和风格(即,用隐性矢量编码结构和外观变异)合成现实和尖锐图像的基因模型,特别是在高分辨率方面,仍然是一个长期的问题。通过重新配置,一个模型可以保存从给定布局到具有不同风格的多种貌似图像的内在一对一映像,并且能够适应布局和样式潜伏代码的扰动。在本文件中,我们为基因化对抗网络(称为LostGANs)提供一个基于布局和风格的基于风格的结构架构和风格架构架构,可以训练端对端通过可重新配置的布局和风格生成图像的图像。在Vanilla StyleGAN的启发下,拟议的LostGAN由两个新组成部分组成:(i)以较弱的超强方式学习精细刻度的面码面框图,以缩小版图和样式潜透度的图像之间的距离。A-ANSlus-deal-deal-dealstal-dal-dal-dalmatial-dal-dal-dal-dal-dalmagister-dal-dal-dal-dalmagistration) 和Sildal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-dal-d-d-d-d-s-dal-dal-dal-dal-s-dal-s-d-d-d-d-d-d-I) 和Smadal-d-d-dal-smadal-smadal-smadal-dal-dal-s-I和Lii-inal-inal-s-s-s-s-inal-I和Smadal-inal-I和Smad-inal-inal-inal-inal-inal-inal-d-