Training generative adversarial networks (GANs) with limited real image data generally results in deteriorated performance and collapsed models. To conquer this challenge, we are inspired by the latest observations, that one can discover independently trainable and highly sparse subnetworks (a.k.a., lottery tickets) from GANs. Treating this as an inductive prior, we suggest a brand-new angle towards data-efficient GAN training: by first identifying the lottery ticket from the original GAN using the small training set of real images; and then focusing on training that sparse subnetwork by re-using the same set. Both steps have lower complexity and are more data-efficient to train. We find our coordinated framework to offer orthogonal gains to existing real image data augmentation methods, and we additionally offer a new feature-level augmentation that can be applied together with them. Comprehensive experiments endorse the effectiveness of our proposed framework, across various GAN architectures (SNGAN, BigGAN, and StyleGAN-V2) and diverse datasets (CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet). Our training framework also displays powerful few-shot generalization ability, i.e., generating high-fidelity images by training from scratch with just 100 real images, without any pre-training. Codes are available at: https://github.com/VITA-Group/Ultra-Data-Efficient-GAN-Training.
翻译:为了克服这一挑战,我们受到最新观察的启发,我们可以从GANs中发现独立可培训和高度稀少的子网络(a.k.a.a.,彩票),从GANs中发现可独立培训和高度分散的子网络(a.k.a.a.broach tickets)。我们将此视为一个诱导性的先导,我们建议从全新的角度看数据效率高的GAN培训:首先利用小型真实图像组的培训,从原始GAN中确定彩票;然后侧重于通过重新使用同一数据集,对稀少的子网络进行培训。这两个步骤都比较复杂,而且数据效率更高。我们找到我们的协调框架,为现有的真实图像增强方法提供或高层次的收益,我们还提供了可以与它们一起应用的新的地级增强。全面实验认可了我们提出的框架的有效性,包括各种GAN结构(SNGAN、BigGAN和StyGAN-V2)和多种数据集(CIFAR-10、CIFAR-100、Tin-ImageNet和图像网络),我们的培训框架还展示了从GRAD-stal-taimfal-tailalimal-tailalimalimalimalimalimalimalims.