This work addresses a new problem of learning generative adversarial networks (GANs) from multiple data collections that are each i) owned separately and privately by different clients and ii) drawn from a non-identical distribution that comprises different classes. Given such multi-client and non-iid data as input, we aim to achieve a distribution involving all the classes input data can belong to, while keeping the data decentralized and private in each client storage. Our key contribution to this end is a new decentralized approach for learning GANs from non-iid data called Forgiver-First Update (F2U), which a) asks clients to train an individual discriminator with their own data and b) updates a generator to fool the most `forgiving' discriminators who deem generated samples as the most real. Our theoretical analysis proves that this updating strategy indeed allows the decentralized GAN to learn a generator's distribution with all the input classes as its global optimum based on f-divergence minimization. Moreover, we propose a relaxed version of F2U called Forgiver-First Aggregation (F2A), which adaptively aggregates the discriminators while emphasizing forgiving ones to perform well in practice. Our empirical evaluations with image generation tasks demonstrated the effectiveness of our approach over state-of-the-art decentralized learning methods.
翻译:这项工作解决了从由不同客户单独和私人拥有的多种数据收集中学习由不同客户单独和私人拥有的基因对抗网络(GANs)的新问题。鉴于这种多客户和非二人数据作为投入,我们的目标是实现包含所有类别输入数据的分布,同时保持每个客户储存中的分散和私有数据。我们为此做出的关键贡献是采用新的分散方法,从非二人数据中学习GANs,称为F2U(F2U),该方法a)要求客户用他们自己的数据来培训个人歧视者,b)更新一个生成器,以愚弄认为生成样本最真实的最“宽恕”歧视者。我们的理论分析证明,这一更新战略确实使分散的GAN能够学习发电机的分布,而所有输入课程都是基于F-调控最小化的全球最佳版本。此外,我们提议采用宽松的F2U(F2U)版本,称为“原谅-第一更新” (F2A),该版本要求客户用他们自己的数据来培训个人歧视者,b)更新一个生成者,以欺骗最真实的“宽恕”歧视者,而他们认为生成样本的人。我们的理论分析方法强调如何学习我们的分化方法。