In many real-world applications of machine learning, data are distributed across many clients and cannot leave the devices they are stored on. Furthermore, each client's data, computational resources and communication constraints may be very different. This setting is known as federated learning, in which privacy is a key concern. Differential privacy is commonly used to provide mathematical privacy guarantees. This work, to the best of our knowledge, is the first to consider federated, differentially private, Bayesian learning. We build on Partitioned Variational Inference (PVI) which was recently developed to support approximate Bayesian inference in the federated setting. We modify the client-side optimisation of PVI to provide an (${\epsilon}$, ${\delta}$)-DP guarantee. We show that it is possible to learn moderately private logistic regression models in the federated setting that achieve similar performance to models trained non-privately on centralised data.
翻译:在机器学习的许多现实应用中,数据分布于许多客户,不能留下存储的装置。此外,每个客户的数据、计算资源和通信限制可能大不相同。这种环境被称为联合学习,隐私是其中的一个关键问题。不同的隐私通常用于提供数学隐私权保障。根据我们所知,这项工作是第一个考虑联邦化的、差别化的私人巴耶斯人学习的。我们利用了最近开发的分割变异推断(PVI)来支持联邦环境中近似贝叶斯人的推断。我们修改了PVI客户端的优化,以提供$(hepslon)$,$(delta)$($)-DP保证。我们表明,有可能在联邦化环境中学习中度私人后勤回归模型,这些模型在集中数据方面获得类似训练的模型。