Federated learning (FL) enables collaborative model training while preserving each participant's privacy, which is particularly beneficial to the medical field. FedAvg is a standard algorithm that uses fixed weights, often originating from the dataset sizes at each client, to aggregate the distributed learned models on a server during the FL process. However, non-identical data distribution across clients, known as the non-i.i.d problem in FL, could make this assumption for setting fixed aggregation weights sub-optimal. In this work, we design a new data-driven approach, namely Auto-FedAvg, where aggregation weights are dynamically adjusted, depending on data distributions across data silos and the current training progress of the models. We disentangle the parameter set into two parts, local model parameters and global aggregation parameters, and update them iteratively with a communication-efficient algorithm. We first show the validity of our approach by outperforming state-of-the-art FL methods for image recognition on a heterogeneous data split of CIFAR-10. Furthermore, we demonstrate our algorithm's effectiveness on two multi-institutional medical image analysis tasks, i.e., COVID-19 lesion segmentation in chest CT and pancreas segmentation in abdominal CT.
翻译:FedAvg是一种标准算法,它使用固定的重量,通常来自每个客户的数据集大小,在FL过程中将一个服务器上传播的模型汇总起来。然而,客户之间非同质的数据分布,称为FL的非i.i.d问题,可以使这一假设成为设定固定总重亚最佳值的假设。在这项工作中,我们设计了一种新的数据驱动方法,即Auto-FedAvg, 集重根据数据库的数据分布和模型当前培训进度进行动态调整。我们将设定的参数分解成两个部分,即地方模型参数和全球汇总参数,并用通信效率算法反复更新。我们首先通过业绩优于最先进的FL方法来显示我们的方法的有效性,以图像识别CIFAR-10的混杂数据。此外,我们还展示了我们的算法在两个多机构医学段段分析中的有效性,即 COVI 。