Byzantine attacks hinder the deployment of federated learning algorithms. Although we know that the benign gradients and Byzantine attacked gradients are distributed differently, to detect the malicious gradients is challenging due to (1) the gradient is high-dimensional and each dimension has its unique distribution and (2) the benign gradients and the attacked gradients are always mixed (two-sample test methods cannot apply directly). To address the above, for the first time, we propose MANDERA which is theoretically guaranteed to efficiently detect all malicious gradients under Byzantine attacks with no prior knowledge or history about the number of attacked nodes. More specifically, we transfer the original updating gradient space into a ranking matrix. By such an operation, the scales of different dimensions of the gradients in the ranking space become identical. The high-dimensional benign gradients and the malicious gradients can be easily separated. The effectiveness of MANDERA is further confirmed by experimentation on four Byzantine attack implementations (Gaussian, Zero Gradient, Sign Flipping, Shifted Mean), comparing with state-of-the-art defenses. The experiments cover both IID and Non-IID datasets.
翻译:虽然我们知道良性梯度和被攻击的拜占庭梯度分布不同,但发现恶性梯度具有挑战性,因为(1) 梯度是高度,每个维度都有独特的分布,(2) 良性梯度和被攻击梯度总是混在一起(两样试验方法不能直接适用),为了解决上述问题,我们首次提议MANDERA在理论上保证在拜占庭攻击中有效探测到所有恶意梯度,事先对被攻击节点的数量没有了解或历史。更具体地说,我们将原先更新的梯度空间转移到一个排名矩阵。通过这种操作,排名空间不同梯度的大小将变得相同。高度良性梯度和被攻击梯度可以很容易地分开(两样试验方法不能直接适用),对拜占庭攻击实施过程的试验(Gaussian、Zero Gradient、Sign Flipping、变换版)进一步证实了MANDERA的有效性,与状态防御系统相比,实验覆盖IID和Non-D数据集。