Imbalanced Data Clustering using Equilibrium K-Means

Traditional centroid-based clustering algorithms, such as hard K-means (HKM, or Lloyd's algorithm) and fuzzy K-means (FKM, or Bezdek's algorithm), display degraded performance when true underlying groups of data have varying sizes (i.e., imbalanced data). This paper introduces equilibrium K-means (EKM), a novel fuzzy clustering algorithm that has the robustness to imbalanced data by preventing centroids from crowding together in the center of large clusters. EKM is simple, alternating between two steps; fast, with the same time and space complexity as FKM; and scalable to large datasets. We evaluate the performance of EKM on two synthetic and ten real datasets, comparing it to other centroid-based algorithms, including HKM, FKM, maximum-entropy fuzzy clustering (MEFC), two FKM variations designed for imbalanced data, and the Gaussian mixture model. The results show that EKM performs competitively on balanced data and significantly outperforms other algorithms on imbalanced data. Deep clustering experiments on the MNIST dataset demonstrate the significance of making representation have an EKM-friendly structure when dealing with imbalanced data; In comparison to deep clustering with HKM, deep clustering with EKM obtains a more discriminative representation and a 35% improvement in clustering accuracy. Additionally, we reformulate HKM, FKM, MEFC, and EKM in a general form of gradient descent, where fuzziness is introduced differently and more simply than in Bezdek's work, and demonstrate how the general form facilitates a uniform study of KM algorithms.