Semblance: A Rank-Based Kernel on Probability Spaces for Niche Detection

In data science, determining proximity between observations is critical to many downstream analyses such as clustering, classification and prediction. However, when the underlying probability distribution of the data is unclear, the function used to compute similarity between data points is often arbitrarily chosen. Here, we present a novel definition of proximity, Semblance, that uses the empirical distribution of a feature across all observations to inform the similarity between each pair. The advantage of Semblance lies in its distribution-free formulation and its ability to place greater emphasis on proximity between observation pairs that fall at the outskirts of the data distribution, as opposed to those that fall towards the center. We prove that Semblance is a valid Mercer kernel, thus allowing its principled use in kernel-based learning algorithms. Semblance can be applied to any data modality, and we demonstrate its consistently improved performance against conventional methods through simulations and three real case studies from diverse applications - cell-type classification in single-cell transcriptomics, image reconstruction, and financial forecasting.