Topological Stability: Guided Determination of the Nearest Neighbors in Non-Linear Dimensionality Reduction Techniques

In machine learning field, dimensionality reduction is one of the important tasks. It mitigates the undesired properties of high-dimensional spaces to facilitate classification, compression, and visualization of high-dimensional data. During the last decade, researchers proposed a large number of new (nonlinear) techniques for dimensionality reduction. Most of these techniques are based on the intuition that data lies on or near a complex low-dimensional manifold that is embedded in the high-dimensional space. New techniques for dimensionality reduction aim at identifying and extracting the manifold from the high-dimensional space. Isomap is one of widely-used low-dimensional embedding methods, where geodesic distances on a weighted graph are incorporated with the classical scaling (metric multidimensional scaling). Isomap chooses the nearest neighbors based on the distance only which causes bridges and topological instability. In this paper we pay our attention to topological stability that was not considered in Isomap.because at any point on the manifold , that point and its nearest neighbors forms a vector subspace and the orthogonal to that subspace is orthogonal to all vectors spans the vector subspace. Our approach uses the point itself and its two nearest neighbors to find the bases of the subspace and the orthogonal to that subspace which belongs to the orthogonal complementary subspace. Our approach then adds new points to the two nearest neighbors based on the distance and the angle between each new point and the orthogonal to the subspace. The superior performance of the new approach in choosing the nearest neighbors is confirmed through experimental work with several datasets.