Information-Theoretic Unsupervised Feature Selection for High-Dimensional Spatial Data

Abstract

[Abstract]: High-dimensional unlabelled datasets present significant challenges for efficient analysis, storage and interpretation. Unsupervised feature selection offers a way to retain the most informative variables while discarding redundant or uninformative ones, enabling more scalable processing. We introduce a spatially aware, unsupervised method that uses information theoretic criteria to identify informative variables while limiting redundancy, producing compact and spatially dispersed subsets of features. Our approach avoids dependence on labelled data or modelspecific wrappers, making it suitable for large unstructured datasets. Experiments on MNIST and EMNIST datasets, including high-resolution upscaled versions, show that the selected features preserve both discriminative structure and reconstruction quality better than chosen supervised and unsupervised baselines, demonstrating the effectiveness of entropy and mutual information coupling in unlabelled high-dimensional settings.