Minimal algorithmic information loss methods for dimension reduction, feature selection and network sparsification

We present a novel, domain-agnostic, model-independent, unsupervised, and universally applicable Machine Learning approach for dimensionality reduction based on the principles of algorithmic complexity. Specifically, but without loss of generality, we focus on addressing the challenge of reducing ce...

Full description

Saved in:
Bibliographic Details
Published in:INFORMATION SCIENCES Vol. 720; p. 122520
Main Authors: Zenil, Hector, Kiani, Narsis A., Adams, Alyssa, Abrahão, Felipe S., Rueda-Toicen, Antonio, Zea, Allan A., Ozelim, Luan, Tegnér, Jesper
Format: Journal Article Publication
Language:English
Published: Elsevier Inc 01.12.2025
Subjects:
Algorithmic image segmentation
Data dimensionality reduction
Lossy algorithmic complexity
Machine learning
Network complexity
Recursive compression
Algorithmic image segmentation
Data dimensionality reduction
Lossy algorithmic complexity
Recursive compression
Machine learning
Network complexity
ISSN:0020-0255
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a novel, domain-agnostic, model-independent, unsupervised, and universally applicable Machine Learning approach for dimensionality reduction based on the principles of algorithmic complexity. Specifically, but without loss of generality, we focus on addressing the challenge of reducing certain dimensionality aspects, such as the number of edges in a network, while retaining essential features of interest. These features include preserving crucial network properties like degree distribution, clustering coefficient, edge betweenness, and degree and eigenvector centralities but can also go beyond edges to nodes and weights for network pruning and trimming. Our approach outperforms classical statistical Machine Learning techniques and state-of-the-art dimensionality reduction algorithms by preserving a greater number of data features that statistical algorithms would miss, particularly nonlinear patterns stemming from deterministic recursive processes that may look statistically random but are not. Moreover, previous approaches heavily rely on a priori feature selection, which requires constant supervision. Our findings demonstrate the effectiveness of the algorithms in overcoming some of these limitations while maintaining a time-efficient computational profile. Our approach not only matches, but also exceeds, the performance of established and state-of-the-art dimensionality reduction algorithms. We extend the applicability of our method to lossy compression tasks involving images and any multi-dimensional data. This highlights the versatility and broad utility of the approach in multiple domains. •Unsupervised, model-free dimensionality reduction using complexity theory.•Preserves key data properties better than traditional techniques.•Captures nonlinear patterns missed by statistical ML methods.•No a priori feature selection or supervision required.•Next generation cognitive neuro-symbolic ML for multi-modal data.
ISSN:0020-0255
DOI:10.1016/j.ins.2025.122520