A-SFS: Semi-supervised feature selection based on multi-task self-supervision

Feature selection is an important process in machine learning. It builds an interpretable and robust model by selecting the features that contribute the most to the prediction target. However, most mature feature selection algorithms, including supervised and semi-supervised, fail to fully exploit t...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Knowledge-based systems Jg. 252; S. 109449
Hauptverfasser: Qiu, Zhifeng, Zeng, Wanxin, Liao, Dahua, Gui, Ning
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier B.V 27.09.2022
Schlagworte:
Attention mechanism
Autoencoder
Feature selection
Self-supervised
Attention mechanism
Self-supervised
Feature selection
Autoencoder
ISSN:0950-7051, 1872-7409
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Feature selection is an important process in machine learning. It builds an interpretable and robust model by selecting the features that contribute the most to the prediction target. However, most mature feature selection algorithms, including supervised and semi-supervised, fail to fully exploit the complex potential structure between features. We believe that these structures are very important for the feature selection process, especially when labels are lacking and data is noisy. To this end, we innovatively introduces a deep learning-based self-supervised mechanism into feature selection problems, namely batch-Attention-based Self-supervision Feature Selection(A-SFS). Firstly, a multi-task self-supervised autoencoder is designed to uncover the hidden structural among features with the support of two pretext tasks. Guided by the integrated information from the multi-self-supervised learning model, a batch-attention mechanism is designed to generate feature weights according to batch-based feature selection patterns to alleviate the impacts introduced from a handful of noisy data. This method is compared to 14 major strong benchmarks, including LightGBM and XGBoost. Experimental results show that A-SFS achieves the highest accuracy in most datasets. Furthermore, this design significantly reduces the reliance on labels, with only 1/10 labeled data are needed to achieve the same performance as those state of art baselines. Results show that A-SFS is also most robust to the noisy and missing data. •A new feature selection method based on self-supervised pattern discovery.•A multi-task self-supervised model for latent structure discovery.•Batch-attention-based feature weight generation.
ISSN:0950-7051
1872-7409
DOI:10.1016/j.knosys.2022.109449