Self-Supervised Learning-Based Time Series Classification via Hierarchical Sparse Convolutional Masked-Autoencoder

In recent years, the use of time series analysis has become widespread, prompting researchers to explore methods to improve classification. Time series self-supervised learning has emerged as a significant area of study, aiming to uncover patterns in unlabeled data for richer information. Contrastiv...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE open journal of signal processing Jg. 5; S. 964 - 975
Hauptverfasser: Yu, Ting, Xu, Kele, Wang, Xu, Ding, Bo, Feng, Dawei
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York IEEE 2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Classification
Computer architecture
Convolution
Convolutional neural networks
Decoders
Decoding
Effectiveness
Machine learning
Self-supervised learning
Task analysis
Time series
Time series analysis
Time series classification
time series pre-training
Transformers
ISSN:2644-1322, 2644-1322
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In recent years, the use of time series analysis has become widespread, prompting researchers to explore methods to improve classification. Time series self-supervised learning has emerged as a significant area of study, aiming to uncover patterns in unlabeled data for richer information. Contrastive self-supervised learning, particularly, has gained attention for time series classification. However, it introduces inductive bias by generating positive and negative samples. Another approach involves Masked Autoencoders (MAE), which are effective for various data types. However, due to their reliance on the Transformer architecture, they demand significant computational resources during the pre-training phase. Recently, inspired by the remarkable advancements achieved by convolutional networks in the domain of time series forecasting, we aspire to employ convolutional networks utilizing a strategy of mask recovery for pre-training time series models. This study introduces a novel model termed Hierarchical Sparse Convolutional Masked-Autoencoder, "HSC-MAE", which seamlessly integrates convolutional operations with the MAE architecture to adeptly capture time series features across varying scales. Furthermore, the HSC-MAE model incorporates dedicated decoders that amalgamate global and local information, enhancing its capacity to comprehend intricate temporal patterns. To gauge the effectiveness of the proposed approach, an extensive array of experiments was conducted across nine distinct datasets. The experimental outcomes stand as a testament to the efficacy of HSC-MAE in effectively mitigating the aforementioned challenges.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2644-1322
2644-1322
DOI:10.1109/OJSP.2024.3435673