Self-Supervised Learning-Based Time Series Classification via Hierarchical Sparse Convolutional Masked-Autoencoder

In recent years, the use of time series analysis has become widespread, prompting researchers to explore methods to improve classification. Time series self-supervised learning has emerged as a significant area of study, aiming to uncover patterns in unlabeled data for richer information. Contrastiv...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE open journal of signal processing Ročník 5; s. 964 - 975
Hlavní autori: Yu, Ting, Xu, Kele, Wang, Xu, Ding, Bo, Feng, Dawei
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York IEEE 2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Classification
Computer architecture
Convolution
Convolutional neural networks
Decoders
Decoding
Effectiveness
Machine learning
Self-supervised learning
Task analysis
Time series
Time series analysis
Time series classification
time series pre-training
Transformers
ISSN:2644-1322, 2644-1322
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:In recent years, the use of time series analysis has become widespread, prompting researchers to explore methods to improve classification. Time series self-supervised learning has emerged as a significant area of study, aiming to uncover patterns in unlabeled data for richer information. Contrastive self-supervised learning, particularly, has gained attention for time series classification. However, it introduces inductive bias by generating positive and negative samples. Another approach involves Masked Autoencoders (MAE), which are effective for various data types. However, due to their reliance on the Transformer architecture, they demand significant computational resources during the pre-training phase. Recently, inspired by the remarkable advancements achieved by convolutional networks in the domain of time series forecasting, we aspire to employ convolutional networks utilizing a strategy of mask recovery for pre-training time series models. This study introduces a novel model termed Hierarchical Sparse Convolutional Masked-Autoencoder, "HSC-MAE", which seamlessly integrates convolutional operations with the MAE architecture to adeptly capture time series features across varying scales. Furthermore, the HSC-MAE model incorporates dedicated decoders that amalgamate global and local information, enhancing its capacity to comprehend intricate temporal patterns. To gauge the effectiveness of the proposed approach, an extensive array of experiments was conducted across nine distinct datasets. The experimental outcomes stand as a testament to the efficacy of HSC-MAE in effectively mitigating the aforementioned challenges.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2644-1322
2644-1322
DOI:10.1109/OJSP.2024.3435673