MAE-CoReNet: Masking Autoencoder-Convolutional Reformer Net

Transformers are widely used in computer vision for feature extraction, object detection, and image classification. Many methods boost performance by adding convolutional or attention layers, yet large models cause high training costs. This manuscript suggests a Masking Autoencoder-Convolutional Ref...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:International journal of information system modeling and design Jg. 16; H. 1; S. 1 - 23
Hauptverfasser: Wang, Di, Wang, Li, Zhou, Yuyang
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Hershey IGI Global 01.01.2025
Schlagworte:
Accuracy
Artificial intelligence
Classification
Computer vision
Convergence
Datasets
Digital twins
Efficiency
Image classification
Information systems
Machine learning
Masking
Medical research
Object recognition
ISSN:1947-8186, 1947-8194
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract Transformers are widely used in computer vision for feature extraction, object detection, and image classification. Many methods boost performance by adding convolutional or attention layers, yet large models cause high training costs. This manuscript suggests a Masking Autoencoder-Convolutional Reformer Net (MAE-CoReNet) to enhance accuracy with less training time. It employs an attention mechanism with Locality-sensitive Hashing (LSH) to cut training time and increase classification accuracy. Also, it uses the masking technique for module pre-training to improve results. The experimental results show that the model in this manuscript performs well on the CIFAR-100 dataset. Compared to the Convolutional Attention Transformer Network (CoAtNet), MAE-CoReNet's convergence speed decreases from 65 to 25 epochs, and its accuracy increases from 53.1% to 90.2%. Additionally, when compared with other models on the ImageNet22k dataset, this model achieves the highest accuracy and fastest convergence speed. Its convergence speed is 55 epochs and the accuracy is 89.5%.
AbstractList Transformers are widely used in computer vision for feature extraction, object detection, and image classification. Many methods boost performance by adding convolutional or attention layers, yet large models cause high training costs. This manuscript suggests a Masking Autoencoder-Convolutional Reformer Net (MAE-CoReNet) to enhance accuracy with less training time. It employs an attention mechanism with Locality-sensitive Hashing (LSH) to cut training time and increase classification accuracy. Also, it uses the masking technique for module pre-training to improve results. The experimental results show that the model in this manuscript performs well on the CIFAR-100 dataset. Compared to the Convolutional Attention Transformer Network (CoAtNet), MAE-CoReNet's convergence speed decreases from 65 to 25 epochs, and its accuracy increases from 53.1% to 90.2%. Additionally, when compared with other models on the ImageNet22k dataset, this model achieves the highest accuracy and fastest convergence speed. Its convergence speed is 55 epochs and the accuracy is 89.5%.
Author Zhou, Yuyang
Wang, Li
Wang, Di
AuthorAffiliation China Academy of Space Technology, China
Xi'an Jiaotong University, China
AuthorAffiliation_xml – name: China Academy of Space Technology, China
– name: Xi'an Jiaotong University, China
Author_xml – sequence: 1
  givenname: Di
  surname: Wang
  fullname: Wang, Di
  organization: Xi'an Jiaotong University, China
– sequence: 2
  givenname: Li
  surname: Wang
  fullname: Wang, Li
  organization: Xi'an Jiaotong University, China
– sequence: 3
  givenname: Yuyang
  surname: Zhou
  fullname: Zhou, Yuyang
  organization: China Academy of Space Technology, China
BookMark eNptkE1Lw0AQhhepYK09eg94Tt3vDz2VWLXSKlQ9L8lmU1LbbN1NBP-9K6l6cQ4zA_PwMjynYNC4xgJwjuCEQiQv5w_z5-XNhAhElDoCQ6SoSGXsg99d8hMwDmEDYzEqBONDcL2cztLMreyjba-SZR7e6madTLvW2ca40vp4bD7ctmtr1-TbZGUr53fWJ5E_A8dVvg12fJgj8Ho7e8nu08XT3TybLlKDJWnTAjMuraoI4UowhaWiyCqKeYm4NAUyjOG8NBhxLgqLaYkkpALDImJIGkVG4KLP3Xv33tnQ6o3rfPwmaIKEoFQxCSOV9pTxLgRvK7339S73nxpB_a1I94p0ryjyWc_X6_ov8KBCH0zoHxP_ZiBOvgBS1m60
Cites_doi 10.1007/978-3-030-58452-8_13
10.1109/5.726791
10.1109/CVPR.2009.5206848
10.1109/WACV48630.2021.00374
10.1007/978-3-031-19836-6_20
10.3390/machines13010036
10.18653/v1/D18-1049
10.48550/arXiv.2010.11929
10.1109/CVPR52688.2022.01553
10.48550/arXiv.2009.14794
10.1007/s11263-015-0816-y
10.1109/CVPR.2018.00474
10.1007/978-3-031-73016-0_19
ContentType Journal Article
Copyright 2025. This work is published under https://creativecommons.org/licenses/by/4.0/ (the "License"). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2025. This work is published under https://creativecommons.org/licenses/by/4.0/ (the "License"). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID AAYXX
CITATION
7SC
8FD
8FE
8FG
ABJCF
AFKRA
ARAPS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
GNUQQ
HCIFZ
JQ2
K7-
L6V
L7M
L~C
L~D
M7S
P62
PHGZM
PHGZT
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
DOI 10.4018/IJISMD.371399
DatabaseName CrossRef
Computer and Information Systems Abstracts
Technology Research Database
ProQuest SciTech Collection
ProQuest Technology Collection
Materials Science & Engineering Collection
ProQuest Central UK/Ireland
Advanced Technologies & Computer Science Collection
ProQuest Central Essentials
ProQuest Central
Technology Collection
ProQuest One
ProQuest Central Korea
ProQuest Central Student
SciTech Premium Collection
ProQuest Computer Science Collection
Computer Science Database
ProQuest Engineering Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
Engineering Database
ProQuest Advanced Technologies & Aerospace Collection
ProQuest Central Premium
ProQuest One Academic (New)
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering Collection
DatabaseTitle CrossRef
Computer Science Database
ProQuest Central Student
Technology Collection
Technology Research Database
Computer and Information Systems Abstracts – Academic
ProQuest One Academic Middle East (New)
ProQuest Advanced Technologies & Aerospace Collection
ProQuest Central Essentials
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
SciTech Premium Collection
ProQuest One Community College
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
ProQuest Central Korea
ProQuest Central (New)
Advanced Technologies Database with Aerospace
Engineering Collection
Advanced Technologies & Aerospace Collection
Engineering Database
ProQuest One Academic Eastern Edition
ProQuest Technology Collection
ProQuest SciTech Collection
Computer and Information Systems Abstracts Professional
ProQuest One Academic UKI Edition
Materials Science & Engineering Collection
ProQuest One Academic
ProQuest One Academic (New)
DatabaseTitleList Computer Science Database

CrossRef
Database_xml – sequence: 1
  dbid: BENPR
  name: ProQuest Central
  url: https://www.proquest.com/central
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 1947-8194
EndPage 23
ExternalDocumentID 10_4018_IJISMD_371399
CoReNet_Masking_Autoenco10_4018_IJISMD_37139916
GroupedDBID 0R~
4.4
AAYVP
ABBKS
ABEPT
ABJCF
ABPHS
AFFHD
AFKRA
ALMA_UNASSIGNED_HOLDINGS
ARAPS
BAAKF
BENPR
BGLVJ
BYHXH
CBWLS
CCPQU
CDTDJ
CIGCI
CKMBR
CNQXE
COVLG
CTSEY
EBS
H13
HCIFZ
HZ~
IAO
ICD
ITC
IVC
K7-
M7S
MV1
N95
NEEBM
O9-
PHGZM
PHGZT
PQGLB
PTHSS
RIF
AAYXX
CITATION
7SC
8FD
8FE
8FG
AZQEC
DWQXO
GNUQQ
JQ2
L6V
L7M
L~C
L~D
P62
PKEHL
PQEST
PQQKQ
PQUKI
PRINS
ID FETCH-LOGICAL-c283t-b2568e9f336975928941e9426d168cb1c552adc21667be24d1804720b41e18c93
IEDL.DBID M7S
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001483055700004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1947-8186
IngestDate Mon Nov 17 12:10:48 EST 2025
Thu Nov 20 00:44:56 EST 2025
Tue Nov 18 04:10:29 EST 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 1
Language English
License http://creativecommons.org/licenses/by/3.0/deed.en_US
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c283t-b2568e9f336975928941e9426d168cb1c552adc21667be24d1804720b41e18c93
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0009-0004-4407-7242
0009-0009-3685-0625
0000-0002-7470-0187
PQID 3177449580
PQPubID 2045837
PageCount 23
ParticipantIDs proquest_journals_3177449580
igi_journals_CoReNet_Masking_Autoenco10_4018_IJISMD_37139916
crossref_primary_10_4018_IJISMD_371399
PublicationCentury 2000
PublicationDate 2025-01-01
PublicationDateYYYYMMDD 2025-01-01
PublicationDate_xml – month: 01
  year: 2025
  text: 2025-01-01
  day: 01
PublicationDecade 2020
PublicationPlace Hershey
PublicationPlace_xml – name: Hershey
PublicationTitle International journal of information system modeling and design
PublicationYear 2025
Publisher IGI Global
Publisher_xml – name: IGI Global
References S.Wang (IJISMD.371399-19) 2020
IJISMD.371399-20
IJISMD.371399-1
IJISMD.371399-10
IJISMD.371399-2
IJISMD.371399-12
IJISMD.371399-13
A.Vaswani (IJISMD.371399-18) 2017; 30
IJISMD.371399-14
IJISMD.371399-15
IJISMD.371399-7
IJISMD.371399-8
IJISMD.371399-3
IJISMD.371399-4
IJISMD.371399-5
IJISMD.371399-6
N.Kitaev (IJISMD.371399-9) 2020
IJISMD.371399-16
IJISMD.371399-17
A.Andoni (IJISMD.371399-0) 2015; Vol. 28
M. V.Koroteev (IJISMD.371399-11) 2021
References_xml – ident: IJISMD.371399-2
  doi: 10.1007/978-3-030-58452-8_13
– year: 2021
  ident: IJISMD.371399-11
  article-title: BERT: A review of applications in natural language processing and understanding.
– volume: Vol. 28
  year: 2015
  ident: IJISMD.371399-0
  publication-title: Practical and optimal LSH for angular distance
– ident: IJISMD.371399-14
  doi: 10.1109/5.726791
– ident: IJISMD.371399-6
  doi: 10.1109/CVPR.2009.5206848
– volume: 30
  start-page: 5998
  year: 2017
  ident: IJISMD.371399-18
  article-title: Attention is all you need.
  publication-title: Advances in Neural Information Processing Systems
– ident: IJISMD.371399-15
  doi: 10.1109/WACV48630.2021.00374
– ident: IJISMD.371399-3
– ident: IJISMD.371399-13
– ident: IJISMD.371399-1
  doi: 10.1007/978-3-031-19836-6_20
– ident: IJISMD.371399-12
  doi: 10.3390/machines13010036
– ident: IJISMD.371399-5
– year: 2020
  ident: IJISMD.371399-9
  article-title: Reformer: The efficient transformer.
– year: 2020
  ident: IJISMD.371399-19
  article-title: Linformer: Self-attention with linear complexity.
– ident: IJISMD.371399-20
  doi: 10.18653/v1/D18-1049
– ident: IJISMD.371399-7
  doi: 10.48550/arXiv.2010.11929
– ident: IJISMD.371399-8
  doi: 10.1109/CVPR52688.2022.01553
– ident: IJISMD.371399-4
  doi: 10.48550/arXiv.2009.14794
– ident: IJISMD.371399-16
  doi: 10.1007/s11263-015-0816-y
– ident: IJISMD.371399-17
  doi: 10.1109/CVPR.2018.00474
– ident: IJISMD.371399-10
  doi: 10.1007/978-3-031-73016-0_19
SSID ssj0000547756
Score 2.3039815
Snippet Transformers are widely used in computer vision for feature extraction, object detection, and image classification. Many methods boost performance by adding...
SourceID proquest
crossref
igi
SourceType Aggregation Database
Index Database
Publisher
StartPage 1
SubjectTerms Accuracy
Artificial intelligence
Classification
Computer vision
Convergence
Datasets
Digital twins
Efficiency
Image classification
Information systems
Machine learning
Masking
Medical research
Object recognition
Title MAE-CoReNet: Masking Autoencoder-Convolutional Reformer Net
URI http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/IJISMD.371399
https://www.proquest.com/docview/3177449580
Volume 16
WOSCitedRecordID wos001483055700004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVPQU
  databaseName: Computer Science Database
  customDbUrl:
  eissn: 1947-8194
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000547756
  issn: 1947-8186
  databaseCode: K7-
  dateStart: 20240101
  isFulltext: true
  titleUrlDefault: http://search.proquest.com/compscijour
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: Engineering Database
  customDbUrl:
  eissn: 1947-8194
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000547756
  issn: 1947-8186
  databaseCode: M7S
  dateStart: 20240101
  isFulltext: true
  titleUrlDefault: http://search.proquest.com
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: ProQuest Central
  customDbUrl:
  eissn: 1947-8194
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0000547756
  issn: 1947-8186
  databaseCode: BENPR
  dateStart: 20240101
  isFulltext: true
  titleUrlDefault: https://www.proquest.com/central
  providerName: ProQuest
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LT8JAEJ6IePDi24gP0oPxtsK223arJgYVIiqEgCbcmu4DwwUQir_f2XYrB40Xz7vddmfn9U033wCcS658PRKKKBoiQOEjRZJICRII4TEvQdicU-a_hN0uHw6jni24Ley1ysInZo5aTaWpkdcwzoUMs3lev519ENM1yvxdtS00SlA2LAk0u7o3-K6xYDoShlkDV3wnOmPKg5xmE0EFr7Wf2oPOw6WHMC1jfl2FpdL4ffzDN2cBp7X930_dgS2bajqNXDd2YU1P9mC7aOPgWKveh-tOo0nup33d1emV00kWpnruNJbp1JBcKj3HwcmnVVFcsK9Noosr4PwDeGs1X-8fie2pQCQmEikRmOJwHY08L4hCP0K4xaiOMEwrGnApqPR9N1HSpUEQCu0yRbnhk6wLnEa5jLxDWJ9MJ_oIHCW4wvNkCWJZFsmAGwN3KRtJhjiJ8QpcFCKNZzl1RoyQw8g-zmUf57KvwA0KPLbGs4jtjmO74bjY8K8P06ACp8UxrNZYncHx38MnsOmalr5ZVeUU1tP5Up_BhvxMx4t5Fcp3zW6vX4XSc0iqmXZ9Aahx0bw
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3JTsMwEB2xSXBhR5Q1B-BmwIkbOyxCFYsoXYRYpN5MvBT10kIbQPwU38g4Cz2AuHHgbMfS5I1n5tnJG4AtLUzZtpUhhnIkKKJtSBwZRUKlAhbESJszyfw6bzZFqxVdj8BH8S-M-6yyiIlpoDY97c7I9zDPcYbVvNg_eXomrmuUu10tWmhkblGz729I2QbH1TPEd9v3L87vTi9J3lWAaEylCVGY5IWN2kEQRrwcIeFg1EaYqAwNhVZUl8t-bLRPw5Ar6zNDhVNU3Fc4jQrtxJcw5I-zQHC3r2qcfJ3pYPnDedowFm3E4E9FmMl6IokRe9Wr6m3jbDdAWpgqzQ7T4GjnsfMtF6QJ7mLmv72aWZjOS2mvkvn-HIzY7jzMFG0qvDxqLcBho3JOTns3tmmTA68RD9ztgFd5SXpOxNPYPg52X_MtiAveWFfI4wo4fxHu_8SEJRjr9rp2GTyjhEF_ZTFydRbpULgA5lPW1gx5IBMl2CkglE-ZNIhESuWwlhnWMsO6BEcIsMyDw0DmFsvcYFkY_OPDNCzBWgH7cI0h5iu_D2_C5OVdoy7r1WZtFaZ81744PUFag7Gk_2LXYUK_Jp1BfyP1ZQ8e_tpDPgG_-Sf8
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=MAE-CoReNet%3A+Masking+Autoencoder-Convolutional+Reformer+Net&rft.jtitle=International+journal+of+information+system+modeling+and+design&rft.au=Wang%2C+Di&rft.au=Wang%2C+Li&rft.au=Zhou%2C+Yuyang&rft.date=2025-01-01&rft.pub=IGI+Global&rft.issn=1947-8186&rft.eissn=1947-8194&rft.volume=16&rft.issue=1&rft.spage=1&rft.epage=23&rft_id=info:doi/10.4018%2FIJISMD.371399
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1947-8186&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1947-8186&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1947-8186&client=summon