Zobrazit v EDS

Graph Masked Autoencoder Enhanced Predictor for Neural Architecture Search

Uloženo v:
Podrobná bibliografie
Název: Graph Masked Autoencoder Enhanced Predictor for Neural Architecture Search
Autoři: Kun Jing, Jungang Xu, Pengfei Li
Zdroj: Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence. :3114-3120
Informace o vydavateli: International Joint Conferences on Artificial Intelligence Organization, 2022.
Rok vydání: 2022
Témata: 0202 electrical engineering, electronic engineering, information engineering, 02 engineering and technology
Popis: Performance estimation of neural architecture is a crucial component of neural architecture search (NAS). Meanwhile, neural predictor is a current mainstream performance estimation method. However, it is a challenging task to train the predictor with few architecture evaluations for efficient NAS. In this paper, we propose a graph masked autoencoder (GMAE) enhanced predictor, which can reduce the dependence on supervision data by self-supervised pre-training with untrained architectures. We compare our GMAE-enhanced predictor with existing predictors in different search spaces, and experimental results show that our predictor has high query utilization. Moreover, GMAE-enhanced predictor with different search strategies can discover competitive architectures in different search spaces. Code and supplementary materials are available at https://github.com/kunjing96/GMAENAS.git.
Druh dokumentu: Article
DOI: 10.24963/ijcai.2022/432
Přístupové číslo: edsair.doi...........5cd3f94a20b667ca8fa31c13bf6c621c
Databáze: OpenAIRE
Popis
Abstrakt:Performance estimation of neural architecture is a crucial component of neural architecture search (NAS). Meanwhile, neural predictor is a current mainstream performance estimation method. However, it is a challenging task to train the predictor with few architecture evaluations for efficient NAS. In this paper, we propose a graph masked autoencoder (GMAE) enhanced predictor, which can reduce the dependence on supervision data by self-supervised pre-training with untrained architectures. We compare our GMAE-enhanced predictor with existing predictors in different search spaces, and experimental results show that our predictor has high query utilization. Moreover, GMAE-enhanced predictor with different search strategies can discover competitive architectures in different search spaces. Code and supplementary materials are available at https://github.com/kunjing96/GMAENAS.git.
DOI:10.24963/ijcai.2022/432