Natural Language is Not Enough: Benchmarking Multi-Modal Generative AI for Verilog Generation

Natural language interfaces have exhibited considerable potential in the automation of Verilog generation derived from high-level specifications through the utilization of large language models, garnering significant attention. Nevertheless, this paper elucidates that visual representations contribu...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Digest of technical papers - IEEE/ACM International Conference on Computer-Aided Design s. 1 - 9
Hlavní autori: Chang, Kaiyan, Chen, Zhirong, Zhou, Yunhao, Zhu, Wenlong, Wang, Kun, Xu, Haobo, Li, Cangyuan, Wang, Mengdi, Liang, Shengwen, Li, Huawei, Han, Yinhe, Wang, Ying
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: ACM 27.10.2024
Predmet:
Benchmark testing
Database languages
Design automation
Design methodology
Generative AI
Hardware
Hardware design languages
Large language models
Natural language processing
Visualization
ISSN:1558-2434
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Natural language interfaces have exhibited considerable potential in the automation of Verilog generation derived from high-level specifications through the utilization of large language models, garnering significant attention. Nevertheless, this paper elucidates that visual representations contribute essential contextual information critical to design intent for hardware architectures possessing spatial complexity, potentially surpassing the efficacy of natural-language-only inputs. Expanding upon this premise, our paper introduces an open-source benchmark1 for multi-modal generative models tailored for Verilog synthesis from visual-linguistic inputs, addressing both singu-lar and complex modules. Additionally, we introduce an open-source visual and natural language Verilog query language framework to facilitate efficient and user-friendly multi-modal queries. To evaluate the performance of the proposed multi-modal hardware generative AI in Verilog generation tasks, we compare it with a popular method that relies solely on natural language. Our results demonstrate a sig-nificant accuracy improvement in the multi-modal generated Verilog compared to queries based solely on natural language. We hope to reveal a new approach to hardware design in the large- hardware-design-model era, thereby fostering a more diversified and productive approach to hardware design.
ISSN:1558-2434
DOI:10.1145/3676536.3676679