Natural Language is Not Enough: Benchmarking Multi-Modal Generative AI for Verilog Generation

Natural language interfaces have exhibited considerable potential in the automation of Verilog generation derived from high-level specifications through the utilization of large language models, garnering significant attention. Nevertheless, this paper elucidates that visual representations contribu...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Digest of technical papers - IEEE/ACM International Conference on Computer-Aided Design s. 1 - 9
Hlavní autoři: Chang, Kaiyan, Chen, Zhirong, Zhou, Yunhao, Zhu, Wenlong, Wang, Kun, Xu, Haobo, Li, Cangyuan, Wang, Mengdi, Liang, Shengwen, Li, Huawei, Han, Yinhe, Wang, Ying
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: ACM 27.10.2024
Témata:
Benchmark testing
Database languages
Design automation
Design methodology
Generative AI
Hardware
Hardware design languages
Large language models
Natural language processing
Visualization
ISSN:1558-2434
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Natural language interfaces have exhibited considerable potential in the automation of Verilog generation derived from high-level specifications through the utilization of large language models, garnering significant attention. Nevertheless, this paper elucidates that visual representations contribute essential contextual information critical to design intent for hardware architectures possessing spatial complexity, potentially surpassing the efficacy of natural-language-only inputs. Expanding upon this premise, our paper introduces an open-source benchmark1 for multi-modal generative models tailored for Verilog synthesis from visual-linguistic inputs, addressing both singu-lar and complex modules. Additionally, we introduce an open-source visual and natural language Verilog query language framework to facilitate efficient and user-friendly multi-modal queries. To evaluate the performance of the proposed multi-modal hardware generative AI in Verilog generation tasks, we compare it with a popular method that relies solely on natural language. Our results demonstrate a sig-nificant accuracy improvement in the multi-modal generated Verilog compared to queries based solely on natural language. We hope to reveal a new approach to hardware design in the large- hardware-design-model era, thereby fostering a more diversified and productive approach to hardware design.
ISSN:1558-2434
DOI:10.1145/3676536.3676679