Automatic Front-end Code Generation from image Via Multi-Head Attention

Code generation from Graphical User Interface (GUI) screenshots is a challenging task in machine learning. Existing methods (e.g., Pix2code) can handle simple datasets well but struggle with complex datasets requiring hundreds of code tokens. This paper proposes a novel method for generating front-e...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings (International Conference on Computer Engineering and Applications. Online) s. 869 - 872
Hlavní autoři: Zhang, Zhihang, Ding, Ye, Huang, Chenlin
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 07.04.2023
Témata:
Codes
Computational modeling
Computer architecture
Graphical user interfaces
Image recognition
Machine learning
Neural networks
Object recognition
Scene understanding
Task analysis
User interface programming
ISSN:2159-1288
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Code generation from Graphical User Interface (GUI) screenshots is a challenging task in machine learning. Existing methods (e.g., Pix2code) can handle simple datasets well but struggle with complex datasets requiring hundreds of code tokens. This paper proposes a novel method for generating front-end code based on multi-head attention. Our method uses a special technique called multi-head attention to analyze a GUI screenshot's feature vector, generate the code tokens, and link the analysis and generation processes. This architecture gives our method a significant advantage over similar models in terms of effectiveness. We conduct experiments on two types of datasets: Pix2code datasets and our own datasets. The experimental results demonstrate that our method achieves the best performance among existing methods.
ISSN:2159-1288
DOI:10.1109/ICCEA58433.2023.10135462