Automatic Front-end Code Generation from image Via Multi-Head Attention
Code generation from Graphical User Interface (GUI) screenshots is a challenging task in machine learning. Existing methods (e.g., Pix2code) can handle simple datasets well but struggle with complex datasets requiring hundreds of code tokens. This paper proposes a novel method for generating front-e...
Uloženo v:
| Vydáno v: | Proceedings (International Conference on Computer Engineering and Applications. Online) s. 869 - 872 |
|---|---|
| Hlavní autoři: | , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
07.04.2023
|
| Témata: | |
| ISSN: | 2159-1288 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Code generation from Graphical User Interface (GUI) screenshots is a challenging task in machine learning. Existing methods (e.g., Pix2code) can handle simple datasets well but struggle with complex datasets requiring hundreds of code tokens. This paper proposes a novel method for generating front-end code based on multi-head attention. Our method uses a special technique called multi-head attention to analyze a GUI screenshot's feature vector, generate the code tokens, and link the analysis and generation processes. This architecture gives our method a significant advantage over similar models in terms of effectiveness. We conduct experiments on two types of datasets: Pix2code datasets and our own datasets. The experimental results demonstrate that our method achieves the best performance among existing methods. |
|---|---|
| AbstractList | Code generation from Graphical User Interface (GUI) screenshots is a challenging task in machine learning. Existing methods (e.g., Pix2code) can handle simple datasets well but struggle with complex datasets requiring hundreds of code tokens. This paper proposes a novel method for generating front-end code based on multi-head attention. Our method uses a special technique called multi-head attention to analyze a GUI screenshot's feature vector, generate the code tokens, and link the analysis and generation processes. This architecture gives our method a significant advantage over similar models in terms of effectiveness. We conduct experiments on two types of datasets: Pix2code datasets and our own datasets. The experimental results demonstrate that our method achieves the best performance among existing methods. |
| Author | Ding, Ye Huang, Chenlin Zhang, Zhihang |
| Author_xml | – sequence: 1 givenname: Zhihang surname: Zhang fullname: Zhang, Zhihang organization: School of Cyberspace Security, Dongguan University of Technology,Dongguan,China – sequence: 2 givenname: Ye surname: Ding fullname: Ding, Ye email: dingye@dgut.edu.cn organization: School of Cyberspace Security, Dongguan University of Technology,Dongguan,China – sequence: 3 givenname: Chenlin surname: Huang fullname: Huang, Chenlin organization: Academy of Computer Science, National University of Defense Technology,Changsha,China |
| BookMark | eNo1j8tOwzAURA0CiVL6Byz8Aym-ftzYyyjqSypiA2wrp7lGQY2NEnfB3xMEXc1iRkdz7tlNTJEY4yCWAMI97ep6VRmrlVpKIdUSBCijUV6xhSudVUYoXRptr9lMgnEFSGvv2GIcP4UQSgKgwxnbVOecep-7I18PKeaCYsvr1BLfUKRhKlLkYUg973r_Qfy98_z5fMpdsSXf8ipnir-bB3Yb_GmkxX_O2dt69Vpvi_3LZldX-6IDcLlAJ83RBIcYNHkpLVKJeGyDsQ2hbADRtzKgCkoEQB8mKQ2-KbUqQ6O1mrPHP25HRIevYXo1fB8u7uoHO7lPEg |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/ICCEA58433.2023.10135462 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798350347548 |
| EISSN | 2159-1288 |
| EndPage | 872 |
| ExternalDocumentID | 10135462 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK OCL RIE RIL |
| ID | FETCH-LOGICAL-i119t-6925c5f966f4ea2286e766cdf58be62b166ad2f63f30f16af01341ab7437fb443 |
| IEDL.DBID | RIE |
| IngestDate | Wed Aug 27 02:19:52 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i119t-6925c5f966f4ea2286e766cdf58be62b166ad2f63f30f16af01341ab7437fb443 |
| PageCount | 4 |
| ParticipantIDs | ieee_primary_10135462 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-April-7 |
| PublicationDateYYYYMMDD | 2023-04-07 |
| PublicationDate_xml | – month: 04 year: 2023 text: 2023-April-7 day: 07 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (International Conference on Computer Engineering and Applications. Online) |
| PublicationTitleAbbrev | ICCEA |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211696 |
| Score | 1.8279012 |
| Snippet | Code generation from Graphical User Interface (GUI) screenshots is a challenging task in machine learning. Existing methods (e.g., Pix2code) can handle simple... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 869 |
| SubjectTerms | Codes Computational modeling Computer architecture Graphical user interfaces Image recognition Machine learning Neural networks Object recognition Scene understanding Task analysis User interface programming |
| Title | Automatic Front-end Code Generation from image Via Multi-Head Attention |
| URI | https://ieeexplore.ieee.org/document/10135462 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwELWgYmAqH0V8ywNrSmzH52SsopayVB0Adauc-CxlIEEl5fdjO00RAwObZcmSdSff-ex77xHywFlhYvR_6sxLmEHBIm2UjASWvLQgEaUOYhNqsUhXq2y5A6sHLAwihuYzHPth-Ms3Tbn1T2XuhDMhEx9xD5WCDqy1f1ARrpSBDPpunTh7fM7z6cQlWCHGXiR83C__JaQS8shs-M8dnJDRDyKPLve55pQcYH1Ghr0kA92d0HPyNNm2TWBhpTNPTRBhbWjeGKQdv7R3A_WQElq9u0hC3ypNAwY3mjtn00nbdu2PI_I6m77k82inlRBVjGVtBBmXpbSueLEJas5TQAVQGivTAoEXDEAbbkFYEVsG2saeyU0X7gKhbJEk4oIM6qbGS0K18szFQkl38U7QFWSgM8GQQ2aNSFNzRUbeMOuPjg5j3dvk-o_5G3LszR_aXdQtGbSbLd6Ro_KrrT4398GJ3wPrmz8 |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELZQQYKpPIp444E1JX4mGauopRWl6lBQt8qJz1IGElRSfj-20xQxMLBZHizrTvb5zvd9H0IPlGQ6BPenTpyEmcxIoHQkAgY5zY0UAEJ5sYloNouXy2S-Bat7LAwA-OYz6Luh_8vXVb5xpTJ7wgkT3N24-4JzGjZwrV1JhdlkRiay7dcJk8dJmg4HNsQy1ncy4f12gV9SKj6SjLr_3MMx6v1g8vB8F21O0B6Up6jbijLg7Rk9Q0-DTV15HlY8cuQEAZQap5UG3DBMO0dgByrBxbu9S_BbobBH4QZj6248qOumAbKHXkfDRToOtmoJQUFIUgcyoSIXxqYvhoOiNJYQSZlrI-IMJM2IlEpTI5lhoSFSmdBxuanMPiEik3HOzlGnrEq4QFhFjruYRcI-vTnYlEyqhBGgMjGaxbG-RD1nmNVHQ4ixam1y9cf8PTocL16mq-lk9nyNjpwrfPNLdIM69XoDt-gg_6qLz_Wdd-g3Sn6ehg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28International+Conference+on+Computer+Engineering+and+Applications.+Online%29&rft.atitle=Automatic+Front-end+Code+Generation+from+image+Via+Multi-Head+Attention&rft.au=Zhang%2C+Zhihang&rft.au=Ding%2C+Ye&rft.au=Huang%2C+Chenlin&rft.date=2023-04-07&rft.pub=IEEE&rft.eissn=2159-1288&rft.spage=869&rft.epage=872&rft_id=info:doi/10.1109%2FICCEA58433.2023.10135462&rft.externalDocID=10135462 |