Automatic Front-end Code Generation from image Via Multi-Head Attention
Code generation from Graphical User Interface (GUI) screenshots is a challenging task in machine learning. Existing methods (e.g., Pix2code) can handle simple datasets well but struggle with complex datasets requiring hundreds of code tokens. This paper proposes a novel method for generating front-e...
Gespeichert in:
| Veröffentlicht in: | Proceedings (International Conference on Computer Engineering and Applications. Online) S. 869 - 872 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Tagungsbericht |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
07.04.2023
|
| Schlagworte: | |
| ISSN: | 2159-1288 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Code generation from Graphical User Interface (GUI) screenshots is a challenging task in machine learning. Existing methods (e.g., Pix2code) can handle simple datasets well but struggle with complex datasets requiring hundreds of code tokens. This paper proposes a novel method for generating front-end code based on multi-head attention. Our method uses a special technique called multi-head attention to analyze a GUI screenshot's feature vector, generate the code tokens, and link the analysis and generation processes. This architecture gives our method a significant advantage over similar models in terms of effectiveness. We conduct experiments on two types of datasets: Pix2code datasets and our own datasets. The experimental results demonstrate that our method achieves the best performance among existing methods. |
|---|---|
| AbstractList | Code generation from Graphical User Interface (GUI) screenshots is a challenging task in machine learning. Existing methods (e.g., Pix2code) can handle simple datasets well but struggle with complex datasets requiring hundreds of code tokens. This paper proposes a novel method for generating front-end code based on multi-head attention. Our method uses a special technique called multi-head attention to analyze a GUI screenshot's feature vector, generate the code tokens, and link the analysis and generation processes. This architecture gives our method a significant advantage over similar models in terms of effectiveness. We conduct experiments on two types of datasets: Pix2code datasets and our own datasets. The experimental results demonstrate that our method achieves the best performance among existing methods. |
| Author | Ding, Ye Huang, Chenlin Zhang, Zhihang |
| Author_xml | – sequence: 1 givenname: Zhihang surname: Zhang fullname: Zhang, Zhihang organization: School of Cyberspace Security, Dongguan University of Technology,Dongguan,China – sequence: 2 givenname: Ye surname: Ding fullname: Ding, Ye email: dingye@dgut.edu.cn organization: School of Cyberspace Security, Dongguan University of Technology,Dongguan,China – sequence: 3 givenname: Chenlin surname: Huang fullname: Huang, Chenlin organization: Academy of Computer Science, National University of Defense Technology,Changsha,China |
| BookMark | eNo1j8tOwzAURA0CiVL6Byz8Aym-ftzYyyjqSypiA2wrp7lGQY2NEnfB3xMEXc1iRkdz7tlNTJEY4yCWAMI97ep6VRmrlVpKIdUSBCijUV6xhSudVUYoXRptr9lMgnEFSGvv2GIcP4UQSgKgwxnbVOecep-7I18PKeaCYsvr1BLfUKRhKlLkYUg973r_Qfy98_z5fMpdsSXf8ipnir-bB3Yb_GmkxX_O2dt69Vpvi_3LZldX-6IDcLlAJ83RBIcYNHkpLVKJeGyDsQ2hbADRtzKgCkoEQB8mKQ2-KbUqQ6O1mrPHP25HRIevYXo1fB8u7uoHO7lPEg |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/ICCEA58433.2023.10135462 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798350347548 |
| EISSN | 2159-1288 |
| EndPage | 872 |
| ExternalDocumentID | 10135462 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK OCL RIE RIL |
| ID | FETCH-LOGICAL-i119t-6925c5f966f4ea2286e766cdf58be62b166ad2f63f30f16af01341ab7437fb443 |
| IEDL.DBID | RIE |
| IngestDate | Wed Aug 27 02:19:52 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i119t-6925c5f966f4ea2286e766cdf58be62b166ad2f63f30f16af01341ab7437fb443 |
| PageCount | 4 |
| ParticipantIDs | ieee_primary_10135462 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-April-7 |
| PublicationDateYYYYMMDD | 2023-04-07 |
| PublicationDate_xml | – month: 04 year: 2023 text: 2023-April-7 day: 07 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (International Conference on Computer Engineering and Applications. Online) |
| PublicationTitleAbbrev | ICCEA |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211696 |
| Score | 1.8277982 |
| Snippet | Code generation from Graphical User Interface (GUI) screenshots is a challenging task in machine learning. Existing methods (e.g., Pix2code) can handle simple... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 869 |
| SubjectTerms | Codes Computational modeling Computer architecture Graphical user interfaces Image recognition Machine learning Neural networks Object recognition Scene understanding Task analysis User interface programming |
| Title | Automatic Front-end Code Generation from image Via Multi-Head Attention |
| URI | https://ieeexplore.ieee.org/document/10135462 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELagYmDiVcRbHlhdEsc5x2MVtRQJVR0Adav8lDKQoJLy-7GdpoiBgc2KdJJ1J9_l7Pu-D6F7Dg6osYboInGEcZ0QoXwyFNTIXGpN00615JnP58VyKRZbsHrEwlhr4_CZHYVlfMs3jd6EqzJ_wr0lCxl3n3PowFq7C5XMtzIgoJ_WScTDU1lOxr7AZtkoiISPevNfQiqxjkyP_rmDYzT8QeThxa7WnKA9W5-io16SAW9P6Bl6HG_aJrKw4mmgJiC2NrhsjMUdv3QIAw6QEly9-0yC3yqJIwaXzHyw8bhtu_HHIXqdTl7KGdlqJZAqTUVLQNBc5843L45ZSWkBlgNo4_JCWaAqBZCGOshclrgUpEsCk5tU_geCO8VYdo4GdVPbC4SLwhtkPgcyK5j3gDJaOWEYgDKJ72gv0TA4ZvXR0WGsep9c_fH9Gh0G98dxF36DBu16Y2_Rgf5qq8_1XQziN5awnFU |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELZQQYKpPIp444HVJXGcSzxWUUsrStWhoG5V_JIykKCS8vuxnaaIgYEtimQrutPd-Zz7vg-hhwQMUKUVkWlgCEtkQLiwyZBTlce5lDRsVEumyWyWLpd8vgWreyyM1toPn-m-e_T_8lUlN-6qzEa4Xclcxt2PGaNBA9faXalEtpkBDu28TsAfJ1k2HNgSG0V9JxPebzf4JaXiK8mo-89vOEa9H0wenu-qzQna0-Up6raiDHgbo2foabCpK8_DikeOnIDoUuGsUho3DNPOEdiBSnDxbnMJfity7FG4ZGzdjQd13QxA9tDraLjIxmSrlkCKMOQ1AU5jGRvbvhimc0pT0AmAVCZOhQYqQoBcUQORiQITQm4Cx-WWC3uESIxgLDpHnbIq9QXCaWoXRDYLMs2ZtYBQUhiuGIBQge1pL1HPGWb10RBirFqbXP3x_h4djhcv09V0Mnu-RkfOFX74JblBnXq90bfoQH7Vxef6zjv0G-pnn5w |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28International+Conference+on+Computer+Engineering+and+Applications.+Online%29&rft.atitle=Automatic+Front-end+Code+Generation+from+image+Via+Multi-Head+Attention&rft.au=Zhang%2C+Zhihang&rft.au=Ding%2C+Ye&rft.au=Huang%2C+Chenlin&rft.date=2023-04-07&rft.pub=IEEE&rft.eissn=2159-1288&rft.spage=869&rft.epage=872&rft_id=info:doi/10.1109%2FICCEA58433.2023.10135462&rft.externalDocID=10135462 |