DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Models
The rise of Large Language Models (LLMs) has streamlined frontend interface creation through tools like Vercel's v0, yet surfaced challenges in design quality (e.g., accessibility, and usability). Current solutions, often limited by their focus, generalisability, or data dependency, fall short...
Uloženo v:
| Vydáno v: | Proceedings / International Conference on Software Engineering s. 2483 - 2494 |
|---|---|
| Hlavní autoři: | , , , , , , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
26.04.2025
|
| Témata: | |
| ISSN: | 1558-1225 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | The rise of Large Language Models (LLMs) has streamlined frontend interface creation through tools like Vercel's v0, yet surfaced challenges in design quality (e.g., accessibility, and usability). Current solutions, often limited by their focus, generalisability, or data dependency, fall short in addressing these complexities. Moreover, none of them examine the quality of LLM-generated UI design. In this work, we introduce DesignRepair, a novel dual-stream design guideline-aware system to examine and repair the UI design quality issues from both code aspect and rendered page aspect. We utilised the mature and popular Material Design as our knowledge base to guide this process. Specifically, we first constructed a comprehensive knowledge base encoding Google's Material Design principles into low-level component knowledge base and high-level system design knowledge base. After that, DesignRepair employs a LLM for the extraction of key components and utilizes the Playwright tool for precise page analysis, aligning these with the established knowledge bases. Finally, we integrate Retrieval-Augmented Generation with state-of-the-art LLMs like GPT-4 to holistically refine and repair frontend code through a strategic divide and conquer approach. Our extensive evaluations validated the efficacy and utility of our approach, demonstrating significant enhancements in adherence to design guidelines, accessibility, and user experience metrics. |
|---|---|
| AbstractList | The rise of Large Language Models (LLMs) has streamlined frontend interface creation through tools like Vercel's v0, yet surfaced challenges in design quality (e.g., accessibility, and usability). Current solutions, often limited by their focus, generalisability, or data dependency, fall short in addressing these complexities. Moreover, none of them examine the quality of LLM-generated UI design. In this work, we introduce DesignRepair, a novel dual-stream design guideline-aware system to examine and repair the UI design quality issues from both code aspect and rendered page aspect. We utilised the mature and popular Material Design as our knowledge base to guide this process. Specifically, we first constructed a comprehensive knowledge base encoding Google's Material Design principles into low-level component knowledge base and high-level system design knowledge base. After that, DesignRepair employs a LLM for the extraction of key components and utilizes the Playwright tool for precise page analysis, aligning these with the established knowledge bases. Finally, we integrate Retrieval-Augmented Generation with state-of-the-art LLMs like GPT-4 to holistically refine and repair frontend code through a strategic divide and conquer approach. Our extensive evaluations validated the efficacy and utility of our approach, demonstrating significant enhancements in adherence to design guidelines, accessibility, and user experience metrics. |
| Author | Chen, Jieshan Luo, Yuyu Mohammadi, Gelareh Lu, Qinghua Zhu, Liming Quigley, Aaron Luo, Tianqi Xing, Zhenchang Yuan, Mingyue |
| Author_xml | – sequence: 1 givenname: Mingyue surname: Yuan fullname: Yuan, Mingyue email: mingyue.yuan@unsw.edu.au organization: University of New South Wales,Australia – sequence: 2 givenname: Jieshan surname: Chen fullname: Chen, Jieshan email: jieshan.chen@data61.csiro.au organization: CSIRO's Data61,Australia – sequence: 3 givenname: Zhenchang surname: Xing fullname: Xing, Zhenchang email: zhenchang.xing@data61.csiro.au organization: CSIRO's Data61,Australia – sequence: 4 givenname: Aaron surname: Quigley fullname: Quigley, Aaron email: a.quigley@data61.csiro.au organization: University of New South Wales,Australia – sequence: 5 givenname: Yuyu surname: Luo fullname: Luo, Yuyu email: yuyuluo@hkust-gz.edu.cn organization: The Hong Kong University of Science and Technology,(Guangzhou),China – sequence: 6 givenname: Tianqi surname: Luo fullname: Luo, Tianqi email: tluo553@hkust-gz.edu.cn organization: The Hong Kong University of Science and Technology,(Guangzhou),China – sequence: 7 givenname: Gelareh surname: Mohammadi fullname: Mohammadi, Gelareh email: g.mohammadi@unsw.edu.au organization: University of New South Wales,Australia – sequence: 8 givenname: Qinghua surname: Lu fullname: Lu, Qinghua email: qinghua.lu@data61.csiro.au organization: University of New South Wales,Australia – sequence: 9 givenname: Liming surname: Zhu fullname: Zhu, Liming email: liming.zhu@data61.csiro.au organization: University of New South Wales,Australia |
| BookMark | eNotUMtOwzAQNAgk2tI_6ME_kLJ-bBxzq_qiUhAShROHykm2wah1KydRxd8TqVxmVvPYwwzZXTgFYmwiYCoE2KfNfLtEVNpMJUicAvTiDRtbYzOlBAKmVtyygUDMEiElPrBh0_wAQKqtHbCvBTW-Du90dj4-80XnDsm2jeSO_OrwdecrOvhAyeziIvFVPIWWQsWvHX7x7TfPXaypx1B3rj9eT32leWT3e3doaPzPI_a5Wn7MX5L8bb2Zz_LEyRTaREoq9qVOoXLCFCoDjakoVQqysBUZ0hZLp5FAWFMUfVjqMnOVwUobh6pUIza5_vVEtDtHf3Txd9evo0DKTP0BC1BVMQ |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/ICSE55347.2025.00109 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798331505691 |
| EISSN | 1558-1225 |
| EndPage | 2494 |
| ExternalDocumentID | 11030228 |
| Genre | orig-research |
| GroupedDBID | -~X .4S .DC 29O 5VS 6IE 6IF 6IH 6IK 6IL 6IM 6IN 8US AAJGR AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS ARCSS AVWKF BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO EDO FEDTE I-F IEGSK IJVOP IPLJI M43 OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-a260t-22ebfc460da17b3804561c3602b9de7e495ca45e0197bb2eb24c8ad75d47a53c3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001538318100194&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 01:40:27 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a260t-22ebfc460da17b3804561c3602b9de7e495ca45e0197bb2eb24c8ad75d47a53c3 |
| PageCount | 12 |
| ParticipantIDs | ieee_primary_11030228 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-April-26 |
| PublicationDateYYYYMMDD | 2025-04-26 |
| PublicationDate_xml | – month: 04 year: 2025 text: 2025-April-26 day: 26 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings / International Conference on Software Engineering |
| PublicationTitleAbbrev | ICSE |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0006499 |
| Score | 2.2896473 |
| Snippet | The rise of Large Language Models (LLMs) has streamlined frontend interface creation through tools like Vercel's v0, yet surfaced challenges in design quality... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 2483 |
| SubjectTerms | Codes Design Guideline Design methodology Frontend Code Repair Knowledge based systems Large language models Maintenance engineering Software development management Software engineering System analysis and design UI Design Usability User experience |
| Title | DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Models |
| URI | https://ieeexplore.ieee.org/document/11030228 |
| WOSCitedRecordID | wos001538318100194&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JS8NAGB1s8eCpLhV35uB1bDJLJvEmXVQopaBCwUOZ5QsUaippo3_fmUmsXjx4CSELgRky877lvYfQtRYyy6nX2M-pJlymhmgDlkgjKVVSCKWCiOtYTibpbJZNG7J64MIAQGg-gxt_Gmr5dmUqnyrrxd4Ti9K0hVpSypqstV12E4fdG25cHGW9x_7TUAjGpYsBqc-bhJ7DXw4qYQMZdf756X3U_aHi4el2kzlAO1Acos63FwNufs0j9DoIrRgOT6tFeYsHlVoSX3FWb7i-g-8rr2jlQCW5-1Ql4JGXLoDC4vod7DOyeOwbw92xTmJi75S2XHfRy2j43H8gjXECUS482RBKQeeGJ5FVsdQs9bAtNiyJqM4sSHBBkVFcgIN3Umv3MOUmVVYKy6USzLBj1C5WBZwgDIxxamlOgSoumc7SzLgoI2eg_FIQnaKuH6z5e62NMf8ep7M_rp-jPT8fvh5DkwvU3pQVXKJd87FZrMurMKNfRMGh0w |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JSwMxGA1aBT3VpeJuDl5jO1-SZsabdLHFWgpWKHgoSeYbKNSpdNG_bzIzrV48eBmGWRhImOR9y3uPkFsjVZSA19hPwDChQsuMxZgpqwC0klLrTMS1p_r9cDSKBgVZPePCIGLWfIZ3_jSr5cczu_KpsmrgPbEAwm2yI4WAIKdrbRbeukPvBTsuqEXVbuOlJSUXykWB4DMnWdfhLw-VbAtpl__58QNS-SHj0cFmmzkkW5gekfLajYEWP-cxeWtmzRgOUevJ_J42V3rKfM1Zv9P8Dn1ceU0rByvZw5eeI2178QJMY5q_Q31OlvZ8a7g75mlM6r3SposKeW23ho0OK6wTmHYBypIBoEmsqNdiHSjDQw_cAsvrNTBRjApdWGS1kOgAnjLGPQzChjpWMhZKS275CSmlsxRPCUXOBcSQAIIWipsojKyLMxKO2i8GtTNS8YM1_sjVMcbrcTr_4_oN2esMn3vjXrf_dEH2_dz46gzUL0lpOV_hFdm1n8vJYn6dze433umlGg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%2F+International+Conference+on+Software+Engineering&rft.atitle=DesignRepair%3A+Dual-Stream+Design+Guideline-Aware+Frontend+Repair+with+Large+Language+Models&rft.au=Yuan%2C+Mingyue&rft.au=Chen%2C+Jieshan&rft.au=Xing%2C+Zhenchang&rft.au=Quigley%2C+Aaron&rft.date=2025-04-26&rft.pub=IEEE&rft.eissn=1558-1225&rft.spage=2483&rft.epage=2494&rft_id=info:doi/10.1109%2FICSE55347.2025.00109&rft.externalDocID=11030228 |