LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning
The automation of code review activities, a long-standing pursuit in software engineering, has been primarily addressed by numerous domain-specific pre-trained models. Despite their success, these models frequently demand extensive resources for pre-training from scratch. In contrast, Large Language...
Saved in:
| Published in: | Proceedings - International Symposium on Software Reliability Engineering pp. 647 - 658 |
|---|---|
| Main Authors: | , , , , |
| Format: | Conference Proceeding |
| Language: | English |
| Published: |
IEEE
09.10.2023
|
| Subjects: | |
| ISSN: | 2332-6549 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | The automation of code review activities, a long-standing pursuit in software engineering, has been primarily addressed by numerous domain-specific pre-trained models. Despite their success, these models frequently demand extensive resources for pre-training from scratch. In contrast, Large Language Models (LLMs) provide an intriguing alternative, given their remarkable capabilities when supplemented with domain-specific knowledge. However, their potential for automating code review tasks remains largely unexplored.In response to this research gap, we present LLaMA-Reviewer, an innovative framework that leverages the capabilities of LLaMA, a popular LLM, in the realm of code review. Mindful of resource constraints, this framework employs parameter-efficient fine-tuning (PEFT) methods, delivering high performance while using less than 1% of trainable parameters.An extensive evaluation of LLaMA-Reviewer is conducted on two diverse, publicly available datasets. Notably, even with the smallest LLaMA base model consisting of 6.7B parameters and a limited number of tuning epochs, LLaMA-Reviewer equals the performance of existing code-review-focused models.The ablation experiments provide insights into the influence of various fine-tuning process components, including input representation, instruction tuning, and different PEFT methods. To foster continuous progress in this field, the code and all PEFT-weight plugins have been made open-source. |
|---|---|
| AbstractList | The automation of code review activities, a long-standing pursuit in software engineering, has been primarily addressed by numerous domain-specific pre-trained models. Despite their success, these models frequently demand extensive resources for pre-training from scratch. In contrast, Large Language Models (LLMs) provide an intriguing alternative, given their remarkable capabilities when supplemented with domain-specific knowledge. However, their potential for automating code review tasks remains largely unexplored.In response to this research gap, we present LLaMA-Reviewer, an innovative framework that leverages the capabilities of LLaMA, a popular LLM, in the realm of code review. Mindful of resource constraints, this framework employs parameter-efficient fine-tuning (PEFT) methods, delivering high performance while using less than 1% of trainable parameters.An extensive evaluation of LLaMA-Reviewer is conducted on two diverse, publicly available datasets. Notably, even with the smallest LLaMA base model consisting of 6.7B parameters and a limited number of tuning epochs, LLaMA-Reviewer equals the performance of existing code-review-focused models.The ablation experiments provide insights into the influence of various fine-tuning process components, including input representation, instruction tuning, and different PEFT methods. To foster continuous progress in this field, the code and all PEFT-weight plugins have been made open-source. |
| Author | Yang, Li Zuo, Chun Li, Xiaojia Yu, Lei Lu, Junyi |
| Author_xml | – sequence: 1 givenname: Junyi surname: Lu fullname: Lu, Junyi email: lujunyi21@mails.ucas.ac.cn organization: Institute of Software,Chinese Academy of Sciences,Beijing,China – sequence: 2 givenname: Lei surname: Yu fullname: Yu, Lei email: yulei21@mails.ucas.ac.cn organization: Institute of Software,Chinese Academy of Sciences,Beijing,China – sequence: 3 givenname: Xiaojia surname: Li fullname: Li, Xiaojia email: lixj21@mails.tsinghua.edu.cn organization: Tsinghua University,School of Software,Beijing,China – sequence: 4 givenname: Li surname: Yang fullname: Yang, Li email: yangli2017@iscas.ac.cn organization: Institute of Software,Chinese Academy of Sciences,Beijing,China – sequence: 5 givenname: Chun surname: Zuo fullname: Zuo, Chun email: zuochun@sinosoft.com.cn organization: Sinosoft Company Limited,Beijing,China |
| BookMark | eNotjttKAzEYhKMoWGvfQCEvkJrDnuLdUlotbFHa3pe_mz_bSJuVbLbFt3eh3swMzMcwj-TOtx4JeRF8KgTXr8vNZj1PdZEUU8mlmnLOZXZDJjrXhUq5EqlO1C0ZSaUky9JEP5DHrvseKJ4IOSJdVcGqZGs8O7xgeKOlOYOvnW_orDVIrwUt-9ieILrW04uLB1pBaHBQ3_QwhNWAHjsaD6HtmwP9ggAnjBjY3FpXO_SRLpxHtu39sPxE7i0cO5z8-5hsF_Pt7INVn-_LWVkxN5yLTGizVypTKGuZYZJnAmwO3IKQyqQq0xpB1JlNc4PG2LrmSVELC-k-2RtZqDF5vs46RNz9BHeC8LsTXGqtVaH-ANkLXu8 |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/ISSRE59848.2023.00026 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Xplore IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798350315943 |
| EISSN | 2332-6549 |
| EndPage | 658 |
| ExternalDocumentID | 10299938 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Science and Technology Service Network Plan funderid: 10.13039/501100013315 |
| GroupedDBID | 6IE 6IF 6IH 6IK 6IL 6IN AAJGR AAWTH ABLEC ACGFS ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IPLJI M43 OCL RIE RIL RNS |
| ID | FETCH-LOGICAL-i204t-19db3363e2c26e4761af7a0fa123d53699ea1c6f57deddfcc048c1fa5b4bd283 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 54 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001096886300057&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:30:37 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i204t-19db3363e2c26e4761af7a0fa123d53699ea1c6f57deddfcc048c1fa5b4bd283 |
| PageCount | 12 |
| ParticipantIDs | ieee_primary_10299938 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-Oct.-9 |
| PublicationDateYYYYMMDD | 2023-10-09 |
| PublicationDate_xml | – month: 10 year: 2023 text: 2023-Oct.-9 day: 09 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings - International Symposium on Software Reliability Engineering |
| PublicationTitleAbbrev | ISSRE |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0020412 |
| Score | 2.5702355 |
| Snippet | The automation of code review activities, a long-standing pursuit in software engineering, has been primarily addressed by numerous domain-specific pre-trained... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 647 |
| SubjectTerms | Automation Code Review Automation Codes Deep Learning Large Language Models (LLMs) LLaMA Parameter-Efficient Fine-Tuning (PEFT) Quality assurance Software engineering Software Quality Assurance Software reliability Task analysis Tuning |
| Title | LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning |
| URI | https://ieeexplore.ieee.org/document/10299938 |
| WOSCitedRecordID | wos001096886300057&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09a8MwEBVN6NAp_UjpNxq6qrViW9J1CyGhhTSEJkO2oE8IlLgkdn5_JdlJu3ToYoyNkbmzfPLze_cQemRC-iqsOElcLkhGAwmABrQJlOTaP1JKiWg2wScTsVjAtBGrRy2MtTaSz-xT2I3_8k2hqwCV-RnuX56QihZqcc5qsdbh6yo0jmokOjSB57fZ7GOYg4j8rV5oYxobKPyyUIkVZNT559inqPujxcPTQ5U5Q0d2fY46ezMG3MzNC7Qdj-V7n9RYv9284OiXrP01eFAYi-sTuF-VRa1WxAGCxePABPfbGrXEwRrtc4sb9x48lYG75cchw9hqwt8iHvl1KZlXAU_povloOB-8ksZRgax8fEpCwag0Zant6R6zGWdUOi4TJ339MnnKAKykmrmcG2uM09rPb02dzFWmjF-IXKL2uljbK4RzAyaIZiFJdWYFyMwpZ8BBpnzyOVyjbojh8qvumbHch-_mj-O36CSkKdLk4A61y01l79Gx3pWr7eYhZvob15qraw |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LSsNAFL1oFXRVHxXfzsLtaB6Tx7grpaXFtBSbRXdlnlCQprSJ3-_MJK1uXLgJISFMuDeTOzk55x6A5zhlpgrzBHs6SjHxLQnAt2gT5SwR5pHiPHVmE8lkks7ndNqI1Z0WRinlyGfqxe66f_myEJWFyswMNy9PGqaHcBQREni1XGv_fWVbRzUiHd-jr6PZ7KMf0dQxuALbyNS1UPhlouJqyKD9z9HPoPOjxkPTfZ05hwO1uoD2zo4BNbPzErZZxsZdXKP9avOGnGOyMNegXiEVqk-gblUWtV4RWRAWZZYLbrY1bomsOdrnFjX-PWjKLHvLjIP7rtmEuUU0MCtTnFcWUelAPujnvSFuPBXw0sSnxD6VPAzjUAUiiBVJYp_phHmamQomozCmVDFfxDpKpJJSC2FmuPA1izjh0ixFrqC1KlbqGlAkqbSyWeqFgqiUMqK5llRTwk36E3oDHRvDxbrumrHYhe_2j-NPcDLMx9kiG03e7-DUpsyR5ug9tMpNpR7gWHyVy-3m0WX9G-w3rrI |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+-+International+Symposium+on+Software+Reliability+Engineering&rft.atitle=LLaMA-Reviewer%3A+Advancing+Code+Review+Automation+with+Large+Language+Models+through+Parameter-Efficient+Fine-Tuning&rft.au=Lu%2C+Junyi&rft.au=Yu%2C+Lei&rft.au=Li%2C+Xiaojia&rft.au=Yang%2C+Li&rft.date=2023-10-09&rft.pub=IEEE&rft.eissn=2332-6549&rft.spage=647&rft.epage=658&rft_id=info:doi/10.1109%2FISSRE59848.2023.00026&rft.externalDocID=10299938 |