Early Detection of Performance Regressions by Bridging Local Performance Data and Architectural Models
During software development, developers often make numerous modifications to the software to address existing issues or implement new features. However, certain changes may inadvertently have a detrimental impact on the overall system performance. To ensure that the performance of new software re-le...
Gespeichert in:
| Veröffentlicht in: | Proceedings / International Conference on Software Engineering S. 2841 - 2853 |
|---|---|
| Hauptverfasser: | , , , , , , |
| Format: | Tagungsbericht |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
26.04.2025
|
| Schlagworte: | |
| ISSN: | 1558-1225 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | During software development, developers often make numerous modifications to the software to address existing issues or implement new features. However, certain changes may inadvertently have a detrimental impact on the overall system performance. To ensure that the performance of new software re-leases does not degrade (i.e., absence of performance regressions), existing practices rely on system-level performance testing, such as load testing, or component-level performance testing, such as microbenchmarking, to detect performance regressions. However, performance testing for the entire system is often expensive and time-consuming, posing challenges to adapting to the rapid release cycles common in modern DevOps practices. In addition, system-level performance testing cannot be conducted until the system is fully built and deployed. On the other hand, component-level testing focuses on isolated components, neglecting overall system performance and the impact of system workloads. In this paper, we propose a novel approach to early detection of performance regressions by bridging the local performance data generated by component-level testing and the system-level architectural models. Our approach uses local performance data to identify deviations at the component level, and then propagate these deviations to the architectural model. We then use the architectural model to predict regressions in the performance of the overall system. In an evaluation of our approach on two representative open-source benchmark systems, we show that it can effectively detect end-to-end system performance regressions from local performance deviations with different intensities and under various system workloads. More importantly, our approach can detect regressions as early as in the development phase, in contrast to existing approaches that require the system to be fully built and deployed. Our approach is lightweight and can complement traditional system performance testing when testing resources are scarce. |
|---|---|
| AbstractList | During software development, developers often make numerous modifications to the software to address existing issues or implement new features. However, certain changes may inadvertently have a detrimental impact on the overall system performance. To ensure that the performance of new software re-leases does not degrade (i.e., absence of performance regressions), existing practices rely on system-level performance testing, such as load testing, or component-level performance testing, such as microbenchmarking, to detect performance regressions. However, performance testing for the entire system is often expensive and time-consuming, posing challenges to adapting to the rapid release cycles common in modern DevOps practices. In addition, system-level performance testing cannot be conducted until the system is fully built and deployed. On the other hand, component-level testing focuses on isolated components, neglecting overall system performance and the impact of system workloads. In this paper, we propose a novel approach to early detection of performance regressions by bridging the local performance data generated by component-level testing and the system-level architectural models. Our approach uses local performance data to identify deviations at the component level, and then propagate these deviations to the architectural model. We then use the architectural model to predict regressions in the performance of the overall system. In an evaluation of our approach on two representative open-source benchmark systems, we show that it can effectively detect end-to-end system performance regressions from local performance deviations with different intensities and under various system workloads. More importantly, our approach can detect regressions as early as in the development phase, in contrast to existing approaches that require the system to be fully built and deployed. Our approach is lightweight and can complement traditional system performance testing when testing resources are scarce. |
| Author | Eismann, Simon Bezemer, Cor- Paul Costa, Diego Elias Li, Heng van Hoorn, Andre Liao, Lizhi Shang, Weiyi |
| Author_xml | – sequence: 1 givenname: Lizhi surname: Liao fullname: Liao, Lizhi email: lizhi.liao@mun.ca organization: Memorial University of Newfoundland – sequence: 2 givenname: Simon surname: Eismann fullname: Eismann, Simon email: simon.eismann@uni-wuerzburg.de organization: University of Wurzburg – sequence: 3 givenname: Heng surname: Li fullname: Li, Heng email: heng.li@polymtl.ca organization: Polytechnique Montreal – sequence: 4 givenname: Cor- Paul surname: Bezemer fullname: Bezemer, Cor- Paul email: bezemer@ualberta.ca organization: University of Alberta – sequence: 5 givenname: Diego Elias surname: Costa fullname: Costa, Diego Elias email: diego.costa@concordia.ca organization: Concordia University – sequence: 6 givenname: Andre surname: van Hoorn fullname: van Hoorn, Andre email: andre.van.hoorn@uni-hamburg.de organization: University of Hamburg – sequence: 7 givenname: Weiyi surname: Shang fullname: Shang, Weiyi email: wshang@uwaterloo.ca organization: University of Waterloo |
| BookMark | eNpVkMtOwzAURA0Cibb0D7rwD6T4kevYy9KmUKkIxGNd2c51CEoTZIdF_54g2LAaaeboLGZKLrq-Q0IWnC05Z-Zmt34pAWReLAUTsGSMCXVG5qYwWkoODJTh52TCAXTGhYArMk3pY8RUbsyEhNLG9kQ3OKAfmr6jfaBPGEMfj7bzSJ-xjpjSuCTqTvQ2NlXddDXd9962_8iNHSy1XUVX0b83P7qvOCIPfYVtuiaXwbYJ5385I2_b8nV9n-0f73br1T6zQrEhU7kQgWmnmQftnTOofIEAlQantQnBBc9VjrwwY2WZhEJ6cMZ68NrJQs7I4tfbIOLhMzZHG0-H8Shh9PjSN94qWhQ |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/ICSE55347.2025.00026 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798331505691 |
| EISSN | 1558-1225 |
| EndPage | 2853 |
| ExternalDocumentID | 11029855 |
| Genre | orig-research |
| GroupedDBID | -~X .4S .DC 29O 5VS 6IE 6IF 6IH 6IK 6IL 6IM 6IN 8US AAJGR AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS ARCSS AVWKF BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO EDO FEDTE I-F IEGSK IJVOP IPLJI M43 OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-a260t-6422f08b80c58cbb9e6c7e55d85b889ffbfc164e179d85a03573c5b9ac5c8b373 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001538318100222&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 01:40:13 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a260t-6422f08b80c58cbb9e6c7e55d85b889ffbfc164e179d85a03573c5b9ac5c8b373 |
| PageCount | 13 |
| ParticipantIDs | ieee_primary_11029855 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-April-26 |
| PublicationDateYYYYMMDD | 2025-04-26 |
| PublicationDate_xml | – month: 04 year: 2025 text: 2025-April-26 day: 26 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings / International Conference on Software Engineering |
| PublicationTitleAbbrev | ICSE |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0006499 |
| Score | 2.2899268 |
| Snippet | During software development, developers often make numerous modifications to the software to address existing issues or implement new features. However,... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 2841 |
| SubjectTerms | Benchmark testing Data models DevOps Hands per-formance modeling performance engineering performance regression Predictive models regression testing Software engineering Software performance System performance Testing |
| Title | Early Detection of Performance Regressions by Bridging Local Performance Data and Architectural Models |
| URI | https://ieeexplore.ieee.org/document/11029855 |
| WOSCitedRecordID | wos001538318100222&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LawMhEJYm9NBT-kjpGw-9buOu66rHNg9aCCH0RW5B3bEUyiZkN4X--6rZJO2hh-JFRBRGdPxm5ptB6FppHzuYZBFYQbzpBiLN8yxSgklpYmLikEvvdchHIzGZyHFNVg9cGAAIwWdw47vBl5_PzNKbyjpOVSVSMNZADc75iqy1eXYz93evuXExkZ2H7lOfMZpyhwETbzcJ-RN-VFAJCmTQ-ufW-6i9peLh8UbJHKAdKA5Ra12LAddX8wjZkKkY96AKsVUFnlk83pIC8CO8rSJeixLrL3zneVpuQTz0uuzXzJ6qFFZFjm-3PgY3xRdN-yjb6GXQf-7eR3UNhUg5pFJFDl4klggtiGHCaC0hMxwYywXTQkhrtTUOMYG7l25IEco4NUxLZZgRmnJ6jJrFrIAThFMGqSLEOFClUurE4VqshMjAMippeoraXm7T-SpNxnQtsrM_xs_Rnj8a75pJsgvUrBZLuES75rN6LxdX4XC_Ad1ipcc |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5aBT3VR8W3OXhdm91sdpOj9kGLaylapbeSpBMRZCvtVvDfm6TbVg8eJJcQQgITksk3M98MQtdSudjBKAnAcOJMNxCodJwEkjMhdEh06HPpvWRpr8eHQ9EvyeqeCwMAPvgMblzX-_LHEz13prK6VVWR4Ixtoi0Wx1G4oGutHt7E_t5LdlxIRL3beGoxRuPUosDIWU58BoUfNVS8CmlX_7n5HqqtyXi4v1Iz-2gD8gNUXVZjwOXlPETG5yrGTSh8dFWOJwb317QA_Aivi5jXfIbVF75zTC27IM6cNvs1sykLiWU-xrdrL4Od4sqmvc9q6LndGjQ6QVlFIZAWqxSBBRiRIVxxohnXSglIdAqMjTlTnAtjlNEWM4G9mXZIEspSqpkSUjPNFU3pEarkkxyOEY4ZxJIQbWGVjKkVh22h5DwBw6ig8QmqObmNPhaJMkZLkZ3-MX6FdjqDh2yUdXv3Z2jXHZNz1ETJOaoU0zlcoG39WbzNppf-oL8BqhapDg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%2F+International+Conference+on+Software+Engineering&rft.atitle=Early+Detection+of+Performance+Regressions+by+Bridging+Local+Performance+Data+and+Architectural+Models&rft.au=Liao%2C+Lizhi&rft.au=Eismann%2C+Simon&rft.au=Li%2C+Heng&rft.au=Bezemer%2C+Cor-+Paul&rft.date=2025-04-26&rft.pub=IEEE&rft.eissn=1558-1225&rft.spage=2841&rft.epage=2853&rft_id=info:doi/10.1109%2FICSE55347.2025.00026&rft.externalDocID=11029855 |