Extrapolating Coverage Rate in Greybox Fuzzing
A fuzzer can literally run forever. However, as more resources are spent, the coverage rate continuously drops, and the utility of the fuzzer declines. To tackle this coverage-resource tradeoff, we could introduce a policy to stop a campaign whenever the coverage rate drops below a certain threshold...
Uloženo v:
| Vydáno v: | Proceedings / International Conference on Software Engineering s. 1623 - 1634 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
ACM
14.04.2024
|
| Témata: | |
| ISSN: | 1558-1225 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | A fuzzer can literally run forever. However, as more resources are spent, the coverage rate continuously drops, and the utility of the fuzzer declines. To tackle this coverage-resource tradeoff, we could introduce a policy to stop a campaign whenever the coverage rate drops below a certain threshold value, say 10 new branches covered per 15 minutes. During the campaign, can we predict the coverage rate at some point in the future? If so, how well can we predict the future coverage rate as the prediction horizon or the current campaign length increases? How can we tackle the statistical challenge of adaptive bias, which is inherent in greybox fuzzing (i.e., samples are not independent and identically distributed)? In this paper, we i) evaluate existing statistical techniques to predict the coverage rate U(t_{0}+k) at any time t_{0} in the campaign after a period of k units of time in the future and ii) develop a new extrapolation methodology that tackles the adaptive bias. We propose to efficiently simulate a large number of blackbox campaigns from the collected coverage data, estimate the coverage rate for each of these blackbox campaigns and conduct a simple regression to extrapolate the coverage rate for the greybox campaign. Our empirical evaluation using the Fuzztastic fuzzer benchmark demonstrates that our extrapolation methodology exhibits at least one order of magnitude lower error compared to the existing benchmark for 4 out of 5 experimental subjects we investigated. Notably, compared to the existing extrapolation methodology, our extrapola-tor excels in making long-term predictions, such as those extending up to three times the length of the current campaign. |
|---|---|
| AbstractList | A fuzzer can literally run forever. However, as more resources are spent, the coverage rate continuously drops, and the utility of the fuzzer declines. To tackle this coverage-resource tradeoff, we could introduce a policy to stop a campaign whenever the coverage rate drops below a certain threshold value, say 10 new branches covered per 15 minutes. During the campaign, can we predict the coverage rate at some point in the future? If so, how well can we predict the future coverage rate as the prediction horizon or the current campaign length increases? How can we tackle the statistical challenge of adaptive bias, which is inherent in greybox fuzzing (i.e., samples are not independent and identically distributed)? In this paper, we i) evaluate existing statistical techniques to predict the coverage rate U(t_{0}+k) at any time t_{0} in the campaign after a period of k units of time in the future and ii) develop a new extrapolation methodology that tackles the adaptive bias. We propose to efficiently simulate a large number of blackbox campaigns from the collected coverage data, estimate the coverage rate for each of these blackbox campaigns and conduct a simple regression to extrapolate the coverage rate for the greybox campaign. Our empirical evaluation using the Fuzztastic fuzzer benchmark demonstrates that our extrapolation methodology exhibits at least one order of magnitude lower error compared to the existing benchmark for 4 out of 5 experimental subjects we investigated. Notably, compared to the existing extrapolation methodology, our extrapola-tor excels in making long-term predictions, such as those extending up to three times the length of the current campaign. |
| Author | Liyanage, Danushka Bohme, Marcel Tantithamthavorn, Chakkrit Lee, Seongmin |
| Author_xml | – sequence: 1 givenname: Danushka surname: Liyanage fullname: Liyanage, Danushka organization: Monash University,Australia – sequence: 2 givenname: Seongmin surname: Lee fullname: Lee, Seongmin organization: MPI-SP,Germany – sequence: 3 givenname: Chakkrit surname: Tantithamthavorn fullname: Tantithamthavorn, Chakkrit organization: Monash University,Australia – sequence: 4 givenname: Marcel surname: Bohme fullname: Bohme, Marcel organization: MPI-SP,Germany |
| BookMark | eNotjsFKw0AQQFdRsNacvXjIDyTuZHZ2do8S2ioUBNFz2U0nJVCTkkRp-_UG7OldHo93r27arhWlHkHnAIaekTyTxhwtevDuSiWevTNasy6AzbWaAZHLoCjoTiXD0ERNBomtwZnKF8exD4duH8am3aVl9yt92En6EUZJmzZd9XKK3TFd_pzPk_CgbuuwHyS5cK6-lovP8jVbv6_eypd1FgosXOajCDMFsMHXXPlQb514sm7rmQGZDKGNFgFAoqsnKUIFWGkWcmIB5-rpv9uIyObQN9-hP21gGndMjH_9gESC |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK ESBDL RIE RIO |
| DOI | 10.1145/3597503.3639198 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Open Access Journals IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798400702174 |
| EISSN | 1558-1225 |
| EndPage | 1634 |
| ExternalDocumentID | 10548757 |
| Genre | orig-research |
| GroupedDBID | -~X .4S .DC 29O 5VS 6IE 6IF 6IH 6IK 6IL 6IM 6IN 8US AAJGR AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS ARCSS AVWKF BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO EDO ESBDL FEDTE I-F IEGSK IJVOP IPLJI M43 OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-a2328-9bee775a16a9f7c9afd8e9568d97713754536b63111eb8fa9fb1c13c07e58e613 |
| IEDL.DBID | RIE |
| IngestDate | Wed Aug 27 01:52:39 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a2328-9bee775a16a9f7c9afd8e9568d97713754536b63111eb8fa9fb1c13c07e58e613 |
| OpenAccessLink | https://ieeexplore.ieee.org/document/10548757 |
| PageCount | 12 |
| ParticipantIDs | ieee_primary_10548757 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-April-14 |
| PublicationDateYYYYMMDD | 2024-04-14 |
| PublicationDate_xml | – month: 04 year: 2024 text: 2024-April-14 day: 14 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings / International Conference on Software Engineering |
| PublicationTitleAbbrev | ICSE |
| PublicationYear | 2024 |
| Publisher | ACM |
| Publisher_xml | – name: ACM |
| SSID | ssib054357643 ssib055306466 ssj0006499 |
| Score | 2.277363 |
| Snippet | A fuzzer can literally run forever. However, as more resources are spent, the coverage rate continuously drops, and the utility of the fuzzer declines. To... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1623 |
| SubjectTerms | adaptive bias Benchmark testing Closed box coverage rate Extrapolation Fuzzing greybox fuzzing Software engineering statistical method |
| Title | Extrapolating Coverage Rate in Greybox Fuzzing |
| URI | https://ieeexplore.ieee.org/document/10548757 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED5RxMBUHkW85YE1Ja6fmasWBlRVCKRulZ1cUJekKi2q-us5uymPgYHNijzEsn3fZ_u--wDuCLVplZSYGKFsIntcJtaHBBuPmc6cERZjEdcnMxrZySQbN2L1qIVBxJh8ht3QjG_5RZ2vwlUZ7fDIr00LWsborVhrt3gU4b75UVsq2OFoGbhKE5Y1cfumtg-X6l4Qk1ap6AqCaDp4_zJXidgybP_zr46g863SY-Mv_DmGPaxOoL2zaWDNrj2F7mC9XLh5HbLeqjfWD0mbFEXYM9FMNqvYA82mr9dsuNpsqEMHXoeDl_5j0tgkJI7oEIUrj2iMcly7rDR55srCYlABFsTteLC4VUJ7LSiqobcldfI85yJPDSqLBOdnsF_VFZ4Dc8bQ-czJnkSUWqBXRapcGiwmC2ssXkAnjH8631bCmO6GfvnH9ys47BEJCK8vXF7D_nKxwhs4yD-Ws_fFbZy_T_fNlsc |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED5BQYKpPIp444E1JYnt2JmrliJKVaEidavs5IK6JFUfqOqv55ymPAYGNivyEMv2fZ_t--4DuCfUplWSoae41J4IA-Fp6xJsLMZRbBTXWBZx7al-X49G8aASq5daGEQsk8-w6ZrlW35aJEt3VUY7vOTXahf2pBChv5FrbZePJORXP6pLOUOcSDi2UgXmiNh9Vd0nEPKBE5eWPm9yAmk6ev-yVynRpVP_538dQeNbp8cGXwh0DDuYn0B9a9TAqn17Cs32ajEz08LlveXvrOXSNimOsFcimmySs0eaT1usWGe5XlOHBrx12sNW16uMEjxDhIgClkVUSpogMnGmkthkqUanA0yJ3QXO5FbyyEac4hpanVEnGyQBT3yFUiMB-hnU8iLHc2BGKTqhGREKRBFxtDL1pfGdyWSqlcYLaLjxj6ebWhjj7dAv__h-Bwfd4Utv3HvqP1_BYUiUwL3FBOIaaovZEm9gP_lYTOaz23IuPwEFJJoO |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%2F+International+Conference+on+Software+Engineering&rft.atitle=Extrapolating+Coverage+Rate+in+Greybox+Fuzzing&rft.au=Liyanage%2C+Danushka&rft.au=Lee%2C+Seongmin&rft.au=Tantithamthavorn%2C+Chakkrit&rft.au=Bohme%2C+Marcel&rft.date=2024-04-14&rft.pub=ACM&rft.eissn=1558-1225&rft.spage=1623&rft.epage=1634&rft_id=info:doi/10.1145%2F3597503.3639198&rft.externalDocID=10548757 |