Extrapolating Coverage Rate in Greybox Fuzzing

A fuzzer can literally run forever. However, as more resources are spent, the coverage rate continuously drops, and the utility of the fuzzer declines. To tackle this coverage-resource tradeoff, we could introduce a policy to stop a campaign whenever the coverage rate drops below a certain threshold...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings / International Conference on Software Engineering S. 1623 - 1634
Hauptverfasser: Liyanage, Danushka, Lee, Seongmin, Tantithamthavorn, Chakkrit, Bohme, Marcel
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: ACM 14.04.2024
Schlagworte:
adaptive bias
Benchmark testing
Closed box
coverage rate
Extrapolation
Fuzzing
greybox fuzzing
Software engineering
statistical method
ISSN:1558-1225
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:A fuzzer can literally run forever. However, as more resources are spent, the coverage rate continuously drops, and the utility of the fuzzer declines. To tackle this coverage-resource tradeoff, we could introduce a policy to stop a campaign whenever the coverage rate drops below a certain threshold value, say 10 new branches covered per 15 minutes. During the campaign, can we predict the coverage rate at some point in the future? If so, how well can we predict the future coverage rate as the prediction horizon or the current campaign length increases? How can we tackle the statistical challenge of adaptive bias, which is inherent in greybox fuzzing (i.e., samples are not independent and identically distributed)? In this paper, we i) evaluate existing statistical techniques to predict the coverage rate U(t_{0}+k) at any time t_{0} in the campaign after a period of k units of time in the future and ii) develop a new extrapolation methodology that tackles the adaptive bias. We propose to efficiently simulate a large number of blackbox campaigns from the collected coverage data, estimate the coverage rate for each of these blackbox campaigns and conduct a simple regression to extrapolate the coverage rate for the greybox campaign. Our empirical evaluation using the Fuzztastic fuzzer benchmark demonstrates that our extrapolation methodology exhibits at least one order of magnitude lower error compared to the existing benchmark for 4 out of 5 experimental subjects we investigated. Notably, compared to the existing extrapolation methodology, our extrapola-tor excels in making long-term predictions, such as those extending up to three times the length of the current campaign.
ISSN:1558-1225
DOI:10.1145/3597503.3639198