Critical Understanding of Security Vulnerability Detection Plugin Evaluation Reports

Integrated development environment (IDE) plugins aimed at detecting web application security vulnerabilities can help developers create secure applications in the first place. Most of such IDE plugins use static source code analysis approaches. Although several empirical studies evaluated the plugin...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings / Asia Pacific Software Engineering Conference S. 275 - 284
Hauptverfasser: Beba, Sindre, Karlsen, Magnus Melseth, Li, Jingyue, Zhang, Bing
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.12.2021
Schlagworte:
Application security
Codes
Detectors
Empirical study
Open source software
Root cause analysis
Security
Software security
Solids
Source code analysis
Vulnerability detection
ISSN:2640-0715
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Integrated development environment (IDE) plugins aimed at detecting web application security vulnerabilities can help developers create secure applications in the first place. Most of such IDE plugins use static source code analysis approaches. Although several empirical studies evaluated the plugins and compared their precision and recall of detecting web application security, few follow-up studies tried to understand the evaluation results. We analyzed more than 20,000 vulnerability reports based on 7,215 distinct test cases spanning 11 categories of web application vulnerabilities to understand the evaluation results of three open-source IDE plugins, namely, SpotBugs, FindSecBugs, and Early Security Vulnerability Detector (ESVD), which aimed at detecting security vulnerabilities of Java-based web applications. Our results identify many factors besides the source code analysis approach that can dramatically bias the detection performance. Based on our insights, we improved the studied plugins. In addition, our study raises the alarm that, without solid root cause analyses, the evaluation and comparisons of security vulnerability detection approaches and tools could be misleading. Thus, we proposed a guideline on reporting the evaluation results of the security vulnerability detection approaches.
ISSN:2640-0715
DOI:10.1109/APSEC53868.2021.00035