BUGSJS: a benchmark and taxonomy of JavaScript bugs

Summary JavaScript is a popular programming language that is also error‐prone due to its asynchronous, dynamic, and loosely typed nature. In recent years, numerous techniques have been proposed for analyzing and testing JavaScript applications. However, our survey of the literature in this area reve...

Full description

Saved in:
Bibliographic Details
Published in:Software testing, verification & reliability Vol. 31; no. 4
Main Authors: Gyimesi, Péter, Vancsics, Béla, Stocco, Andrea, Mazinanian, Davood, Beszédes, Árpád, Ferenc, Rudolf, Mesbah, Ali
Format: Journal Article
Language:English
Published: Chichester Wiley Subscription Services, Inc 01.06.2021
Subjects:
Applications programs
benchmark
Benchmarks
bug database
bug taxonomy
BugsJS
Empirical analysis
Java
JavaScript
Reproducibility
Software reliability
Software testing
Taxonomy
ISSN:0960-0833, 1099-1689
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Summary JavaScript is a popular programming language that is also error‐prone due to its asynchronous, dynamic, and loosely typed nature. In recent years, numerous techniques have been proposed for analyzing and testing JavaScript applications. However, our survey of the literature in this area revealed that the proposed techniques are often evaluated on different datasets of programs and bugs. The lack of a commonly used benchmark limits the ability to perform fair and unbiased comparisons for assessing the efficacy of new techniques. To fill this gap, we propose BugsJS, a benchmark of 453 real, manually validated JavaScript bugs from 10 popular JavaScript server‐side programs, comprising 444k lines of code (LOC) in total. Each bug is accompanied by its bug report, the test cases that expose it, as well as the patch that fixes it. We extended BugsJS with a rich web interface for visualizing and dissecting the bugs' information, as well as a programmable API to access the faulty and fixed versions of the programs and to execute the corresponding test cases, which facilitates conducting highly reproducible empirical studies and comparisons of JavaScript analysis and testing tools. Moreover, following a rigorous procedure, we performed a classification of the bugs according to their nature. Our internal validation shows that our taxonomy is adequate for characterizing the bugs in BugsJS. We discuss several ways in which the resulting taxonomy and the benchmark can help direct researchers interested in automated testing of JavaScript applications. © 2021 The Authors. Software Testing, Verification & Reliability published by John Wiley & Sons, Ltd. In this study, we propose BugsJS, a benchmark of 453 real JavaScript bugs from 10 server‐side programs. We extended BugsJS with a rich web interface for visualizing and dissecting the bugs' information, as well as a programmable API to access the faulty source code and to execute the corresponding test cases, which facilitates conducting highly reproducible empirical studies and comparisons of JavaScript analysis and testing tools. Moreover, we performed a classification of the bugs according to their nature.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0960-0833
1099-1689
DOI:10.1002/stvr.1751