Prioritizing Test Inputs for DNNs Using Training Dynamics

Deep Neural Network (DNN) testing is one of the most widely-used techniques to guarantee the quality of DNNs. However, DNN testing typically requires the ground truth of test inputs, which is time-consuming and labor-intensive to obtain. To relieve the labeling-cost problem of DNN testing, we propos...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE/ACM International Conference on Automated Software Engineering : [proceedings] s. 1219 - 1231
Hlavní autoři: Shen, Jian, Li, Zhong, Pan, Minxue, Li, Xuandong
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: ACM 27.10.2024
Témata:
Artificial neural networks
Computer architecture
Debugging
Deep learning testing
Deep neural network
Feature extraction
Software
Software engineering
Software testing
Test input prioritization
Training
Training dynamics
Trajectory
ISSN:2643-1572
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Deep Neural Network (DNN) testing is one of the most widely-used techniques to guarantee the quality of DNNs. However, DNN testing typically requires the ground truth of test inputs, which is time-consuming and labor-intensive to obtain. To relieve the labeling-cost problem of DNN testing, we propose TDPR, a test input prioritization technique for DNNs based on training dynamics. The key insight of TDPR is that bug-revealing samples exhibit different learning trajectories compared to normal ones. Based on this, TDPR constructs a learning trajectory for each test input, which characterizes the evolving learning behavior of DNNs. Then, TDPR extracts features from these learning trajectories and applies learning-to-rank techniques to build a ranking model, which can intelligently utilize the generated features to prioritize test inputs. To evaluate TDPR, we conduct extensive experiments on 8 diverse subjects, considering various domains of test inputs, different DNN architectures, and diverse types of test inputs. The evaluation results demonstrate that TDPR outperforms 7 baseline approaches in both prioritizing test inputs and guiding the retraining of DNNs.CCS CONCEPTS* Software and its engineering → Software testing and debugging.
ISSN:2643-1572
DOI:10.1145/3691620.3695498