Taming Uncertainty in Critical Scenario Generation for Testing Automated Driving Systems

Scenario-based testing in simulation has become a cornerstone of industrial practice for systematically assessing autonomous driving systems across diverse and relevant situations. Generating critical scenarios is central to this methodology, yet it remains challenging due to the inherent uncertaint...

Full description

Saved in:
Bibliographic Details
Published in:2025 IEEE Conference on Software Testing, Verification and Validation (ICST) pp. 464 - 475
Main Authors: Grosse, Selma, Molin, Adam, Nickovic, Dejan, Gambi, Alessio, Mateis, Cristinel
Format: Conference Proceeding
Language:English
Published: IEEE 31.03.2025
Subjects:
Autonomous driving
Autonomous vehicles
Measurement
Road transportation
Scenario generation
Scenario-based testing
Sensitivity analysis
Software algorithms
Software testing
Test pattern generators
Testing
Uncertainty
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Scenario-based testing in simulation has become a cornerstone of industrial practice for systematically assessing autonomous driving systems across diverse and relevant situations. Generating critical scenarios is central to this methodology, yet it remains challenging due to the inherent uncertainties resulting from scenario parameterization. While parameterization is essential for modeling unpredictable factors, like weather, an excess of parameters hampers testing effectiveness. To address these challenges, this paper introduces a methodology that guides testers in selecting scenario parameters and managing the associated uncertainties. Our approach integrates specification-driven and optimization-based test generation with sensitivity analysis, enabling testers to assess the impact of scenario parameters on scenario criticality. We implemented our approach using well-established industry technologies and evaluated it in a highway case study on three reference search-based scenario generation methods with varying degrees of exploitativeness. Results from our evaluation suggest that reducing the parameter-induced uncertainty can improve the ability of some testing methods to identify critical scenarios while maintaining the diversity of input parameter values.
DOI:10.1109/ICST62969.2025.10989034