Taming Uncertainty in Critical Scenario Generation for Testing Automated Driving Systems
Scenario-based testing in simulation has become a cornerstone of industrial practice for systematically assessing autonomous driving systems across diverse and relevant situations. Generating critical scenarios is central to this methodology, yet it remains challenging due to the inherent uncertaint...
Gespeichert in:
| Veröffentlicht in: | 2025 IEEE Conference on Software Testing, Verification and Validation (ICST) S. 464 - 475 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Tagungsbericht |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
31.03.2025
|
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Scenario-based testing in simulation has become a cornerstone of industrial practice for systematically assessing autonomous driving systems across diverse and relevant situations. Generating critical scenarios is central to this methodology, yet it remains challenging due to the inherent uncertainties resulting from scenario parameterization. While parameterization is essential for modeling unpredictable factors, like weather, an excess of parameters hampers testing effectiveness. To address these challenges, this paper introduces a methodology that guides testers in selecting scenario parameters and managing the associated uncertainties. Our approach integrates specification-driven and optimization-based test generation with sensitivity analysis, enabling testers to assess the impact of scenario parameters on scenario criticality. We implemented our approach using well-established industry technologies and evaluated it in a highway case study on three reference search-based scenario generation methods with varying degrees of exploitativeness. Results from our evaluation suggest that reducing the parameter-induced uncertainty can improve the ability of some testing methods to identify critical scenarios while maintaining the diversity of input parameter values. |
|---|---|
| AbstractList | Scenario-based testing in simulation has become a cornerstone of industrial practice for systematically assessing autonomous driving systems across diverse and relevant situations. Generating critical scenarios is central to this methodology, yet it remains challenging due to the inherent uncertainties resulting from scenario parameterization. While parameterization is essential for modeling unpredictable factors, like weather, an excess of parameters hampers testing effectiveness. To address these challenges, this paper introduces a methodology that guides testers in selecting scenario parameters and managing the associated uncertainties. Our approach integrates specification-driven and optimization-based test generation with sensitivity analysis, enabling testers to assess the impact of scenario parameters on scenario criticality. We implemented our approach using well-established industry technologies and evaluated it in a highway case study on three reference search-based scenario generation methods with varying degrees of exploitativeness. Results from our evaluation suggest that reducing the parameter-induced uncertainty can improve the ability of some testing methods to identify critical scenarios while maintaining the diversity of input parameter values. |
| Author | Molin, Adam Gambi, Alessio Mateis, Cristinel Nickovic, Dejan Grosse, Selma |
| Author_xml | – sequence: 1 givenname: Selma surname: Grosse fullname: Grosse, Selma organization: DENSO AUTOMOTIVE Deutschland GmbH,Germany – sequence: 2 givenname: Adam surname: Molin fullname: Molin, Adam organization: DENSO AUTOMOTIVE Deutschland GmbH,Germany – sequence: 3 givenname: Dejan surname: Nickovic fullname: Nickovic, Dejan organization: AIT Austrian Institute of Technology GmbH,Austria – sequence: 4 givenname: Alessio surname: Gambi fullname: Gambi, Alessio organization: AIT Austrian Institute of Technology GmbH,Austria – sequence: 5 givenname: Cristinel surname: Mateis fullname: Mateis, Cristinel organization: AIT Austrian Institute of Technology GmbH,Austria |
| BookMark | eNo1j9FKwzAUhiPohc69gWBeYPUkaZvmclSdg4EX7cC7cZacSGBNJY1C316HevXBB98P_w27jGMkxu4FFEKAedi2XV9LU5tCgqyKH9UYUOUFWxptGqVEBY0o5TV763EI8Z3vo6WUMcQ88xB5m0IOFk-8sxQxhZFvKFLCHMbI_Zh4T1M-d-vPPA6YyfHHFL7OppunTMN0y648niZa_nHB9s9Pffuy2r1utu16twpCN3lltRX6CFSXQmvtrCqVlZX0oBAQauesoNqDJdkI4ZQHXyJocOiM9Ect1YLd_e4GIjp8pDBgmg__h9U3IdlReQ |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/ICST62969.2025.10989034 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings Accès UT - IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9798331508142 |
| EndPage | 475 |
| ExternalDocumentID | 10989034 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IL CBEJK RIE RIL |
| ID | FETCH-LOGICAL-i178t-c7c17b0e641777dc343c252f03a0a06ddc1e6f0ce2811d3f0f4a070dad92fb723 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001506893900041&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Thu May 29 05:57:30 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i178t-c7c17b0e641777dc343c252f03a0a06ddc1e6f0ce2811d3f0f4a070dad92fb723 |
| PageCount | 12 |
| ParticipantIDs | ieee_primary_10989034 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-March-31 |
| PublicationDateYYYYMMDD | 2025-03-31 |
| PublicationDate_xml | – month: 03 year: 2025 text: 2025-March-31 day: 31 |
| PublicationDecade | 2020 |
| PublicationTitle | 2025 IEEE Conference on Software Testing, Verification and Validation (ICST) |
| PublicationTitleAbbrev | ICST |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| Score | 1.9035597 |
| Snippet | Scenario-based testing in simulation has become a cornerstone of industrial practice for systematically assessing autonomous driving systems across diverse and... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 464 |
| SubjectTerms | Autonomous driving Autonomous vehicles Measurement Road transportation Scenario generation Scenario-based testing Sensitivity analysis Software algorithms Software testing Test pattern generators Testing Uncertainty |
| Title | Taming Uncertainty in Critical Scenario Generation for Testing Automated Driving Systems |
| URI | https://ieeexplore.ieee.org/document/10989034 |
| WOSCitedRecordID | wos001506893900041&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELZoxcAEiCLe8sCa4kdixyMqVLBUlZpK3So_zlIGElRSpP577CQFMTCw-SlLZ599tu-7D6F7zSKxkReJMZlIUkWzJLdgE--4AwJhTWndkk3I2SxfrdS8B6u3WBgAaJ3PYByT7V--q-02PpUFDVe5IjwdoIGUogNr9T5boerhdbIoBFMi4k9YNt63_sWb0h4b0-N_DniCRj8APDz_PlpO0QFUZ2hV6LeQw8tQ2_7jNztcVnhPVoAXFqpw861xF0o6ShwHkxQXMZBG6Pe4bepgn4LDT5syviPgPlz5CC2nz8XkJemJEZKSyrxJrLRUGgIipVJKZ3nKLcuYJ1wTTYRzloLwxALLKXXcE5_qoNpOO8W8kYyfo2FVV3CBsMwpAc-NBGNS6r3yLlhInoeN0hAN6hKNoljW713si_VeIld_lF-joyj8DrV3g4bNZgu36NB-NuXH5q6dsS_MI5rL |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5aBT2pWPFtDl5X89rN5ijV0mIthW6ht5JNJtCDu1K3gv_eZHerePDgLU8Ck9dkMt98CN1qFoiNXBLleZxEQtE4Sg2YyFlugYBfU1rXZBNyPE7nczVpweo1FgYAauczuAvJ-i_flmYdTGV-h6tUES620U4sBCMNXKv12vKV98PeNEuYSgIChcV3m_a_mFPqi6N_8M8hD1H3B4KHJ9-XyxHaguIYzTP96nN45mvrn_zqEy8LvKErwFMDhX_7lrgJJh1kjr1SirMQSsP3e1hXpddQweLH1TJYEnAbsLyLZv2nrDeIWmqEaEllWkVGGipzAomgUkpruOCGxcwRrokmibWGQuKIAZZSarkjTmi_ua22irlcMn6COkVZwCnCMqUEHM8l5LmgzilnvY7kuD8qc6JBnaFuEMvirYl-sdhI5PyP8hu0N8heRovRcPx8gfbDRDQYvkvUqVZruEK75qNavq-u69n7AgcOnhI |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2025+IEEE+Conference+on+Software+Testing%2C+Verification+and+Validation+%28ICST%29&rft.atitle=Taming+Uncertainty+in+Critical+Scenario+Generation+for+Testing+Automated+Driving+Systems&rft.au=Grosse%2C+Selma&rft.au=Molin%2C+Adam&rft.au=Nickovic%2C+Dejan&rft.au=Gambi%2C+Alessio&rft.date=2025-03-31&rft.pub=IEEE&rft.spage=464&rft.epage=475&rft_id=info:doi/10.1109%2FICST62969.2025.10989034&rft.externalDocID=10989034 |