How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?
Uloženo v:
| Název: | How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception? |
|---|---|
| Autoři: | Birchler, Christian, Mohammed, Tanzil Kombarabettu, Rani, Pooja, Nechita, Teodora, Kehrer, Timo, Panichella, Sebastiano |
| Přispěvatelé: | University of Zurich |
| Zdroj: | Proceedings of the ACM on Software Engineering. 1:929-950 |
| Publication Status: | Preprint |
| Informace o vydavateli: | Association for Computing Machinery (ACM), 2024. |
| Rok vydání: | 2024 |
| Témata: | VR, 10009 Department of Informatics, Software Testing, 005: Computerprogrammierung, Programme und Daten, 02 engineering and technology, 000 Computer science, knowledge & systems, 006: Spezielle Computerverfahren, Software testing, Human perception, Computer Science - Software Engineering, Self-driving car, Self-Driving Cars, 0202 electrical engineering, electronic engineering, information engineering, Simulation, Human Perception |
| Popis: | Software metrics such as coverage or mutation scores have been investigated for the automated quality assessment of test suites. While traditional tools rely on software metrics, the field of self-driving cars (SDCs) has primarily focused on simulation-based test case generation using quality metrics such as the out-of-bound (OOB) parameter to determine if a test case fails or passes. However, it remains unclear to what extent this quality metric aligns with the human perception of the safety and realism of SDCs. To address this (reality) gap, we conducted an empirical study involving 50 participants to investigate the factors that determine how humans perceive SDC test cases as safe, unsafe, realistic, or unrealistic. To this aim, we developed a framework leveraging virtual reality (VR) technologies, called SDC-Alabaster, to immerse the study participants into the virtual environment of SDC simulators. Our findings indicate that the human assessment of safety and realism of failing/passing test cases can vary based on different factors, such as the test's complexity and the possibility of interacting with the SDC. Especially for the assessment of realism, the participants' age leads to a different perception. This study highlights the need for more research on simulation testing quality metrics and the importance of human perception in evaluating SDCs. |
| Druh dokumentu: | Article Other literature type Conference object |
| Popis souboru: | 3643768.pdf - application/pdf |
| Jazyk: | English |
| ISSN: | 2994-970X |
| DOI: | 10.1145/3643768 |
| DOI: | 10.18420/se2025-41 |
| DOI: | 10.48620/88819 |
| DOI: | 10.48620/88841 |
| DOI: | 10.21256/zhaw-30166 |
| DOI: | 10.5167/uzh-264107 |
| Přístupová URL adresa: | http://arxiv.org/abs/2401.14736 https://www.zora.uzh.ch/id/eprint/264107/ https://doi.org/10.5167/uzh-264107 |
| Rights: | CC BY |
| Přístupové číslo: | edsair.doi.dedup.....ce0fcaa3f3299cb8f963254c112efa3b |
| Databáze: | OpenAIRE |
Nájsť tento článok vo Web of Science | Abstrakt: | Software metrics such as coverage or mutation scores have been investigated for the automated quality assessment of test suites. While traditional tools rely on software metrics, the field of self-driving cars (SDCs) has primarily focused on simulation-based test case generation using quality metrics such as the out-of-bound (OOB) parameter to determine if a test case fails or passes. However, it remains unclear to what extent this quality metric aligns with the human perception of the safety and realism of SDCs. To address this (reality) gap, we conducted an empirical study involving 50 participants to investigate the factors that determine how humans perceive SDC test cases as safe, unsafe, realistic, or unrealistic. To this aim, we developed a framework leveraging virtual reality (VR) technologies, called SDC-Alabaster, to immerse the study participants into the virtual environment of SDC simulators. Our findings indicate that the human assessment of safety and realism of failing/passing test cases can vary based on different factors, such as the test's complexity and the possibility of interacting with the SDC. Especially for the assessment of realism, the participants' age leads to a different perception. This study highlights the need for more research on simulation testing quality metrics and the importance of human perception in evaluating SDCs. |
|---|---|
| ISSN: | 2994970X |
| DOI: | 10.1145/3643768 |