Identifying Underrepresented Driving Scenarios in Training Datasets with SafeML Robustness Monitoring for Autonomous Driving Vehicles

This study investigates the evaluation of the representativeness of datasets in autonomous driving systems using statistical distance metrics. Eight different datasets, created in the Carla simulation environment, cover various driving conditions, including normal and impaired lighting scenarios, al...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2024 8th International Conference on System Reliability and Safety (ICSRS) s. 245 - 253
Hlavní autori: Matthias, Bergler, Ramin, Tavakoli-Kolagari, Kristina, Lundqvist
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 20.11.2024
Predmet:
Autonomous Driving
Autonomous vehicles
Lighting
Monitoring
Power measurement
Power system reliability
Real-time systems
Ro-bustness
Robustness
SafeML
Safety
Simulation
Time measurement
Training
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:This study investigates the evaluation of the representativeness of datasets in autonomous driving systems using statistical distance metrics. Eight different datasets, created in the Carla simulation environment, cover various driving conditions, including normal and impaired lighting scenarios, alternative routes, and challenging conditions such as power outages. The datasets were designed to represent similar routes under different conditions. Various distance metrics- Wasserstein, Kuiper, Anderson-Darling, Chernoff, DTS, and CVM-were applied to measure pairwise dataset distances. We anticipated that the dataset for a given route under ideal conditions would exhibit a large distance measure (of any of the listed distance measures) compared to the same route under impaired conditions (e.g., a power failure at the streetlights). However, we were particularly interested in whether a measurable jump at a (potential threshold) value could be recognized even with a smaller drop in dataset condition quality. The results of the study show that a normalization of these distance measures enables precise divergence comparisons and the determination of meaningful threshold values. This in turn means that normalized deviation measures can effectively identify deviations in real time, hence contributing to the development and monitoring of more reliable autonomous driving models.
DOI:10.1109/ICSRS63046.2024.10927539