Testing vision-based control systems using learnable evolutionary algorithms

Vision-based control systems are key enablers of many autonomous vehicular systems, including self-driving cars. Testing such systems is complicated by complex and multidimensional input spaces. We propose an automated testing algorithm that builds on learnable evolutionary algorithms. These algorit...

Full description

Saved in:
Bibliographic Details
Published in:2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE) pp. 1016 - 1026
Main Authors: Abdessalem, Raja Ben, Nejati, Shiva, Briand, Lionel C., Stifter, Thomas
Format: Conference Proceeding
Language:English
Published: New York, NY, USA ACM 27.05.2018
Series:ACM Conferences
Subjects:
Automotive engineering
Automotive Software Systems
Classification algorithms
Control systems
Decision trees
Evolutionary algorithms
Evolutionary computation
Roads
Search-based Software Engineering
Software and its engineering > Software creation and management > Search-based software engineering
Software and its engineering > Software creation and management > Software verification and validation > Software defect analysis > Software testing and debugging
Software Testing
Testing
automotive software systems
software testing
evolutionary algorithms
search-based software engineering
ISBN:9781450356381, 1450356389
ISSN:1558-1225
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Vision-based control systems are key enablers of many autonomous vehicular systems, including self-driving cars. Testing such systems is complicated by complex and multidimensional input spaces. We propose an automated testing algorithm that builds on learnable evolutionary algorithms. These algorithms rely on machine learning or a combination of machine learning and Darwinian genetic operators to guide the generation of new solutions (test scenarios in our context). Our approach combines multiobjective population-based search algorithms and decision tree classification models to achieve the following goals: First, classification models guide the search-based generation of tests faster towards critical test scenarios (i.e., test scenarios leading to failures). Second, search algorithms refine classification models so that the models can accurately characterize critical regions (i.e., the regions of a test input space that are likely to contain most critical test scenarios). Our evaluation performed on an industrial automotive automotive system shows that: (1) Our algorithm outperforms a baseline evolutionary search algorithm and generates 78% more distinct, critical test scenarios compared to the baseline algorithm. (2) Our algorithm accurately characterizes critical regions of the system under test, thus identifying the conditions that are likely to lead to system failures.
ISBN:9781450356381
1450356389
ISSN:1558-1225
DOI:10.1145/3180155.3180160