Software test results exploration and visualization with continuous integration and nightly testing

Software testing is key for quality assurance of embedded systems. However, with increased development pace, the amount of test results data risks growing to a level where exploration and visualization of the results are unmanageable. This paper covers a tool, Tim, implemented at a company developin...

Full description

Saved in:
Bibliographic Details
Published in:International journal on software tools for technology transfer Vol. 24; no. 2; pp. 261 - 285
Main Authors: Strandberg, Per Erik, Afzal, Wasif, Sundmark, Daniel
Format: Journal Article
Language:English
Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.04.2022
Springer Nature B.V
Subjects:
Anomalies
Computer Science
Embedded systems
Exploration
Meetings
Quality assurance
Risk management
Software
Software development
Software Engineering
Software Engineering/Programming and Operating Systems
Software testing
Source code
Test results
Theory of Computation
Visual analytics
Visualization
Software testing
Visual analytics
Test results
ISSN:1433-2779, 1433-2787, 1433-2787
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Software testing is key for quality assurance of embedded systems. However, with increased development pace, the amount of test results data risks growing to a level where exploration and visualization of the results are unmanageable. This paper covers a tool, Tim, implemented at a company developing embedded systems, where software development occurs in parallel branches and nightly testing is partitioned over software branches, test systems and test cases. Tim aims to replace a previous solution with problems of scalability, requirements and technological flora. Tim was implemented with a reference group over several months. For validation, data were collected both from reference group meetings and logs from the usage of the tool. Data were analyzed quantitatively and qualitatively. The main contributions from the study include the implementation of eight views for test results exploration and visualization, the identification of four solutions patterns for these views (filtering, aggregation, previews and comparisons), as well as six challenges frequently discussed at reference group meetings (expectations, anomalies, navigation, integrations, hardware details and plots). Results are put in perspective with related work and future work is proposed, e.g., enhanced anomaly detection and integrations with more systems such as risk management, source code and requirements repositories.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1433-2779
1433-2787
1433-2787
DOI:10.1007/s10009-022-00647-1