Utilising CI environment for efficient and effective testing of NFRs

Continuous integration (CI) is a practice that aims to continuously verify quality aspects of a software intensive system both for functional and non-functional requirements (NFRs). Functional requirements are the inputs of development and can be tested in isolation, utilising either manual or autom...

Full description

Saved in:
Bibliographic Details
Published in:Information and software technology Vol. 117; p. 106199
Main Authors: Yu, Liang, Alégroth, Emil, Chatzipetrou, Panagiota, Gorschek, Tony
Format: Journal Article
Language:English
Published: Elsevier B.V 01.01.2020
Subjects:
Agile
Computer software selection and evaluation
Continuous integration
Continuous integrations
DevOps
Integral equations
NFR
Non-functional requirement
Non-functional requirements
Quality control
SAFe
Scaled agile framework
Software design
DevOps
Non-functional requirement
Scaled agile framework
CI
Agile
Continuous integration
NFR
SAFe
ISSN:0950-5849, 1873-6025, 1873-6025
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Continuous integration (CI) is a practice that aims to continuously verify quality aspects of a software intensive system both for functional and non-functional requirements (NFRs). Functional requirements are the inputs of development and can be tested in isolation, utilising either manual or automated tests. In contrast, some NFRs are difficult to test without functionality, for NFRs are often aspects of functionality and express quality aspects. Lacking this testability attribute makes NFR testing complicated and, therefore, underrepresented in industrial practice. However, the emergence of CI has radically affected software development and created new avenues for software quality evaluation and quality information acquisition. Research has, consequently, been devoted to the utilisation of this additional information for more efficient and effective NFR verification. We aim to identify the state-of-the-art of utilising the CI environment for NFR testing, hereinafter referred to as CI-NFR testing. Through rigorous selection, from an initial set of 747 papers, we identified 47 papers that describe how NFRs are tested in a CI environment. Evidence-based analysis, through coding, is performed on the identified papers in this SLR. Firstly, ten CI approaches are described by the papers selected, each describing different tools and nine different NFRs where reported to be tested. Secondly, although possible, CI-NFR testing is associated with eight challenges that adversely affect its adoption. Thirdly, the identified CI-NFR testing processes are tool-driven, but there is a lack of NFR testing tools that can be used in the CI environment. Finally, we proposed a CI framework for NFRs testing. A synthesised CI framework is proposed for testing various NFRs, and associated CI tools are also mapped. This contribution is valuable as results of the study also show that CI-NFR testing can help improve the quality of NFR testing in practices.
ISSN:0950-5849
1873-6025
1873-6025
DOI:10.1016/j.infsof.2019.106199