Source code properties of defective infrastructure as code scripts

In continuous deployment, software and services are rapidly deployed to end-users using an automated deployment pipeline. Defects in infrastructure as code (IaC) scripts can hinder the reliability of the automated deployment pipeline. We hypothesize that certain properties of IaC source code such as...

Full description

Saved in:
Bibliographic Details
Published in:Information and software technology Vol. 112; pp. 148 - 163
Main Authors: Rahman, Akond, Williams, Laurie
Format: Journal Article
Language:English
Published: Elsevier B.V 01.08.2019
Subjects:
Configuration as code
Continuous deployment
Defect prediction
Devops
Empirical study
Infrastructure as code
Puppet
Infrastructure as code
Configuration as code
Continuous deployment
Empirical study
Devops
Puppet
Defect prediction
ISSN:0950-5849, 1873-6025
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In continuous deployment, software and services are rapidly deployed to end-users using an automated deployment pipeline. Defects in infrastructure as code (IaC) scripts can hinder the reliability of the automated deployment pipeline. We hypothesize that certain properties of IaC source code such as lines of code and hard-coded strings used as configuration values, show correlation with defective IaC scripts. The objective of this paper is to help practitioners in increasing the quality of infrastructure as code (IaC) scripts through an empirical study that identifies source code properties of defective IaC scripts. We apply qualitative analysis on defect-related commits mined from open source software repositories to identify source code properties that correlate with defective IaC scripts. Next, we survey practitioners to assess the practitioner’s agreement level with the identified properties. We also construct defect prediction models using the identified properties for 2439 scripts collected from four datasets. We identify 10 source code properties that correlate with defective IaC scripts. Of the identified 10 properties we observe lines of code and hard-coded string i.e. putting strings as configuration values, to show the strongest correlation with defective IaC scripts. According to our survey analysis, majority of the practitioners show agreement for two properties: include, the property of executing external modules or scripts, and hard-coded string. Using the identified properties, our constructed defect prediction models show a precision of 0.70∼0.78, and a recall of 0.54∼0.67. Based on our findings, we recommend practitioners to allocate sufficient inspection and testing efforts on IaC scripts that include any of the identified 10 source code properties of IaC scripts.
ISSN:0950-5849
1873-6025
DOI:10.1016/j.infsof.2019.04.013