Live programming in practice: A controlled experiment on state machines for robotic behaviors - Campusano, Miguel; Fabry, Johan; Bergel, Alexandre (Information and software technology, 2019)
Live programming environments are gaining momentum across multiple programming languages. A tenet of live programming is a development feedback cycle, resulting in faster development practices. Although practitioners of live programming consider it a positive inclusion in their workflow, no in-depth investigations have yet been conducted on its benefits in a realistic scenario, nor using complex API. This paper carefully studies the advantage of using live programming in defining nested state machines for robot behaviors. We analyzed two important aspects of developing robotic behaviors using these machines: program comprehension and program writing. We analyzed both development practices in terms of speed and accuracy. We conducted two controlled experiments, one for program comprehension and another for program writing. We measured the speed and accuracy of randomized assigned participants on completing programming tasks, against a baseline. In a robotic behavior context, we found that a live programming system for nested state machine programs does not significantly outperform a non-live language in program comprehension nor in program writing in terms of speed and accuracy. However, the feedback of test subjects indicates their preference for the live programming system. The results of this work seem to contradict the studies of live programming in other areas, even while participants still favor using live programming techniques. We learned that the complex API chosen in this work has a strong negative influence on the results. To the best of our knowledge, this is the first in-depth live programming experiment in a complex domain.