Effects of specifying robotic missions in behavior trees and state machines - Dragule, Swaib; Bainomugisha, Engineer; Pelliccione, Patrizio; Berger, Thorsten (Journal of computer languages (Online), 2025)
The task of defining the robot’s mission is moving from professional developers and roboticists to the end-users. Robot missions, traditionally implemented in source code with text-based programming languages, present challenges for non-programmers. To this end, many domain-specific languages (DSLs) have been established in robotics. They are typically built upon an established paradigm, where behavior trees and state machines have become the most popular ones in robotics. These paradigms offer different levels of abstraction and control structures, which promise to improve the comprehension, correctness, and usability of missions. However, so far, there are no evaluation and validation studies to determine the effects of using either paradigm for mission specification by end-users. We present a controlled experiment on the effectiveness and efficiency of these paradigms for specifying robot missions by end-users. It measures mission comprehension, correctness and usability by examining language constructs, documentation, and usage. Our findings indicate that participants rated both paradigms above the neutral midpoint that is, greater than three on a 5-point scale in comprehension, with negligible variance in preference. However, state machine received marginally higher ratings in overall usability. The results further indicate that in the concrete syntax of the DSLs used in the experiments, user interfaces need improvement, more tutorials (including videos/audios) are required. End-users also need basic training in behavior trees, state machines, programming, and robotics. While the DSLs provide reasonable abstraction compared to text-based languages, further refinement is needed to enhance usability and correctness. We discuss actionable insights for improving the usability of behavior trees and state machines in robotics and automation. [Display omitted] •Comprehension — Behavior Trees (Groot) Above-average comprehension. State Machines (FlexBE) Slightly better comprehension, higher usability.•Abstraction Level — Both DSLs provide reasonable abstraction over text-based languages.•Mission Correctness — Comparable results, with emphasis on the need for improved mission correctness.•Usability — Both tools need better UIs. More tutorials, including videos/audios, are required. Users require basic training in behavior trees, state machines, programming, and robotics.•Actionable Insights — Enhance UIs and documentation, develop comprehensive tutorials, and provide basic training for end-users.