Predicting Code Comprehension: A Novel Approach to Align Human Gaze with Code using Deep Neural Networks

The better the code quality and the less complex the code, the easier it is for software developers to comprehend and evolve it. Yet, how do we best detect quality concerns in the code? Existing measures to assess code quality, such as McCabe’s cyclomatic complexity, are decades old and neglect the...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Proceedings of the ACM on software engineering Ročník 1; číslo FSE; s. 1982 - 2004
Hlavní autori: Alakmeh, Tarek, Reich, David, Jäger, Lena, Fritz, Thomas
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York, NY, USA ACM 12.07.2024
Predmet:
Computing methodologies
Empirical software validation
Human-centered computing
Laboratory experiments
Neural networks
Software and its engineering
lab experiment
code-fixation attention
eye-tracking
code comprehension
neural networks
ISSN:2994-970X, 2994-970X
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:The better the code quality and the less complex the code, the easier it is for software developers to comprehend and evolve it. Yet, how do we best detect quality concerns in the code? Existing measures to assess code quality, such as McCabe’s cyclomatic complexity, are decades old and neglect the human aspect. Research has shown that considering how a developer reads and experiences the code can be an indicator of its quality. In our research, we built on these insights and designed, trained, and evaluated the first deep neural network that aligns a developer’s eye gaze with the code tokens the developer looks at to predict code comprehension and perceived difficulty. To train and analyze our approach, we performed an experiment in which 27 participants worked on a range of 16 short code comprehension tasks while we collected fine-grained gaze data using an eye tracker. The results of our evaluation show that our deep neural sequence model that integrates both the human gaze and the stimulus code, can predict (a) code comprehension and (b) the perceived code difficulty significantly better than current state-of-the-art reference methods. We also show that aligning human gaze with code leads to better performance than models that rely solely on either code or human gaze. We discuss potential applications and propose future work to build better human-inclusive code evaluation systems.
ISSN:2994-970X
2994-970X
DOI:10.1145/3660795