Predicting Code Comprehension: A Novel Approach to Align Human Gaze with Code using Deep Neural Networks
The better the code quality and the less complex the code, the easier it is for software developers to comprehend and evolve it. Yet, how do we best detect quality concerns in the code? Existing measures to assess code quality, such as McCabe’s cyclomatic complexity, are decades old and neglect the...
Saved in:
| Published in: | Proceedings of the ACM on software engineering Vol. 1; no. FSE; pp. 1982 - 2004 |
|---|---|
| Main Authors: | , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
New York, NY, USA
ACM
12.07.2024
|
| Subjects: | |
| ISSN: | 2994-970X, 2994-970X |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | The better the code quality and the less complex the code, the easier it is for software developers to comprehend and evolve it. Yet, how do we best detect quality concerns in the code? Existing measures to assess code quality, such as McCabe’s cyclomatic complexity, are decades old and neglect the human aspect. Research has shown that considering how a developer reads and experiences the code can be an indicator of its quality. In our research, we built on these insights and designed, trained, and evaluated the first deep neural network that aligns a developer’s eye gaze with the code tokens the developer looks at to predict code comprehension and perceived difficulty. To train and analyze our approach, we performed an experiment in which 27 participants worked on a range of 16 short code comprehension tasks while we collected fine-grained gaze data using an eye tracker. The results of our evaluation show that our deep neural sequence model that integrates both the human gaze and the stimulus code, can predict (a) code comprehension and (b) the perceived code difficulty significantly better than current state-of-the-art reference methods. We also show that aligning human gaze with code leads to better performance than models that rely solely on either code or human gaze. We discuss potential applications and propose future work to build better human-inclusive code evaluation systems. |
|---|---|
| AbstractList | The better the code quality and the less complex the code, the easier it is for software developers to comprehend and evolve it. Yet, how do we best detect quality concerns in the code? Existing measures to assess code quality, such as McCabe’s cyclomatic complexity, are decades old and neglect the human aspect. Research has shown that considering how a developer reads and experiences the code can be an indicator of its quality. In our research, we built on these insights and designed, trained, and evaluated the first deep neural network that aligns a developer’s eye gaze with the code tokens the developer looks at to predict code comprehension and perceived difficulty. To train and analyze our approach, we performed an experiment in which 27 participants worked on a range of 16 short code comprehension tasks while we collected fine-grained gaze data using an eye tracker. The results of our evaluation show that our deep neural sequence model that integrates both the human gaze and the stimulus code, can predict (a) code comprehension and (b) the perceived code difficulty significantly better than current state-of-the-art reference methods. We also show that aligning human gaze with code leads to better performance than models that rely solely on either code or human gaze. We discuss potential applications and propose future work to build better human-inclusive code evaluation systems. |
| ArticleNumber | 88 |
| Author | Alakmeh, Tarek Fritz, Thomas Jäger, Lena Reich, David |
| Author_xml | – sequence: 1 givenname: Tarek orcidid: 0009-0008-5512-3549 surname: Alakmeh fullname: Alakmeh, Tarek email: tarek.alakmeh@uzh.ch organization: University of Zurich, Zurich, Switzerland – sequence: 2 givenname: David orcidid: 0000-0002-3524-3788 surname: Reich fullname: Reich, David email: david.reich@uni-potsdam.de organization: University of Potsdam, Zurich, Germany – sequence: 3 givenname: Lena orcidid: 0000-0001-9018-9713 surname: Jäger fullname: Jäger, Lena email: jaeger@cl.uzh.ch organization: University of Zurich, Zurich, Switzerland – sequence: 4 givenname: Thomas orcidid: 0000-0002-1834-6240 surname: Fritz fullname: Fritz, Thomas email: fritz@ifi.uzh.ch organization: University of Zurich, Zurich, Switzerland |
| BookMark | eNptkM1LAzEQxYNUsNbi3VNunlaTJvsRb0vVVpDqQcHbkiaz3ejuZklSi_71bmkVES_zhpnfPHhzjAatbQGhU0ouKOXxJUsSkor4AA0nQvBIpORl8Ks_QmPvXwkh_YTSlAxR9ehAGxVMu8JTq6EvTeeggtYb217hHC_sO9Q47zpnpapwsDivzarF83UjWzyTn4A3JlS767XfGl0DdHgBayfrXsLGujd_gg5LWXsY73WEnm9vnqbz6P5hdjfN7yNJJ5M4khwYMC4UKTMNMSVJH0dnWUmAxjxNU8FkQmkpFMsUYZpozQVnSxEzxvRSsxE63_kqZ713UBadM410HwUlxfZHxf5HPRn9IZUJMvSxg5Om_oc_2_FSNT-m38svgY1wxg |
| CitedBy_id | crossref_primary_10_1016_j_cola_2025_101327 crossref_primary_10_1145_3725840 |
| Cites_doi | 10.1145/3610177 10.1109/ICSM.2004.1357820 10.1109/CSEET.2016.36 10.1109/EMIP.2019.00012 10.1145/2578153.2578218 10.1162/neco.1997.9.8.1735 10.1109/ICPC52881.2021.00019 10.1109/ICSE43902.2021.00056 10.1016/j.visres.2019.05.003 10.1145/1864349 10.1145/2568225.2568266 10.1007/BF00977789 10.1007/s10664-019-09751-4 10.1145/3588015.3588410 10.1145/985921 10.1109/FIE.2018.8658592 10.1007/s10664-018-9664-z 10.1145/1067445 10.1109/ICSME46990.2020.00041 10.1145/1357054 10.1145/3448018.3457421 10.1016/j.ergon.2005.04.005 10.1016/0042-6989(85)90134-8 10.1145/1134285.1134349 10.1145/3025453.3025573 10.1145/3540250.3549084 10.1145/3448018.3457424 10.3389/fpsyg.2022.871143 10.1145/2987491.2987536 10.1109/ICSE.2005.1553571 10.1145/1117309 10.1109/2.402076 10.48550/arxiv.2009.08366 10.1007/s10586-017-0746-2 10.1109/TSE.2009.50 10.5555/54968.54975 10.18653/v1 10.1145/2786805.2786838 10.1145/2025113.2025156 10.48550/arxiv.2002.08155 10.1007/11550907_126 10.18260/1-2 10.1145/2168556 10.1109/TSE.1987.233475 10.1109/78.650093 10.1016/S0020-7373(83)80031-5 10.1109/ICPC.2015.36 10.1037/0033-295X.87.4.329 10.1145/3626522 10.1109/ICPC52881.2021.00025 10.3758/s13414-011-0219-2 10.1109/TSE.1976.233837 10.1109/ICPC.2012.6240511 10.1016/0164-1212(79)90022-0 10.1037/0033-2909.124.3.372 10.1016/j.ijhcs.2011.09.003 10.1007/s11704-020-0422-1 10.1145/2635868.2635895 10.1145/2884781.2884803 10.1109/PRDC.2007.28 |
| ContentType | Journal Article |
| Copyright | Owner/Author |
| Copyright_xml | – notice: Owner/Author |
| DBID | AAYXX CITATION |
| DOI | 10.1145/3660795 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | CrossRef |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 2994-970X |
| EndPage | 2004 |
| ExternalDocumentID | 10_1145_3660795 3660795 |
| GroupedDBID | AAKMM ACM AEJOY AKRVB ALMA_UNASSIGNED_HOLDINGS LHSKQ M~E AAYXX CITATION ROL |
| ID | FETCH-LOGICAL-a1225-a4e3e349c0f8de5106795d88f0e15477793a611f9c38c03d0dd4943b95333dbd3 |
| ISSN | 2994-970X |
| IngestDate | Tue Nov 18 21:33:43 EST 2025 Sat Nov 29 07:49:00 EST 2025 Mon Jul 14 20:49:06 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | FSE |
| Keywords | lab experiment code-fixation attention eye-tracking code comprehension neural networks |
| Language | English |
| License | This work is licensed under a Creative Commons Attribution International 4.0 License. https://creativecommons.org/licenses/by/4.0 |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-a1225-a4e3e349c0f8de5106795d88f0e15477793a611f9c38c03d0dd4943b95333dbd3 |
| ORCID | 0000-0002-1834-6240 0000-0001-9018-9713 0000-0002-3524-3788 0009-0008-5512-3549 |
| OpenAccessLink | https://dl.acm.org/doi/10.1145/3660795 |
| PageCount | 23 |
| ParticipantIDs | crossref_primary_10_1145_3660795 crossref_citationtrail_10_1145_3660795 acm_primary_3660795 |
| PublicationCentury | 2000 |
| PublicationDate | 20240712 2024-07-12 |
| PublicationDateYYYYMMDD | 2024-07-12 |
| PublicationDate_xml | – month: 07 year: 2024 text: 20240712 day: 12 |
| PublicationDecade | 2020 |
| PublicationPlace | New York, NY, USA |
| PublicationPlace_xml | – name: New York, NY, USA |
| PublicationTitle | Proceedings of the ACM on software engineering |
| PublicationTitleAbbrev | ACM PACMSE |
| PublicationYear | 2024 |
| Publisher | ACM |
| Publisher_xml | – name: ACM |
| References | Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. Findings of the Association for Computational Linguistics Findings of ACL: EMNLP 2020, 2, 1536–1547. isbn:9781952148903 https://doi.org/10.48550/arxiv.2002.08155 10.48550/arxiv.2002.08155 Seolhwa Lee, Danial Hooshyar, Hyesung Ji, Kichun Nam, and Heuiseok Lim. 2018. Mining biometric data to predict programmer expertise and task difficulty. Cluster Computing, https://doi.org/10.1007/s10586-017-0746-2 10.1007/s10586-017-0746-2 Jacob Devlin, Ming Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1 (2019), 4171–4186. isbn:9781950737130 https://doi.org/10.18653/v1/N19-1423 10.18653/v1/N19-1423 Hiroto Harada and Minoru Nakayama. 2021. Estimation of Reading Ability of Program Codes Using Features of Eye Movements. In ACM Symposium on Eye Tracking Research and Applications (ETRA ’21 Short Papers). Association for Computing Machinery, New York, NY, USA. Article 32, 5 pages. isbn:9781450383455 https://doi.org/10.1145/3448018.3457421 10.1145/3448018.3457421 Stephanie Yang, Amreen Amin Poonawala, Tian-Shun Allan Jiang, and Bertrand Schneider. 2023. Can Synchronous Code Editing and Awareness Tools Support Remote Tutoring? Effects on Learning and Teaching. Proc. ACM Hum.-Comput. Interact., 7, CSCW2 (2023), Article 328, oct, 30 pages. https://doi.org/10.1145/3610177 10.1145/3610177 Kristoffer Gunnarsson and Olivia Herber. 2020. The Most Popular Programming Languages of GitHub’s Trending Repositories. Ben Shneiderman and Richard Mayer. 1979. Syntactic/Semantic Interactions in Programmer Behavior: A Model and Experimental Results. International Journal of Computer & Information Sciences, 8, 3 (1979), https://doi.org/10.1007/BF00977789 10.1007/BF00977789 Anneliese Von Mayrhauser and A. Marie Vans. 1995. Program Comprehension During Software Maintenance and Evolution. Computer, 28, 8 (1995), 44–55. issn:00189162 https://doi.org/10.1109/2.402076 10.1109/2.402076 Meir M Lehman. 1980. On understanding laws, evolution, and conservation in the large-program life cycle. Journal of Systems and Software, 1 (1980), 213–221. https://doi.org/10.1016/0164-1212(79)90022-0 10.1016/0164-1212(79)90022-0 Keith Rayner. 1998. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 3 (1998), Nov., 372–422. https://doi.org/10.1037/0033-2909.124.3.372 10.1037/0033-2909.124.3.372 SR Research Ltd. 2020. SR Research Experiment Builder. Guido Salvaneschi, Sven Amann, Sebastian Proksch, and Mira Mezini. 2014. An empirical study on program comprehension with reactive programming. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2014). Association for Computing Machinery, New York, NY, USA. 564–575. isbn:978-1-4503-3056-5 https://doi.org/10.1145/2635868.2635895 10.1145/2635868.2635895 Lisa Schibelius, Amanda Ross, and Andrew Katz. 2022. An Empirical Study of Programming Languages Specified in Engineering Job Postings. In ASEE Annual Conference & Exposition. https://doi.org/10.18260/1-2–41235 10.18260/1-2--41235 N. Moha, Y. Guéhéneuc, L. Duchien, and A. Le Meur. 2010. DECOR: A Method for the Specification and Detection of Code and Design Smells. IEEE Transactions on Software Engineering, 36, 1 (2010), 20–36. https://doi.org/10.1109/TSE.2009.50 10.1109/TSE.2009.50 SR Research. 2023. https://www.sr-research.com/eyelink-portable-duo/ Retrieved Feb 1, 2023 Elliot Soloway, Beth Adelson, and Kate Ehrlich. 1988. Knowledge and processes in the comprehension of computer programs. In The nature of expertise.. Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, US. 129–152. isbn:0-89859-711-0 (Hardcover) Todd Sedano. 2016. Code Readability Testing, an Empirical Study. In 2016 IEEE 29th International Conference on Software Engineering Education and Training (CSEET). 111–117. https://doi.org/10.1109/CSEET.2016.36 ISSN: 2377-570X 10.1109/CSEET.2016.36 Norman Peitek, Annabelle Bergum, Maurice Rekrut, Jonas Mucke, Matthias Nadig, Chris Parnin, Janet Siegmund, and Sven Apel. 2022. Correlates of Programmer Efficacy and Their Link to Experience: A Combined EEG and Eye-Tracking Study. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2022). 120–131. https://doi.org/10.1145/3540250.3549084 10.1145/3540250.3549084 Nachiappan Nagappan and Thomas Ball. 2005. Use of Relative Code Churn Measures to Predict System Defect Density. In Proceedings of the 27th international conference on Software engineering. 284–292. https://doi.org/10.1109/ICSE.2005.1553571 10.1109/ICSE.2005.1553571 Hidetake Uwano, Masahide Nakamura, Akito Monden, and Ken-Ichi Matsumoto. 2006. Analyzing Individual Performance of Source Code Review Using Reviewers’ Eye Movement. Proceedings of the 2006 symposium on Eye tracking research & applications - ETRA ’06, 133–140. isbn:1595933050 https://doi.org/10.1145/1117309 10.1145/1117309 Kenneth Holmqvist, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost Van de Weijer. 2011. Eye tracking: A comprehensive guide to methods and measures. Oxford University Press, Oxford. Thomas J. Mccabe. 1976. A Complexity Measure. IEEE Transactions on Software Engineering, SE-2, 4 (1976), 308–320. issn:00985589 https://doi.org/10.1109/TSE.1976.233837 10.1109/TSE.1976.233837 Mike Schuster and Kuldip Paliwal. 1997. Bidirectional recurrent neural networks. IEEE transactions on Signal Processing, 45, 11 (1997), 12, 2673–2681. https://doi.org/10.1109/78.650093 10.1109/78.650093 William H Dubay. 2004. The Principles of Readability. Shamsi T Iqbal, Sam Zheng, and Brian P Bailey. 2004. Task-Evoked Pupillary Response to Mental Workload in Human-Computer Interaction. Extended abstracts of the 2004 conference on Human factors and computing systems - CHI ’04, 1477–1480. isbn:1581137036 https://doi.org/10.1145/985921 10.1145/985921 Marcel Adam Just and Patricia A. Carpenter. 1980. A theory of reading: from eye fixations to comprehension.. Psychological review, 87 4 (1980), 329–54. https://doi.org/10.1037/0033-295X.87.4.329 10.1037/0033-295X.87.4.329 Michele Lacchia. 2022. Radon 4.1.0 documentation — Using radon programmatically. https://radon.readthedocs.io/en/latest/api.html Sarah D’Angelo and Andrew Begel. 2017. Improving Communication Between Pair Programmers Using Shared Gaze Awareness. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17). Association for Computing Machinery, New York, NY, USA. 6245–6290. isbn:9781450346559 https://doi.org/10.1145/3025453.3025573 10.1145/3025453.3025573 TIOBE Software BV. 2022. TIOBE Index - TIOBE. https://www.tiobe.com/tiobe-index Sarah Fakhoury, Devjeet Roy, Yuzhan Ma, Venera Arnaoudova, and Olusola Adesope. 2020. Measuring the impact of lexical and structural inconsistencies on developers’ cognitive load during bug localization. Empirical Software Engineering, 25, 3 (2020), May, 2140–2178. issn:1382-3256, 1573-7616 https://doi.org/10.1007/s10664-019-09751-4 10.1007/s10664-019-09751-4 Jeffrey Dean and Rajat Monga. 2015. TensorFlow: Large-scale machine learning on heterogeneous systems. https://www.tensorflow.org Radu Marinescu. 2004. Detection Strategies: Metrics-Based Rules for Detecting Design Flaws. In Proceedings of the 20th IEEE International Conference on Software Maintenance. 350–359. https://doi.org/10.1109/ICSM.2004.1357820 10.1109/ICSM.2004.1357820 Sam. 2020. What determines whether a validation is "GOOD", "FAIR" or "POOR"? https://www.sr-research.com/support/thread-244.html Wade M Vagias. 2006. Likert-Type Scale Response Anchors Level of Agreement. Clemson International Institute for Tourism & Research Development, Department of Parks, Recreation and Tourism Management. Clemson University.. Janet Feigenspan, Christian Kästner, Jörg Liebig, Sven Apel, and Stefan Hanenberg. 2012. Measuring programming experience. In 2012 20th IEEE International Conference on Program Comprehension (ICPC). 73–82. https://doi.org/10.1109/ICPC.2012.6240511 ISSN: 1092-8138 10.1109/ICPC.2012.6240511 Hongyu Zhang, Xiuzhen Zhang, and Ming Gu. 2007. Predicting Defective Software Components from Code Complexity Measures. In 13th Pacific Rim International Symposium on Dependable Computing (PRDC 2007). 93–96. https://doi.org/10.1109/PRDC.2007.28 10.1109/PRDC.2007.28 Norman Peitek, Sven Apel, Chris Parnin, André Brechmann, and Janet Siegmund. 2021. Program Comprehension and Code Complexity Metrics: An fMRI Study. In Proceedings of the 43rd International Conference on Software Engineering (ICSE ’21). IEEE Press, 524–536. isbn:9781450390859 https://doi.org/10.1109/ICSE43902.2021.00056 10.1109/ICSE43902.2021.00056 Marvin Wyrich, Justus Bogner, and Stefan Wagner. 2023. 40 Years of Designing Code Comprehension Experiments: A Systematic Mapping Study. ACM Comput. Surv., 56, 4 (2023), Article 106, 42 pages. issn:0360-0300 https://doi.org/10.1145/3626522 10.1145/3626522 2007. Eye movements: A window on mind and brain, Roger P. G. van Gompel, Martin H. Fischer, Wayne S. Murray, and Robin L. Hill (Eds.). Elsevier. Talita Vieira Ribeiro and Guilherme Travassos. 2017. Who is Right? Evaluating Empirical Contradictions in the Readability and Comprehensibility of Source Code. Naser Al Madi, Cole S. Peterson, Bonita Sharif, and Jonathan I. Maletic. 2021. From Novice to Expert: Analysis of Token Level Effects in a Longitudinal Eye Tracking Study. In 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC). 172–183. https://doi.org/10.1109/ICPC52881.2021.00025 10.1109/ICPC52881.2021.00025 Moshe Eizenman, P.E Hallett, and R.C. Frecker. 1985 e_1_2_1_81_1 Soloway Elliot (e_1_2_1_70_1) e_1_2_1_20_1 e_1_2_1_41_1 e_1_2_1_66_1 e_1_2_1_68_1 e_1_2_1_24_1 e_1_2_1_45_1 e_1_2_1_62_1 e_1_2_1_83_1 e_1_2_1_22_1 e_1_2_1_43_1 e_1_2_1_64_1 e_1_2_1_28_1 e_1_2_1_49_1 e_1_2_1_26_1 e_1_2_1_47_1 Holmqvist Kenneth (e_1_2_1_35_1) e_1_2_1_71_1 e_1_2_1_31_1 e_1_2_1_54_1 Vagias Wade M (e_1_2_1_77_1) e_1_2_1_56_1 e_1_2_1_79_1 e_1_2_1_6_1 e_1_2_1_12_1 e_1_2_1_50_1 e_1_2_1_73_1 e_1_2_1_4_1 e_1_2_1_10_1 e_1_2_1_33_1 e_1_2_1_52_1 e_1_2_1_75_1 e_1_2_1_16_1 e_1_2_1_39_1 e_1_2_1_14_1 e_1_2_1_37_1 Whalley Jacqueline L. (e_1_2_1_80_1) 2006; 52 e_1_2_1_58_1 e_1_2_1_18_1 e_1_2_1_82_1 e_1_2_1_42_1 e_1_2_1_65_1 e_1_2_1_40_1 e_1_2_1_67_1 e_1_2_1_23_1 e_1_2_1_46_1 e_1_2_1_61_1 Research SR (e_1_2_1_60_1) 2023 e_1_2_1_21_1 e_1_2_1_44_1 e_1_2_1_63_1 e_1_2_1_27_1 e_1_2_1_25_1 e_1_2_1_48_1 e_1_2_1_69_1 e_1_2_1_29_1 Boehm Barry W. (e_1_2_1_8_1) Abelson Harold (e_1_2_1_2_1) 1996 e_1_2_1_7_1 e_1_2_1_30_1 e_1_2_1_55_1 e_1_2_1_76_1 e_1_2_1_5_1 e_1_2_1_57_1 e_1_2_1_78_1 e_1_2_1_3_1 e_1_2_1_13_1 e_1_2_1_34_1 e_1_2_1_51_1 e_1_2_1_72_1 e_1_2_1_1_1 e_1_2_1_11_1 e_1_2_1_32_1 e_1_2_1_53_1 e_1_2_1_74_1 e_1_2_1_17_1 e_1_2_1_38_1 e_1_2_1_15_1 e_1_2_1_36_1 e_1_2_1_59_1 e_1_2_1_9_1 e_1_2_1_19_1 |
| References_xml | – reference: Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2020. GraphCodeBERT: Pre-training Code Representations with Data Flow. 9, https://doi.org/10.48550/arxiv.2009.08366 10.48550/arxiv.2009.08366 – reference: Keith Rayner. 1998. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 3 (1998), Nov., 372–422. https://doi.org/10.1037/0033-2909.124.3.372 10.1037/0033-2909.124.3.372 – reference: Hongyu Zhang, Xiuzhen Zhang, and Ming Gu. 2007. Predicting Defective Software Components from Code Complexity Measures. In 13th Pacific Rim International Symposium on Dependable Computing (PRDC 2007). 93–96. https://doi.org/10.1109/PRDC.2007.28 10.1109/PRDC.2007.28 – reference: Elizabeth R Schotter, Bernhard Angele, and Keith Rayner. 2012. Parafoveal processing in reading. Attention, perception & psychophysics, 74 (2012), 5–35. https://doi.org/10.3758/s13414-011-0219-2 10.3758/s13414-011-0219-2 – reference: Wolfgang Fuhl and Enkelejda Kasneci. 2018. Eye movement velocity and gaze data generator for evaluation, robustness testing and assess of eye tracking software and visualization tools. arXiv preprint arXiv:1808.09296. – reference: Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. Findings of the Association for Computational Linguistics Findings of ACL: EMNLP 2020, 2, 1536–1547. isbn:9781952148903 https://doi.org/10.48550/arxiv.2002.08155 10.48550/arxiv.2002.08155 – reference: H. F. Li and W. K. Cheung. 1987. An Empirical Study of Software Metrics. IEEE Transactions on Software Engineering, SE-13, 6 (1987), 697–708. issn:00985589 https://doi.org/10.1109/TSE.1987.233475 10.1109/TSE.1987.233475 – reference: Lisa Schibelius, Amanda Ross, and Andrew Katz. 2022. An Empirical Study of Programming Languages Specified in Engineering Job Postings. In ASEE Annual Conference & Exposition. https://doi.org/10.18260/1-2–41235 10.18260/1-2--41235 – reference: SR Research Ltd. 2016. EyeLink ® Portable Duo User Manual EyeLink ® SR Research. – reference: Paul Prasse, David Robert Reich, Silvia Makowski, Seoyoung Ahn, Tobias Scheffer, and Lena A. Jäger. 2023. SP-EyeGAN: Generating Synthetic Eye Movement Data with Generative Adversarial Networks. In 2023 Symposium on Eye Tracking Research and Applications. ACM, Tubingen Germany. 1–9. isbn:9798400701504 https://doi.org/10.1145/3588015.3588410 10.1145/3588015.3588410 – reference: Ben Shneiderman and Richard Mayer. 1979. Syntactic/Semantic Interactions in Programmer Behavior: A Model and Experimental Results. International Journal of Computer & Information Sciences, 8, 3 (1979), https://doi.org/10.1007/BF00977789 10.1007/BF00977789 – reference: Jacqueline L. Whalley, Raymond Lister, Errol Thompson, Tony Clear, Phil Robbins, P. K. Ajith Kumar, and Christine Prasad. 2006. An Australasian study of reading and comprehension skills in novice programmers, using the bloom and SOLO taxonomies. In Proceedings of the 8th Australasian Conference on Computing Education - Volume 52 (ACE ’06). Australian Computer Society, Inc., 243–252. isbn:1920682341 https://doi.org/10292/15405 – reference: Charles Bigelow. 2019. Typeface features and legibility research. Vision Research, 165 (2019), 12, 162–172. issn:18785646 https://doi.org/10.1016/j.visres.2019.05.003 10.1016/j.visres.2019.05.003 – reference: Janet Feigenspan, Christian Kästner, Jörg Liebig, Sven Apel, and Stefan Hanenberg. 2012. Measuring programming experience. In 2012 20th IEEE International Conference on Program Comprehension (ICPC). 73–82. https://doi.org/10.1109/ICPC.2012.6240511 ISSN: 1092-8138 10.1109/ICPC.2012.6240511 – reference: Nachiappan Nagappan and Thomas Ball. 2005. Use of Relative Code Churn Measures to Predict System Defect Density. In Proceedings of the 27th international conference on Software engineering. 284–292. https://doi.org/10.1109/ICSE.2005.1553571 10.1109/ICSE.2005.1553571 – reference: Taek Lee, Jaechang Nam, DongGyun Han, Sunghun Kim, and Hoh Peter In. 2011. Micro Interaction Metrics for Defect Prediction. In Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering. 311–321. https://doi.org/10.1145/2025113.2025156 10.1145/2025113.2025156 – reference: Hidetake Uwano, Masahide Nakamura, Akito Monden, and Ken-Ichi Matsumoto. 2006. Analyzing Individual Performance of Source Code Review Using Reviewers’ Eye Movement. Proceedings of the 2006 symposium on Eye tracking research & applications - ETRA ’06, 133–140. isbn:1595933050 https://doi.org/10.1145/1117309 10.1145/1117309 – reference: Norman Peitek, Sven Apel, Chris Parnin, André Brechmann, and Janet Siegmund. 2021. Program Comprehension and Code Complexity Metrics: An fMRI Study. In Proceedings of the 43rd International Conference on Software Engineering (ICSE ’21). IEEE Press, 524–536. isbn:9781450390859 https://doi.org/10.1109/ICSE43902.2021.00056 10.1109/ICSE43902.2021.00056 – reference: Simone Grassini, Giulia Virginia Segurini, and Mika Koivisto. 2022. Watching Nature Videos Promotes Physiological Restoration: Evidence From the Modulation of Alpha Waves in Electroencephalography. Frontiers in Psychology, 13 (2022), 6, https://doi.org/10.3389/fpsyg.2022.871143 10.3389/fpsyg.2022.871143 – reference: Stephanie Yang, Amreen Amin Poonawala, Tian-Shun Allan Jiang, and Bertrand Schneider. 2023. Can Synchronous Code Editing and Awareness Tools Support Remote Tutoring? Effects on Learning and Teaching. Proc. ACM Hum.-Comput. Interact., 7, CSCW2 (2023), Article 328, oct, 30 pages. https://doi.org/10.1145/3610177 10.1145/3610177 – reference: Jacob Devlin, Ming Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1 (2019), 4171–4186. isbn:9781950737130 https://doi.org/10.18653/v1/N19-1423 10.18653/v1/N19-1423 – reference: Elliot Soloway, Beth Adelson, and Kate Ehrlich. 1988. Knowledge and processes in the comprehension of computer programs. In The nature of expertise.. Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, US. 129–152. isbn:0-89859-711-0 (Hardcover) – reference: Sarah Fakhoury, Devjeet Roy, Yuzhan Ma, Venera Arnaoudova, and Olusola Adesope. 2020. Measuring the impact of lexical and structural inconsistencies on developers’ cognitive load during bug localization. Empirical Software Engineering, 25, 3 (2020), May, 2140–2178. issn:1382-3256, 1573-7616 https://doi.org/10.1007/s10664-019-09751-4 10.1007/s10664-019-09751-4 – reference: Xinying Song, Alex Salcianu, Yang Song, Dave Dopson, and Denny Zhou. 2021. Fast WordPiece Tokenization. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic. 2089–2103. https://doi.org/10.18653/v1/2021.emnlp-main.160 10.18653/v1/2021.emnlp-main.160 – reference: Sebastian Nielebock, Dariusz Krolikowski, Jacob Krüger, Thomas Leich, and Frank Ortmeier. 2019. Commenting source code: is it worth it for small programming tasks? Empirical Software Engineering, 24, 3 (2019), June, 1418–1457. issn:1573-7616 https://doi.org/10.1007/s10664-018-9664-z 10.1007/s10664-018-9664-z – reference: Bonita Sharif, Michael Falcone, and Jonathan I Maletic. 2012. An Eye-tracking Study on the Role of Scan Time in Finding Source Code Defects. Proceedings of the Symposium on Eye Tracking Research and Applications, 381–384. isbn:9781450312219 https://doi.org/10.1145/2168556 10.1145/2168556 – reference: Roman Bednarik. 2012. Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations. International Journal of Human-Computer Studies, 70, 2 (2012), 143–155. issn:1071-5819 https://doi.org/10.1016/j.ijhcs.2011.09.003 10.1016/j.ijhcs.2011.09.003 – reference: Marvin Wyrich, Justus Bogner, and Stefan Wagner. 2023. 40 Years of Designing Code Comprehension Experiments: A Systematic Mapping Study. ACM Comput. Surv., 56, 4 (2023), Article 106, 42 pages. issn:0360-0300 https://doi.org/10.1145/3626522 10.1145/3626522 – reference: Radu Marinescu. 2004. Detection Strategies: Metrics-Based Rules for Detecting Design Flaws. In Proceedings of the 20th IEEE International Conference on Software Maintenance. 350–359. https://doi.org/10.1109/ICSM.2004.1357820 10.1109/ICSM.2004.1357820 – reference: N. Moha, Y. Guéhéneuc, L. Duchien, and A. Le Meur. 2010. DECOR: A Method for the Specification and Detection of Code and Design Smells. IEEE Transactions on Software Engineering, 36, 1 (2010), 20–36. https://doi.org/10.1109/TSE.2009.50 10.1109/TSE.2009.50 – reference: Alex Graves, Santiago Fernández, and Jürgen Schmidhuber. 2005. Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition. In Artificial Neural Networks: Formal Models and Their Applications – ICANN 2005, Wł odzisł aw Duch, Janusz Kacprzyk, Erkki Oja, and Sł awomir Zadrożny (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 799–804. isbn:978-3-540-28756-8 https://doi.org/10.1007/11550907_126 10.1007/11550907_126 – reference: Norman Peitek, Annabelle Bergum, Maurice Rekrut, Jonas Mucke, Matthias Nadig, Chris Parnin, Janet Siegmund, and Sven Apel. 2022. Correlates of Programmer Efficacy and Their Link to Experience: A Combined EEG and Eye-Tracking Study. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2022). 120–131. https://doi.org/10.1145/3540250.3549084 10.1145/3540250.3549084 – reference: Todd Sedano. 2016. Code Readability Testing, an Empirical Study. In 2016 IEEE 29th International Conference on Software Engineering Education and Training (CSEET). 111–117. https://doi.org/10.1109/CSEET.2016.36 ISSN: 2377-570X 10.1109/CSEET.2016.36 – reference: Ermira Daka, José Campos, Gordon Fraser, Jonathan Dorn, and Westley Weimer. 2015. Modeling readability to improve unit tests. In 2015 10th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC/FSE 2015 - Proceedings. Association for Computing Machinery, Inc, 107–118. isbn:9781450336758 https://doi.org/10.1145/2786805.2786838 10.1145/2786805.2786838 – reference: Essi Lahtinen, Kirsti Ala-Mutka, and Hannu-Matti Järvinen. 2005. A Study of the Difficulties of Novice Programmers. Proceedings of the 10th annual SIGCSE conference on Innovation and technology in computer science education - ITiCSE ’05, isbn:1595930248 https://doi.org/10.1145/1067445 10.1145/1067445 – reference: Kenneth Holmqvist, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost Van de Weijer. 2011. Eye tracking: A comprehensive guide to methods and measures. Oxford University Press, Oxford. – reference: Tanya Beelders and Jean Pierre Du Plessis. 2016. The influence of syntax highlighting on scanning and reading behaviour for source code. In ACM International Conference Proceeding Series. 26-28-September-2016, Association for Computing Machinery. isbn:9781450348058 https://doi.org/10.1145/2987491.2987536 10.1145/2987491.2987536 – reference: Ruven Brooks. 1983. Towards a theory of the comprehension of computer programs. International Journal of Man-Machine Studies, 18, 6 (1983), 6, 543–554. issn:0020-7373 https://doi.org/10.1016/S0020-7373(83)80031-5 10.1016/S0020-7373(83)80031-5 – reference: François Chollet. 2015. Keras. https://keras.io – reference: Naser Al Madi, Cole S. Peterson, Bonita Sharif, and Jonathan I. Maletic. 2021. From Novice to Expert: Analysis of Token Level Effects in a Longitudinal Eye Tracking Study. In 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC). 172–183. https://doi.org/10.1109/ICPC52881.2021.00025 10.1109/ICPC52881.2021.00025 – reference: Thomas Fritz, Andrew Begel, Sebastian C. Müller, Serap Yigit-Elliott, and Manuela Züger. 2014. Using Psycho-Physiological Measures to Assess Task Difficulty in Software Development. In Proceedings - International Conference on Software Engineering (ICSE 2014, 1). IEEE Computer Society, 402–413. isbn:9781450327565 issn:02705257 https://doi.org/10.1145/2568225.2568266 10.1145/2568225.2568266 – reference: Toyomi Ishida and Hidetake Uwano. 2019. Synchronized analysis of eye movement and EEG during program comprehension. In Proceedings - 2019 IEEE/ACM 6th International Workshop on Eye Movements in Programming, EMIP 2019. Institute of Electrical and Electronics Engineers Inc., 26–32. isbn:9781728122434 https://doi.org/10.1109/EMIP.2019.00012 10.1109/EMIP.2019.00012 – reference: Guido Salvaneschi, Sven Amann, Sebastian Proksch, and Mira Mezini. 2014. An empirical study on program comprehension with reactive programming. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2014). Association for Computing Machinery, New York, NY, USA. 564–575. isbn:978-1-4503-3056-5 https://doi.org/10.1145/2635868.2635895 10.1145/2635868.2635895 – reference: Nancy Pennington. 1987. Comprehension Strategies in Programming. In Empirical Studies of Programmers: Second Workshop. Ablex Publishing Corp., 100–113. isbn:0893914614 https://doi.org/10.5555/54968.54975 – reference: 2007. Eye movements: A window on mind and brain, Roger P. G. van Gompel, Martin H. Fischer, Wayne S. Murray, and Robin L. Hill (Eds.). Elsevier. – reference: Harold Abelson, Gerald Jay Sussman, and Julie Sussman. 1996. Structure and Interpretation of Computer Programs. The MIT Press, Cambridge. isbn:9780262510875 – reference: Shiwei Cheng, Jialing Wang, Xiaoquan Shen, Yijian Chen, and Anind Dey. 2021. Collaborative eye tracking based code review through real-time shared gaze visualization. Frontiers of Computer Science, 16, 3 (2021), Nov., issn:2095-2236 https://doi.org/10.1007/s11704-020-0422-1 10.1007/s11704-020-0422-1 – reference: Onyeka Ezenwoye. 2018. What language?-The choice of an introductory programming language. In 2018 IEEE Frontiers in Education Conference (FIE). 1–8. https://doi.org/10.1109/FIE.2018.8658592 10.1109/FIE.2018.8658592 – reference: Teresa Busjahn, Roman Bednarik, Andrew Begel, Martha Crosby, James H. Paterson, Carsten Schulte, Bonita Sharif, and Sascha Tamm. 2015. Eye Movements in Code Reading: Relaxing the Linear Order. In IEEE International Conference on Program Comprehension. 2015-August, IEEE Computer Society, 255–265. isbn:9781467381598 https://doi.org/10.1109/ICPC.2015.36 10.1109/ICPC.2015.36 – reference: Talita Vieira Ribeiro and Guilherme Travassos. 2017. Who is Right? Evaluating Empirical Contradictions in the Readability and Comprehensibility of Source Code. – reference: Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Comput., 9, 8 (1997), nov, 1735–1780. issn:0899-7667 https://doi.org/10.1162/neco.1997.9.8.1735 10.1162/neco.1997.9.8.1735 – reference: Sam. 2020. What determines whether a validation is "GOOD", "FAIR" or "POOR"? https://www.sr-research.com/support/thread-244.html – reference: G. Ann Campbell. 2017. Cognitive Complexity - A new way of measuring understandability. SonarSource SA, Switzerland. https://www.sonarsource.com/docs/CognitiveComplexity.pdf – reference: David Grimes, Desney S Tan, Scott E Hudson, Pradeep Shenoy, and Rajesh P N Rao. 2008. Feasibility and Pragmatics of Classifying Working Memory Load with an Electroencephalograph. Proceeding of the twenty-sixth annual CHI conference on Human factors in computing systems - CHI ’08, isbn:9781605580111 https://doi.org/10.1145/1357054 10.1145/1357054 – reference: Hiroto Harada and Minoru Nakayama. 2021. Estimation of Reading Ability of Program Codes Using Features of Eye Movements. In ACM Symposium on Eye Tracking Research and Applications (ETRA ’21 Short Papers). Association for Computing Machinery, New York, NY, USA. Article 32, 5 pages. isbn:9781450383455 https://doi.org/10.1145/3448018.3457421 10.1145/3448018.3457421 – reference: Kristoffer Gunnarsson and Olivia Herber. 2020. The Most Popular Programming Languages of GitHub’s Trending Repositories. – reference: SR Research. 2023. https://www.sr-research.com/eyelink-portable-duo/ Retrieved Feb 1, 2023 – reference: Delano Oliveira, Reydne Bruno, Fernanda Madeiral, and Fernando Castor. 2020. Evaluating Code Readability and Legibility: An Examination of Human-centric Studies. In IEEE International Conference on Software Maintenance and Evolution (ICSME). 348–359. https://doi.org/10.1109/ICSME46990.2020.00041 10.1109/ICSME46990.2020.00041 – reference: Anneliese Von Mayrhauser and A. Marie Vans. 1995. Program Comprehension During Software Maintenance and Evolution. Computer, 28, 8 (1995), 44–55. issn:00189162 https://doi.org/10.1109/2.402076 10.1109/2.402076 – reference: Barry W. Boehm. 1981. Software engineering economics. Prentice-Hall. – reference: Wade M Vagias. 2006. Likert-Type Scale Response Anchors Level of Agreement. Clemson International Institute for Tourism & Research Development, Department of Parks, Recreation and Tourism Management. Clemson University.. – reference: Salwa Aljehane, Bonita Sharif, and Jonathan Maletic. 2021. Determining Differences in Reading Behavior between Experts and Novices by Investigating Eye Movement on Source Code Constructs during a Bug Fixing Task. In Eye Tracking Research and Applications Symposium (ETRA). PartF169257, Association for Computing Machinery. isbn:9781450383455 https://doi.org/10.1145/3448018.3457424 10.1145/3448018.3457424 – reference: Michele Lacchia. 2022. Radon 4.1.0 documentation — Using radon programmatically. https://radon.readthedocs.io/en/latest/api.html – reference: Moshe Eizenman, P.E Hallett, and R.C. Frecker. 1985. Power spectra for ocular drift and tremor. Vision Research, 25, 11 (1985), 1635–1640. issn:0042-6989 https://doi.org/10.1016/0042-6989(85)90134-8 10.1016/0042-6989(85)90134-8 – reference: Mike Schuster and Kuldip Paliwal. 1997. Bidirectional recurrent neural networks. IEEE transactions on Signal Processing, 45, 11 (1997), 12, 2673–2681. https://doi.org/10.1109/78.650093 10.1109/78.650093 – reference: Meir M Lehman. 1980. On understanding laws, evolution, and conservation in the large-program life cycle. Journal of Systems and Software, 1 (1980), 213–221. https://doi.org/10.1016/0164-1212(79)90022-0 10.1016/0164-1212(79)90022-0 – reference: Eija Haapalainen, Seungjun Kim, Jodi F Forlizzi, and Anind K Dey. 2010. Psycho-Physiological Measures for Assessing Cognitive Load. Proceedings of the 12th ACM international conference on Ubiquitous computing, isbn:9781605588438 https://doi.org/10.1145/1864349 10.1145/1864349 – reference: Nachiappan Nagappan, Thomas Ball, and Andreas Zeller. 2006. Mining Metrics to Predict Component Failures. In Proceedings of the 28th international conference on Software engineering. 452–461. https://doi.org/10.1145/1134285.1134349 10.1145/1134285.1134349 – reference: TIOBE Software BV. 2022. TIOBE Index - TIOBE. https://www.tiobe.com/tiobe-index/ – reference: William H Dubay. 2004. The Principles of Readability. – reference: Shamsi T Iqbal, Sam Zheng, and Brian P Bailey. 2004. Task-Evoked Pupillary Response to Mental Workload in Human-Computer Interaction. Extended abstracts of the 2004 conference on Human factors and computing systems - CHI ’04, 1477–1480. isbn:1581137036 https://doi.org/10.1145/985921 10.1145/985921 – reference: Jeffrey Dean and Rajat Monga. 2015. TensorFlow: Large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/ – reference: Thomas J. Mccabe. 1976. A Complexity Measure. IEEE Transactions on Software Engineering, SE-2, 4 (1976), 308–320. issn:00985589 https://doi.org/10.1109/TSE.1976.233837 10.1109/TSE.1976.233837 – reference: Kilseop Ryu and Rohae Myung. 2005. Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. International Journal of Industrial Ergonomics, 35, 11 (2005), 11, 991–1009. issn:01698141 https://doi.org/10.1016/j.ergon.2005.04.005 10.1016/j.ergon.2005.04.005 – reference: SR Research Ltd. 2020. SR Research Experiment Builder. – reference: Steve McConnell. 2004. Code complete. Pearson. – reference: Sebastian C. Müller and Thomas Fritz. 2016. Using (Bio)Metrics to Predict Code Quality Online. In Proceedings of the 38th International Conference on Software Engineering (ICSE ’16). Association for Computing Machinery, New York, NY, USA. 452–463. isbn:9781450339001 https://doi.org/10.1145/2884781.2884803 10.1145/2884781.2884803 – reference: 2024. Supplementary Material. https://zenodo.org/doi/10.5281/zenodo.11123100 – reference: Dror G Feitelson. 2021. Considerations and pitfalls in controlled experiments on code comprehension. In 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC). 106–117. https://doi.org/10.1109/ICPC52881.2021.00019 10.1109/ICPC52881.2021.00019 – reference: Sarah D’Angelo and Andrew Begel. 2017. Improving Communication Between Pair Programmers Using Shared Gaze Awareness. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17). Association for Computing Machinery, New York, NY, USA. 6245–6290. isbn:9781450346559 https://doi.org/10.1145/3025453.3025573 10.1145/3025453.3025573 – reference: Seolhwa Lee, Danial Hooshyar, Hyesung Ji, Kichun Nam, and Heuiseok Lim. 2018. Mining biometric data to predict programmer expertise and task difficulty. Cluster Computing, https://doi.org/10.1007/s10586-017-0746-2 10.1007/s10586-017-0746-2 – reference: Rachel Turner, Michael Falcone, Bonita Sharif, and Alina Lazar. 2014. An Eye-tracking Study Assessing the Comprehension of C++ and Python Source Code. Eye Tracking Research and Applications Symposium (ETRA), 231–234. isbn:9781450327510 https://doi.org/10.1145/2578153.2578218 10.1145/2578153.2578218 – reference: Marcel Adam Just and Patricia A. Carpenter. 1980. A theory of reading: from eye fixations to comprehension.. Psychological review, 87 4 (1980), 329–54. https://doi.org/10.1037/0033-295X.87.4.329 10.1037/0033-295X.87.4.329 – ident: e_1_2_1_74_1 – ident: e_1_2_1_82_1 doi: 10.1145/3610177 – ident: e_1_2_1_45_1 doi: 10.1109/ICSM.2004.1357820 – ident: e_1_2_1_67_1 doi: 10.1109/CSEET.2016.36 – ident: e_1_2_1_37_1 doi: 10.1109/EMIP.2019.00012 – ident: e_1_2_1_75_1 doi: 10.1145/2578153.2578218 – ident: e_1_2_1_34_1 doi: 10.1162/neco.1997.9.8.1735 – ident: e_1_2_1_23_1 doi: 10.1109/ICPC52881.2021.00019 – ident: e_1_2_1_26_1 – volume-title: Software engineering economics ident: e_1_2_1_8_1 – ident: e_1_2_1_55_1 doi: 10.1109/ICSE43902.2021.00056 – ident: e_1_2_1_79_1 – ident: e_1_2_1_18_1 – ident: e_1_2_1_73_1 – ident: e_1_2_1_7_1 doi: 10.1016/j.visres.2019.05.003 – ident: e_1_2_1_32_1 doi: 10.1145/1864349 – ident: e_1_2_1_25_1 doi: 10.1145/2568225.2568266 – ident: e_1_2_1_69_1 doi: 10.1007/BF00977789 – ident: e_1_2_1_21_1 doi: 10.1007/s10664-019-09751-4 – ident: e_1_2_1_58_1 doi: 10.1145/3588015.3588410 – ident: e_1_2_1_36_1 doi: 10.1145/985921 – ident: e_1_2_1_20_1 doi: 10.1109/FIE.2018.8658592 – ident: e_1_2_1_53_1 doi: 10.1007/s10664-018-9664-z – ident: e_1_2_1_40_1 doi: 10.1145/1067445 – ident: e_1_2_1_54_1 doi: 10.1109/ICSME46990.2020.00041 – ident: e_1_2_1_72_1 – ident: e_1_2_1_29_1 doi: 10.1145/1357054 – ident: e_1_2_1_48_1 – ident: e_1_2_1_33_1 doi: 10.1145/3448018.3457421 – ident: e_1_2_1_61_1 doi: 10.1016/j.ergon.2005.04.005 – ident: e_1_2_1_19_1 doi: 10.1016/0042-6989(85)90134-8 – ident: e_1_2_1_52_1 doi: 10.1145/1134285.1134349 – ident: e_1_2_1_15_1 doi: 10.1145/3025453.3025573 – ident: e_1_2_1_56_1 doi: 10.1145/3540250.3549084 – ident: e_1_2_1_4_1 doi: 10.1145/3448018.3457424 – ident: e_1_2_1_27_1 doi: 10.3389/fpsyg.2022.871143 – ident: e_1_2_1_6_1 doi: 10.1145/2987491.2987536 – ident: e_1_2_1_13_1 – ident: e_1_2_1_51_1 doi: 10.1109/ICSE.2005.1553571 – ident: e_1_2_1_76_1 doi: 10.1145/1117309 – volume-title: Roger P ident: e_1_2_1_78_1 – ident: e_1_2_1_46_1 doi: 10.1109/2.402076 – ident: e_1_2_1_31_1 doi: 10.48550/arxiv.2009.08366 – ident: e_1_2_1_41_1 doi: 10.1007/s10586-017-0746-2 – ident: e_1_2_1_49_1 doi: 10.1109/TSE.2009.50 – ident: e_1_2_1_57_1 doi: 10.5555/54968.54975 – ident: e_1_2_1_16_1 – volume-title: Eye tracking: A comprehensive guide to methods and measures ident: e_1_2_1_35_1 – ident: e_1_2_1_17_1 doi: 10.18653/v1 – ident: e_1_2_1_14_1 doi: 10.1145/2786805.2786838 – volume-title: Gerald Jay Sussman, and Julie Sussman year: 1996 ident: e_1_2_1_2_1 – ident: e_1_2_1_42_1 doi: 10.1145/2025113.2025156 – ident: e_1_2_1_1_1 – ident: e_1_2_1_24_1 doi: 10.48550/arxiv.2002.08155 – volume: 52 volume-title: Proceedings of the 8th Australasian Conference on Computing Education - year: 2006 ident: e_1_2_1_80_1 – ident: e_1_2_1_28_1 doi: 10.1007/11550907_126 – ident: e_1_2_1_64_1 doi: 10.18260/1-2 – ident: e_1_2_1_68_1 doi: 10.1145/2168556 – ident: e_1_2_1_44_1 doi: 10.1109/TSE.1987.233475 – ident: e_1_2_1_66_1 doi: 10.1109/78.650093 – ident: e_1_2_1_9_1 doi: 10.1016/S0020-7373(83)80031-5 – ident: e_1_2_1_10_1 doi: 10.1109/ICPC.2015.36 – ident: e_1_2_1_71_1 doi: 10.18653/v1 – volume-title: Likert-Type Scale Response Anchors Level of Agreement ident: e_1_2_1_77_1 – ident: e_1_2_1_38_1 doi: 10.1037/0033-295X.87.4.329 – ident: e_1_2_1_81_1 doi: 10.1145/3626522 – ident: e_1_2_1_3_1 doi: 10.1109/ICPC52881.2021.00025 – ident: e_1_2_1_65_1 doi: 10.3758/s13414-011-0219-2 – ident: e_1_2_1_11_1 – ident: e_1_2_1_47_1 doi: 10.1109/TSE.1976.233837 – volume-title: The nature of expertise. ident: e_1_2_1_70_1 – ident: e_1_2_1_22_1 doi: 10.1109/ICPC.2012.6240511 – ident: e_1_2_1_30_1 – ident: e_1_2_1_43_1 doi: 10.1016/0164-1212(79)90022-0 – ident: e_1_2_1_39_1 – ident: e_1_2_1_59_1 doi: 10.1037/0033-2909.124.3.372 – ident: e_1_2_1_5_1 doi: 10.1016/j.ijhcs.2011.09.003 – ident: e_1_2_1_12_1 doi: 10.1007/s11704-020-0422-1 – ident: e_1_2_1_62_1 doi: 10.1145/2635868.2635895 – volume-title: https://www.sr-research.com/eyelink-portable-duo/ Retrieved year: 2023 ident: e_1_2_1_60_1 – ident: e_1_2_1_63_1 – ident: e_1_2_1_50_1 doi: 10.1145/2884781.2884803 – ident: e_1_2_1_83_1 doi: 10.1109/PRDC.2007.28 |
| SSID | ssj0002991170 |
| Score | 2.261544 |
| Snippet | The better the code quality and the less complex the code, the easier it is for software developers to comprehend and evolve it. Yet, how do we best detect... |
| SourceID | crossref acm |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 1982 |
| SubjectTerms | Computing methodologies Empirical software validation Human-centered computing Laboratory experiments Neural networks Software and its engineering |
| SubjectTermsDisplay | Computing methodologies -- Neural networks Human-centered computing -- Laboratory experiments Software and its engineering -- Empirical software validation |
| Title | Predicting Code Comprehension: A Novel Approach to Align Human Gaze with Code using Deep Neural Networks |
| URI | https://dl.acm.org/doi/10.1145/3660795 |
| Volume | 1 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2994-970X dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002991170 issn: 2994-970X databaseCode: M~E dateStart: 20240101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lb9QwELaWwoELjwKivOQD4lJFZGMnsStxiEpXCLWrFSzS3lZJbLcr0uwq-2hP_AZ-MjN2Xq16gAOXKEom3pXny3gy_maGkPcRl4HIAuGlEr50uDHGyxS8jybKAnCYDQtTW13_NB6PxWwmJ4PB7yYXZlfEZSmur-Xqv6oaroGyMXX2H9TdDgoX4ByUDkdQOxz_SvGTCvdeNm4_X2n7xlf6AonqjsaRHI6XO12gA9qmUyUF9ud0EX3k_7j4rH1-a6MJn7VeHWIlD1Dp2FHH133HdtIuhOuGdpAcn-FWxBoM_RXyy3RX-rCFWZH-vNQ2sjMFkTZr6JteuBZVPcr99tNXu6vPzx3KTnXZLimjamH70vb5TnUoI-AYIx0GPfDV9Hhr_gIsWixjf3bDVvcgOfp-0rO8Q-maGNWruEXEnSsEx2IaLIr82HX3vFVuu75zj9wP4lCiqT_71UXt4E9hqx6XfY1jfazl0bnJL3vOTc9LmT4hj-rPC5o4WDwlA13uk8dN6w5aW_Jn5KJDCUUt0xsoOaIJtRihDUboZkktRqjFCEWMUMSIe9pihCJGqMMIbTDynPwYnUyPv3h10w0vHYJt91KumWZc5r4RSodYYVCGSgjja_C24xjseRoNh0bmTOQ-U75SXHKWIU2ZqUyxF2SvXJb6JaGBn3HlGymUUdzkSkSaSy0YCzLw89PsgOzDnM1XrqzKvJ7JA_KhmcN5Xtepx3Ypxdzl0IedIG0FmzFuiby68xdek4cd-t6QvU211W_Jg3y3Wayrd1bhfwD5CX1r |
| linkProvider | ISSN International Centre |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Predicting+Code+Comprehension%3A+A+Novel+Approach+to+Align+Human+Gaze+with+Code+using+Deep+Neural+Networks&rft.jtitle=Proceedings+of+the+ACM+on+software+engineering&rft.au=Alakmeh%2C+Tarek&rft.au=Reich%2C+David&rft.au=J%C3%A4ger%2C+Lena&rft.au=Fritz%2C+Thomas&rft.date=2024-07-12&rft.pub=ACM&rft.eissn=2994-970X&rft.volume=1&rft.issue=FSE&rft.spage=1982&rft.epage=2004&rft_id=info:doi/10.1145%2F3660795&rft.externalDocID=3660795 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2994-970X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2994-970X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2994-970X&client=summon |