Naturalness of Attention: Revisiting Attention in Code Language Models

Language models for code such as CodeBERT offer the capability to learn advanced source code representation, but their opacity poses barriers to understanding of captured properties. Recent attention analysis studies provide initial interpretability insights by focusing solely on attention weights r...

Full description

Bibliographic Details

Published in:	IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Technologies Results (Online) pp. 107 - 111
Main Authors:	Saad, Mootez, Sharma, Tushar
Format:	Conference Proceeding
Language:	English
Published:	ACM 14.04.2024
Subjects:	Analytical models Attention Analysis Attention mechanisms Codes Explainable AI Focusing Interpretability Java Language Models of Code Norm Analysis Python Software engineering Source coding Syntactics Transformers
ISSN:	2832-7632
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!