Beyond Syntax: How Do LLMs Understand Code?

Within software engineering research, Large Language Models (LLMs) are often treated as 'black boxes', with only their inputs and outputs being considered. In this paper, we take a machine interpretability approach to examine how LLMs internally represent and process code.We focus on varia...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Technologies Results (Online) S. 86 - 90
Hauptverfasser: North, Marc, Atapour-Abarghouei, Amir, Bencomo, Nelly
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 27.04.2025
Schlagworte:
Codes
Computer languages
Encoding
Large language models
Large Language Models (LLMs)
Mechanistic interpretability
Semantics
Software engineering
Streams
Syntactics
Technological innovation
Training
ISSN:2832-7632
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Within software engineering research, Large Language Models (LLMs) are often treated as 'black boxes', with only their inputs and outputs being considered. In this paper, we take a machine interpretability approach to examine how LLMs internally represent and process code.We focus on variable declaration and function scope, training classifier probes on the residual streams of LLMs as they process code written in different programming languages to explore how LLMs internally represent these concepts across different programming languages. We also look for specific attention heads that support these representations and examine how they behave for inputs of different languages.Our results show that LLMs have an understanding - and internal representation - of language-independent coding semantics that goes beyond the syntax of any specific programming language, using the same internal components to process code, regardless of the programming language that the code is written in. Furthermore, we find evidence that these language-independent semantic components exist in the middle layers of LLMs and are supported by language-specific components in the earlier layers that parse the syntax of specific languages and feed into these later semantic components.Finally, we discuss the broader implications of our work, particularly in relation to concerns that AI, with its reliance on large datasets to learn new programming languages, might limit innovation in programming language design. By demonstrating that LLMs have a language-independent representation of code, we argue that LLMs may be able to flexibly learn the syntax of new programming languages while retaining their semantic understanding of universal coding concepts. In doing so, LLMs could promote creativity in future programming language design, providing tools that augment rather than constrain the future of software engineering.
ISSN:2832-7632
DOI:10.1109/ICSE-NIER66352.2025.00023