PyAnalyzer: An Effective and Practical Approach for Dependency Extraction from Python Code

Dependency extraction based on static analysis lays the ground-work for a wide range of applications. However, dynamic language features in Python make code behaviors obscure and nondeter-ministic; consequently, it poses huge challenges for static analyses to resolve symbol-level dependencies. Altho...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Proceedings / International Conference on Software Engineering s. 1372 - 1383
Hlavní autori: Jin, Wuxia, Xu, Shuo, Chen, Dawei, He, Jiajun, Zhong, Dinghong, Fan, Ming, Chen, Hongxu, Zhang, Huijia, Liu, Ting
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: ACM 14.04.2024
Predmet:
Annotations
Codes
Dependency Extraction
Dynamic Features
Feature extraction
Libraries
Memory management
Python
Static analysis
Task analysis
ISSN:1558-1225
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Dependency extraction based on static analysis lays the ground-work for a wide range of applications. However, dynamic language features in Python make code behaviors obscure and nondeter-ministic; consequently, it poses huge challenges for static analyses to resolve symbol-level dependencies. Although prosperous techniques and tools are adequately available, they still lack sufficient capabilities to handle object changes, first-class citizens, varying call sites, and library dependencies. To address the fundamental difficulty for dynamic languages, this work proposes an effective and practical method namely PyAnalyzer for dependency extraction. PyAnalyzer uniformly models functions, classes, and modules into first-class heap objects, propagating the dynamic changes of these objects and class inheritance. This manner better simulates dynamic features like duck typing, object changes, and first-class citizens, resulting in high recall results without compromising pre-cision. Moreover, PyAnalyzer leverages optional type annotations as a shortcut to express varying call sites and resolve library depen-dencies on demand. We collected two micro-benchmarks (278 small programs), two macro-benchmarks (59 real-world applications), and 191 real-world projects (10MSLOC) for comprehensive comparisons with 7 advanced techniques (i.e., Understand, Sourcetrail, Depends, ENRE19, PySonar2, PyCG, and Type4Py). The results demonstrated that PyAnalyzer achieves a high recall and hence improves the F 1 by 24.7% on average, at least 1.4x faster without an obvious compromise of memory efficiency. Our work will benefit diverse client applications.
ISSN:1558-1225
DOI:10.1145/3597503.3640325