PyAnalyzer: An Effective and Practical Approach for Dependency Extraction from Python Code

Dependency extraction based on static analysis lays the ground-work for a wide range of applications. However, dynamic language features in Python make code behaviors obscure and nondeter-ministic; consequently, it poses huge challenges for static analyses to resolve symbol-level dependencies. Altho...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings / International Conference on Software Engineering s. 1372 - 1383
Hlavní autoři: Jin, Wuxia, Xu, Shuo, Chen, Dawei, He, Jiajun, Zhong, Dinghong, Fan, Ming, Chen, Hongxu, Zhang, Huijia, Liu, Ting
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: ACM 14.04.2024
Témata:
Annotations
Codes
Dependency Extraction
Dynamic Features
Feature extraction
Libraries
Memory management
Python
Static analysis
Task analysis
ISSN:1558-1225
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Dependency extraction based on static analysis lays the ground-work for a wide range of applications. However, dynamic language features in Python make code behaviors obscure and nondeter-ministic; consequently, it poses huge challenges for static analyses to resolve symbol-level dependencies. Although prosperous techniques and tools are adequately available, they still lack sufficient capabilities to handle object changes, first-class citizens, varying call sites, and library dependencies. To address the fundamental difficulty for dynamic languages, this work proposes an effective and practical method namely PyAnalyzer for dependency extraction. PyAnalyzer uniformly models functions, classes, and modules into first-class heap objects, propagating the dynamic changes of these objects and class inheritance. This manner better simulates dynamic features like duck typing, object changes, and first-class citizens, resulting in high recall results without compromising pre-cision. Moreover, PyAnalyzer leverages optional type annotations as a shortcut to express varying call sites and resolve library depen-dencies on demand. We collected two micro-benchmarks (278 small programs), two macro-benchmarks (59 real-world applications), and 191 real-world projects (10MSLOC) for comprehensive comparisons with 7 advanced techniques (i.e., Understand, Sourcetrail, Depends, ENRE19, PySonar2, PyCG, and Type4Py). The results demonstrated that PyAnalyzer achieves a high recall and hence improves the F 1 by 24.7% on average, at least 1.4x faster without an obvious compromise of memory efficiency. Our work will benefit diverse client applications.
ISSN:1558-1225
DOI:10.1145/3597503.3640325