PyNose: A Test Smell Detector For Python

Similarly to production code, code smells also occur in test code, where they are called test smells. Test smells have a detrimental effect not only on test code but also on the production code that is being tested. To date, the majority of the research on test smells has been focusing on programmin...

Full description

Saved in:
Bibliographic Details
Published in:IEEE/ACM International Conference on Automated Software Engineering : [proceedings] pp. 593 - 605
Main Authors: Wang, Tongjie, Golubev, Yaroslav, Smirnov, Oleg, Li, Jiawei, Bryksin, Timofey, Ahmed, Iftekhar
Format: Conference Proceeding
Language:English
Published: IEEE 01.11.2021
Subjects:
code change patterns
code smells
Codes
Detectors
empirical studies
Focusing
Java
mining software repositories
Production
Python
Software
Test smells
ISSN:2643-1572
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Similarly to production code, code smells also occur in test code, where they are called test smells. Test smells have a detrimental effect not only on test code but also on the production code that is being tested. To date, the majority of the research on test smells has been focusing on programming languages such as Java and Scala. However, there are no available automated tools to support the identification of test smells for Python, despite its rapid growth in popularity in recent years. In this paper, we strive to extend the research to Python, build a tool for detecting test smells in this language, and conduct an empirical analysis of test smells in Python projects.We started by gathering a list of test smells from existing research and selecting test smells that can be considered language-agnostic or have similar functionality in Python's standard Unittest framework. In total, we identified 17 diverse test smells. Additionally, we searched for Python-specific test smells by mining frequent code change patterns that can be considered as either fixing or introducing test smells. Based on these changes, we proposed our own test smell called Suboptimal Assert. To detect all these test smells, we developed a tool called PYNOSE in the form of a plugin to PyCharm, a popular Python IDE. Finally, we conducted a large-scale empirical investigation aimed at analyzing the prevalence of test smells in Python code. Our results show that 98% of the projects and 84% of the test suites in the studied dataset contain at least one test smell. Our proposed Suboptimal Assert smell was detected in as much as 70.6% of the projects, making it a valuable addition to the list.
AbstractList Similarly to production code, code smells also occur in test code, where they are called test smells. Test smells have a detrimental effect not only on test code but also on the production code that is being tested. To date, the majority of the research on test smells has been focusing on programming languages such as Java and Scala. However, there are no available automated tools to support the identification of test smells for Python, despite its rapid growth in popularity in recent years. In this paper, we strive to extend the research to Python, build a tool for detecting test smells in this language, and conduct an empirical analysis of test smells in Python projects.We started by gathering a list of test smells from existing research and selecting test smells that can be considered language-agnostic or have similar functionality in Python's standard Unittest framework. In total, we identified 17 diverse test smells. Additionally, we searched for Python-specific test smells by mining frequent code change patterns that can be considered as either fixing or introducing test smells. Based on these changes, we proposed our own test smell called Suboptimal Assert. To detect all these test smells, we developed a tool called PYNOSE in the form of a plugin to PyCharm, a popular Python IDE. Finally, we conducted a large-scale empirical investigation aimed at analyzing the prevalence of test smells in Python code. Our results show that 98% of the projects and 84% of the test suites in the studied dataset contain at least one test smell. Our proposed Suboptimal Assert smell was detected in as much as 70.6% of the projects, making it a valuable addition to the list.
Author Smirnov, Oleg
Li, Jiawei
Ahmed, Iftekhar
Golubev, Yaroslav
Wang, Tongjie
Bryksin, Timofey
Author_xml – sequence: 1
  givenname: Tongjie
  surname: Wang
  fullname: Wang, Tongjie
  email: tongjiew@uci.edu
  organization: University of California, Irvine,Irvine,CA,United States
– sequence: 2
  givenname: Yaroslav
  surname: Golubev
  fullname: Golubev, Yaroslav
  email: yaroslav.golubev@jetbrains.com
  organization: University of California, Irvine,Irvine,CA,United States
– sequence: 3
  givenname: Oleg
  surname: Smirnov
  fullname: Smirnov, Oleg
  email: oleg.smirnov@jetbrains.com
  organization: University of California, Irvine,Irvine,CA,United States
– sequence: 4
  givenname: Jiawei
  surname: Li
  fullname: Li, Jiawei
  email: jiawl28@uci.edu
  organization: University of California, Irvine,Irvine,CA,United States
– sequence: 5
  givenname: Timofey
  surname: Bryksin
  fullname: Bryksin, Timofey
  email: timofey.bryksin@jetbrains.com
  organization: University of California, Irvine,Irvine,CA,United States
– sequence: 6
  givenname: Iftekhar
  surname: Ahmed
  fullname: Ahmed, Iftekhar
  email: iftekha@uci.edu
  organization: University of California, Irvine,Irvine,CA,United States
BookMark eNotj81Kw0AURkdRsK19AhFm6SZx7p1_d6G2KhQttK7LJHODgTSRTDZ5ewt28XHO6sA3Zzdd3xFjjyByAOGfi_1ag0aVo0DIvbHOgL5iczBGKyGllddshkbJDLTFO7ZMqSmFck4p682MPe2mzz7RCy_4gdLI9ydqW_5KI1VjP_DNebtp_Om7e3ZbhzbR8sIF-96sD6v3bPv19rEqtllAZ8fMBllBVBhlDMZKDCVpKjUGqMmCLqH2vqqid1GQt8EIZ85GOgYNiBDkgj38dxsiOv4OzSkM0_FyTP4BMh1C2Q
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/ASE51524.2021.9678615
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEL
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEL
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 1665403373
9781665403375
EISSN 2643-1572
EndPage 605
ExternalDocumentID 9678615
Genre orig-research
GroupedDBID 29I
6IE
6IF
6IH
6IK
6IL
6IM
6IN
6J9
AAJGR
AAWTH
ABLEC
ACREN
ADYOE
ADZIZ
AFYQB
ALMA_UNASSIGNED_HOLDINGS
AMTXH
APO
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IPLJI
M43
OCL
RIE
RIL
ID FETCH-LOGICAL-a287t-7a3c1d42d3da6732abe5eb52a1fe715b1f99ccd98d0e97a6086d0ee5da51221a3
IEDL.DBID RIE
ISICitedReferencesCount 22
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000779309000051&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 03:02:55 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a287t-7a3c1d42d3da6732abe5eb52a1fe715b1f99ccd98d0e97a6086d0ee5da51221a3
PageCount 13
ParticipantIDs ieee_primary_9678615
PublicationCentury 2000
PublicationDate 2021-Nov.
PublicationDateYYYYMMDD 2021-11-01
PublicationDate_xml – month: 11
  year: 2021
  text: 2021-Nov.
PublicationDecade 2020
PublicationTitle IEEE/ACM International Conference on Automated Software Engineering : [proceedings]
PublicationTitleAbbrev ASE
PublicationYear 2021
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssib048844796
ssj0051577
Score 2.3632526
Snippet Similarly to production code, code smells also occur in test code, where they are called test smells. Test smells have a detrimental effect not only on test...
SourceID ieee
SourceType Publisher
StartPage 593
SubjectTerms code change patterns
code smells
Codes
Detectors
empirical studies
Focusing
Java
mining software repositories
Production
Python
Software
Test smells
Title PyNose: A Test Smell Detector For Python
URI https://ieeexplore.ieee.org/document/9678615
WOSCitedRecordID wos000779309000051&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA61ePDkoxXf5ODBg2mbbHan8Va0xYOUhVbprWSTCQi1lXYr9N-b7G4rghcPgZBDyOQxX2aSb4aQWwdOuq52LNZJxqTUmmX-Xs6sh2qINNe24MK8vcBw2J1MVFoj9zsuDCIWn8-wFarFW75dmHVwlbWV16xJYJTvASQlV2u7d_w-lLIIHVdqYQ_TABVjh3dUuzfq-yYRvCiCt6qOfmVUKQBlcPi_oRyR5g8zj6Y7zDkmNZyfkMNtagZandQGuUs3w8UKH2iPjr3ep6MPnM3oE-aFk54OfEk3IWxAk7wO-uPHZ1YlRWDaGzc5Ax0ZbqWwkdUJREJnGGMWC80dAo8z7pQyxqqu7aACnXiTxdcwttpDu-A6OiX1-WKOZ4QqG2UGHEYQ4jVwkRnJBTgNcZxAIs05aQTBp59l3ItpJfPF382X5CDMbcnTuyL1fLnGa7JvvvL31fKmWKxvTuCS1A
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwIxEG4ImugJFYxve_DgwQXabbfUG1EIRtyQgIYb6bbTxATBwGLCv7fdXTAmXjw0aXpoOn3M15n2m0HoxgrLbEvZgKsoCRhTKkjcvTwwDqpFqIgyGRfmrS_iuDUey0EJ3W25MACQfT6Duq9mb_lmrlfeVdaQTrNGnlG-wxmjzZyttdk9bicylgWPy_WwA2ohCs4OacpGe9hxTdT7USipF139yqmSQUq38r_BHKDaDzcPD7aoc4hKMDtClU1yBlyc1Sq6Hazj-RLucRuPnObHww-YTvEjpJmbHnddGax94IAaeu12Rg-9oEiLEChn3qSBUKEmhlETGhWJkKoEOCScKmJBEJ4QK6XWRrZME6RQkTNaXA24UQ7cKVHhMSrP5jM4QViaMNHCQih8xAZCE80IFVYJziMRMX2Kql7wyWce-WJSyHz2d_M12uuNXvqT_lP8fI72_TznrL0LVE4XK7hEu_orfV8urrKF-wYoZ5Yb
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=IEEE%2FACM+International+Conference+on+Automated+Software+Engineering+%3A+%5Bproceedings%5D&rft.atitle=PyNose%3A+A+Test+Smell+Detector+For+Python&rft.au=Wang%2C+Tongjie&rft.au=Golubev%2C+Yaroslav&rft.au=Smirnov%2C+Oleg&rft.au=Li%2C+Jiawei&rft.date=2021-11-01&rft.pub=IEEE&rft.eissn=2643-1572&rft.spage=593&rft.epage=605&rft_id=info:doi/10.1109%2FASE51524.2021.9678615&rft.externalDocID=9678615