AutoRestTest: A Tool for Automated REST API Testing Using LLMs and MARL

As REST APIs have become widespread in modern web services, comprehensive testing of these APIs is increasingly crucial. Because of the vast search space of operations, parameters, and parameter values, along with their dependencies and constraints, current testing tools often achieve low code cover...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings (IEEE/ACM International Conference on Software Engineering Companion. Online) pp. 21 - 24
Main Authors: Stennett, Tyler, Kim, Myeongsoo, Sinha, Saurabh, Orso, Alessandro
Format: Conference Proceeding
Language:English
Published: IEEE 27.04.2025
Subjects:
Automated REST API Testing
Codes
Fault detection
Large language models
Monitoring
Multi Agent Reinforcement Learning for Testing
Reinforcement learning
Semantics
Servers
Software engineering
Testing
Web services
ISSN:2574-1934
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract As REST APIs have become widespread in modern web services, comprehensive testing of these APIs is increasingly crucial. Because of the vast search space of operations, parameters, and parameter values, along with their dependencies and constraints, current testing tools often achieve low code coverage, resulting in suboptimal fault detection. To address this limitation, we present AutoRestTest, a novel tool that integrates the Semantic Property Dependency Graph (SPDG) with Multi-Agent Reinforcement Learning (MARL) and large language models (LLMs) for effective REST API testing. AutoRestTest determines operation-dependent parameters using the SPDG and employs five specialized agents (operation, parameter, value, dependency, and header) to identify dependencies of operations and generate operation sequences, parameter combinations, and values. Through an intuitive command-line interface, users can easily configure and monitor tests with successful operation count, unique server errors detected, and time elapsed. Upon completion, AutoRestTest generates a detailed report highlighting errors detected and operations exercised. In this paper, we introduce our tool and present preliminary findings, with a demonstration video available at https://www.youtube.com/watch?v=VVus2W8rap8.
AbstractList As REST APIs have become widespread in modern web services, comprehensive testing of these APIs is increasingly crucial. Because of the vast search space of operations, parameters, and parameter values, along with their dependencies and constraints, current testing tools often achieve low code coverage, resulting in suboptimal fault detection. To address this limitation, we present AutoRestTest, a novel tool that integrates the Semantic Property Dependency Graph (SPDG) with Multi-Agent Reinforcement Learning (MARL) and large language models (LLMs) for effective REST API testing. AutoRestTest determines operation-dependent parameters using the SPDG and employs five specialized agents (operation, parameter, value, dependency, and header) to identify dependencies of operations and generate operation sequences, parameter combinations, and values. Through an intuitive command-line interface, users can easily configure and monitor tests with successful operation count, unique server errors detected, and time elapsed. Upon completion, AutoRestTest generates a detailed report highlighting errors detected and operations exercised. In this paper, we introduce our tool and present preliminary findings, with a demonstration video available at https://www.youtube.com/watch?v=VVus2W8rap8.
Author Kim, Myeongsoo
Orso, Alessandro
Sinha, Saurabh
Stennett, Tyler
Author_xml – sequence: 1
  givenname: Tyler
  surname: Stennett
  fullname: Stennett, Tyler
  email: tyler.stennett@gatech.edu
  organization: Georgia Institute of Technology,Atlanta,Georgia,USA
– sequence: 2
  givenname: Myeongsoo
  surname: Kim
  fullname: Kim, Myeongsoo
  email: mkim754@gatech.edu
  organization: Georgia Institute of Technology,Atlanta,Georgia,USA
– sequence: 3
  givenname: Saurabh
  surname: Sinha
  fullname: Sinha, Saurabh
  email: sinhas@us.ibm.com
  organization: IBM Research,New York,Yorktown Heights,USA
– sequence: 4
  givenname: Alessandro
  surname: Orso
  fullname: Orso, Alessandro
  email: orso@cc.gatech.edu
  organization: Georgia Institute of Technology,Atlanta,Georgia,USA
BookMark eNotUM1OhDAYrEYT15U38NCTN7Bf_2i9EYIrCRsNy543LRSDWegG8ODbC9HDzBxmMpnMPboZ_OAQegISARD9nKeHLEx9fzFD5wcpqaARJVREhBAQVyjQsVaMgWBSMbhGGypiHoJm_A4F0_S1xBgljOt4g3bJ9-xLN83Vghec4Mr7M279iFejN7NrcJkdKpx85HjNdMMnPk4rF8V-wmZo8D4piwd025rz5IJ_3aLja1alb2HxvsvTpAgNVWQOa0tbcMI2sRQtKGdbIutGaQa8XfYq5qyNuTOyFgq0hAVOxg23IIFLatkWPf71ds6502XsejP-nJZbKGdcsV97MlAl
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/ICSE-Companion66252.2025.00015
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9798331536831
EISSN 2574-1934
EndPage 24
ExternalDocumentID 11024348
Genre orig-research
GrantInformation_xml – fundername: NSF
  grantid: CCF-0725202
  funderid: 10.13039/100000001
GroupedDBID 6IE
6IF
6IL
6IN
AAJGR
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
OCL
RIE
RIL
ID FETCH-LOGICAL-a280t-cb2f1e5bd765f18ebf06cd89314f15383ebb74ea6c581961196e67d4b161462b3
IEDL.DBID RIE
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001554070400005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Jun 18 06:01:38 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a280t-cb2f1e5bd765f18ebf06cd89314f15383ebb74ea6c581961196e67d4b161462b3
PageCount 4
ParticipantIDs ieee_primary_11024348
PublicationCentury 2000
PublicationDate 2025-April-27
PublicationDateYYYYMMDD 2025-04-27
PublicationDate_xml – month: 04
  year: 2025
  text: 2025-April-27
  day: 27
PublicationDecade 2020
PublicationTitle Proceedings (IEEE/ACM International Conference on Software Engineering Companion. Online)
PublicationTitleAbbrev ICSE-COMPANION
PublicationYear 2025
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0003203497
Score 2.2894056
Snippet As REST APIs have become widespread in modern web services, comprehensive testing of these APIs is increasingly crucial. Because of the vast search space of...
SourceID ieee
SourceType Publisher
StartPage 21
SubjectTerms Automated REST API Testing
Codes
Fault detection
Large language models
Monitoring
Multi Agent Reinforcement Learning for Testing
Reinforcement learning
Semantics
Servers
Software engineering
Testing
Web services
Title AutoRestTest: A Tool for Automated REST API Testing Using LLMs and MARL
URI https://ieeexplore.ieee.org/document/11024348
WOSCitedRecordID wos001554070400005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA5uiHhSceJvchBvdWmaJp23MjYdbGPMCruNpnkBQVrZWv9-X7I5vXjwEAhtAuWF5vt4yfc-Qu6YtdoaLgMhchEI4xJNiEpBosM4ByUQwZk3m1DTabJY9GZbsbrXwgCAv3wGD67rz_JNVTQuVdZFqOIiEkmLtJSSG7HWLqEScVdqRR2Q-20dze6o_zIINr-V88NBpu-UV9wlUZizwf1lp-LRZHj0z-84Jp0fXR6d7RDnhOxBeUqe0qau5ri5Z9geaUqzqnqnyEWpe4GEFAzFMGc0nY2oG4NTqb8pQMfjyZrmpaGTdD7ukNfhIOs_B1t7hCDnCauDQnMbQqyNkrENE9CWycIg_wiFdftYBForAbksYoR9GWIDqYzQSPKE5Do6I-2yKuGc0NxYbm0iQiNCoeKixwBHsNgUyMCYlRek46Kw_NhUwFh-B-Dyj-dX5NAF2p26cHVN2vWqgRuyX3zWb-vVrV-3LwAZljA
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFH7oFPWk4sTf5iDe6tI0TTpvZWxu2I0xK-w2miYBQVrZOv9-X7o5vXjwEAhtUsorzffxku99AHfUWmU1Ex7nGfe4dokmRCUvUn6YGckRwWltNiFHo2g6bY_XYvVaC2OMqQ-fmQfXrffydZkvXaqshVDFeMCjbdgJ8Sl0JdfapFQC5oqtyD24X1fSbA06L11v9WM5Rxzk-k57xVwahToj3F-GKjWe9A7_-SZH0PxR5pHxBnOOYcsUJ_AUL6tygst7iu2RxCQty3eCbJS4G0hJjSYY6JTE4wFxY3Aqqc8KkCQZLkhWaDKMJ0kTXnvdtNP31gYJXsYiWnm5YtY3odJShNaPjLJU5BoZiM-tW8kCo5TkJhN5iMAvfGxGSM0V0jwumApOoVGUhTkDkmnLrI24r7nPZZi3qcERNNQ5cjBqxTk0XRRmH6saGLPvAFz8cf0W9vvpMJklg9HzJRy4oLs9GCavoFHNl-YadvPP6m0xv6m_4RcECZl3
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28IEEE%2FACM+International+Conference+on+Software+Engineering+Companion.+Online%29&rft.atitle=AutoRestTest%3A+A+Tool+for+Automated+REST+API+Testing+Using+LLMs+and+MARL&rft.au=Stennett%2C+Tyler&rft.au=Kim%2C+Myeongsoo&rft.au=Sinha%2C+Saurabh&rft.au=Orso%2C+Alessandro&rft.date=2025-04-27&rft.pub=IEEE&rft.eissn=2574-1934&rft.spage=21&rft.epage=24&rft_id=info:doi/10.1109%2FICSE-Companion66252.2025.00015&rft.externalDocID=11024348