Effective REST APIs Testing with Error Message Analysis

REST APIs are essential for building modern enterprise systems, but effectively testing them remains challenging, particularly due to difficulties in inferring constraints from specifications. Current testing approaches typically use feedback from HTTP status codes to guide input generation. However...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings of the ACM on software engineering Ročník 2; číslo ISSTA; s. 1978 - 2000
Hlavní autoři: Xu, Lixin, Wu, Huayao, Pan, Zhenyu, Xu, Tongtong, Wang, Shaohua, Niu, Xintao, Nie, Changhai
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York, NY, USA ACM 22.06.2025
Témata:
Software and its engineering
Software testing and debugging
Feedback-Driven Testing
OpenAPI
REST APIs
ISSN:2994-970X, 2994-970X
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:REST APIs are essential for building modern enterprise systems, but effectively testing them remains challenging, particularly due to difficulties in inferring constraints from specifications. Current testing approaches typically use feedback from HTTP status codes to guide input generation. However, they overlook valuable information available in the accompanying error messages, reducing their effectiveness in exploring the APIs’ input spaces. In this paper, we propose EmRest, a black-box testing approach that leverages error message analysis to enhance both valid and exceptional test input generation for REST APIs. For each operation under test, EmRest first identifies all possible value assignment strategies for each of its input parameters. It then repeatedly applies combinatorial testing to sample test inputs based on these strategies, and statistically analyzes the error messages (of 400-range status code) received to infer and exclude invalid combinations of value assignment strategies (i.e., constraints of the input space). Additionally, EmRest seeks to mutate valid value assignment strategies that are finally identified to generate test inputs for exceptional testing. The error messages (of 500-range status code) received are categorized to identify bug-prone operations, for which more testing resources are allocated. Our experimental results on 16 real-world REST APIs demonstrates the effectiveness of EmRest. It achieves higher operation coverage than state-of-the-art approaches in 50% of APIs, and detects 226 unique bugs undetected by other approaches.
ISSN:2994-970X
2994-970X
DOI:10.1145/3728964