Test Oracle Strategies for Model-Based Testing

Testers use model-based testing to design abstract tests from models of the system's behavior. Testers instantiate the abstract tests into concrete tests with test input values and test oracles that check the results. Given the same test inputs, more elaborate test oracles have the potential to...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on software engineering Ročník 43; číslo 4; s. 372 - 395
Hlavní autoři: Nan Li, Offutt, Jeff
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.04.2017
IEEE Computer Society
Témata:
Concrete
Context
Failure
Model testing
model-based testing
Observability
RIPR model
Software
State (computer science)
State machines
subsumption
System testing
test automation
Test oracle
test oracle strategy
Unified modeling language
ISSN:0098-5589, 1939-3520
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Testers use model-based testing to design abstract tests from models of the system's behavior. Testers instantiate the abstract tests into concrete tests with test input values and test oracles that check the results. Given the same test inputs, more elaborate test oracles have the potential to reveal more failures, but may also be more costly. This research investigates the ability for test oracles to reveal failures. We define ten new test oracle strategies that vary in amount and frequency of program state checked. We empirically compared them with two baseline test oracle strategies. The paper presents several main findings. (1) Test oracles must check more than runtime exceptions because checking exceptions alone is not effective at revealing failures. (2) Test oracles do not need to check the entire output state because checking partial states reveals nearly as many failures as checking entire states. (3) Test oracles do not need to check program states multiple times because checking states less frequently is as effective as checking states more frequently. In general, when state machine diagrams are used to generate tests, checking state invariants is a reasonably effective low cost approach to creating test oracles.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0098-5589
1939-3520
DOI:10.1109/TSE.2016.2597136