How to judge testing progress
It is usual to base the assessment of software testing progress on a coverage measure such as code coverage or specification coverage, or on the percentage of the input domain exercised. In this paper it is argued that these characteristics do not provide good indications of the degree to which the...
Uložené v:
| Vydané v: | Information and software technology Ročník 46; číslo 5; s. 323 - 328 |
|---|---|
| Hlavný autor: | |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
Amsterdam
Elsevier B.V
15.04.2004
Elsevier Science Ltd |
| Predmet: | |
| ISSN: | 0950-5849, 1873-6025 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | It is usual to base the assessment of software testing progress on a coverage measure such as code coverage or specification coverage, or on the percentage of the input domain exercised. In this paper it is argued that these characteristics do not provide good indications of the degree to which the software has been tested. Instead we propose that the assessment of testing progress be based on the total percentage of the probability mass that corresponds to the test cases selected and run. To do this, it is necessary to collect data that profiles how the software will be used once it is operational in the field. By so doing, we are able to accurately determine how much testing has been done, and whether it has met the standards of completeness for the product under consideration. |
|---|---|
| AbstractList | It is usual to base the assessment of software testing progress on a coverage measure such as code coverage or specification coverage, or on the percentage of the input domain exercised. In this paper it is argued that these characteristics do not provide good indications of the degree to which the software has been tested. Instead we propose that the assessment of testing progress be based on the total percentage of the probability mass that corresponds to the test cases selected and run. To do this, it is necessary to collect data that profiles how the software will be used once it is operational in the field. By so doing, we are able to accurately determine how much testing has been done, and whether it has met the standards of completeness for the product under consideration. [PUBLICATION ABSTRACT] It is usual to base the assessment of software testing progress on a coverage measure such as code coverage or specification coverage, or on the percentage of the input domain exercised. In this paper it is argued that these characteristics do not provide good indications of the degree to which the software has been tested. Instead we propose that the assessment of testing progress be based on the total percentage of the probability mass that corresponds to the test cases selected and run. To do this, it is necessary to collect data that profiles how the software will be used once it is operational in the field. By so doing, we are able to accurately determine how much testing has been done, and whether it has met the standards of completeness for the product under consideration. |
| Author | Weyuker, Elaine J. |
| Author_xml | – sequence: 1 givenname: Elaine J. surname: Weyuker fullname: Weyuker, Elaine J. email: weyuker@research.att.com organization: AT & T Labs—Research, Room No C213, 180 Park Avenue, Florharn Park, NJ 07932, USA |
| BookMark | eNqFkE1LAzEQhoNUsK3-A4XF-66TzSa78SBI8QsKXvQc9mNSstSkJqnivzdlPXnQ01zmed-ZZ0Fm1lkk5JxCQYGKq7EwVgenixKAFSALgOaIzGlTs1xAyWdkDpJDzptKnpBFCCMArYHBnFw8us8sumzcDxvMIoZo7CbbebfxGMIpOdbtNuDZz1yS1_u7l9Vjvn5-eFrdrvOeMRrzWsiKIwOhq04gR14KzoZGY1N2HVJeAYBkgpWDQFrrsm57rdt2YI2osKs7tiSXU24qft-nI9To9t6mSkWlqErKOKSl62mp9y4Ej1r1JrbROBt9a7aKgjrYUKOabKiDDQVSJRsJrn7BO2_eWv_1H3YzYZi-_zDoVegN2h4H47GPanDm74BvGQt77Q |
| CitedBy_id | crossref_primary_10_1108_13287261211255310 crossref_primary_10_1016_j_tcs_2015_02_002 |
| Cites_doi | 10.1145/227607.227608 10.1109/52.199724 10.1109/32.888628 10.1145/151299.151326 10.1145/584369.584373 10.1109/52.536461 10.1109/32.464549 10.1002/(SICI)1097-024X(199606)26:6<613::AID-SPE23>3.0.CO;2-5 |
| ContentType | Journal Article |
| Copyright | 2003 Elsevier B.V. Copyright Elsevier Science Ltd. Apr 15, 2004 |
| Copyright_xml | – notice: 2003 Elsevier B.V. – notice: Copyright Elsevier Science Ltd. Apr 15, 2004 |
| DBID | AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D |
| DOI | 10.1016/j.infsof.2003.09.008 |
| DatabaseName | CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Computer and Information Systems Abstracts |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Business |
| EISSN | 1873-6025 |
| EndPage | 328 |
| ExternalDocumentID | 666736561 10_1016_j_infsof_2003_09_008 S0950584903002015 |
| Genre | Feature |
| GroupedDBID | --K --M -~X .DC .~1 0R~ 1B1 1~. 1~5 29I 4.4 457 4G. 5GY 5VS 7-5 71M 77K 8P~ 9JN AABNK AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXUO AAYFN AAYOK ABBOA ABFNM ABFRF ABJNI ABMAC ABTAH ABXDB ABYKQ ACDAQ ACGFO ACGFS ACGOD ACNNM ACRLP ACZNC ADBBV ADEZE ADJOM ADMUD AEBSH AEFWE AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHZHX AIALX AIEXJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ASPBG AVWKF AXJTR AZFZN BKOJK BKOMP BLXMC CS3 DU5 EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 FDB FEDTE FGOYB FIRID FNPLU FYGXN G-Q G8K GBLVA GBOLZ HLZ HVGLF HZ~ IHE J1W KOM LG9 M41 MO0 MS~ N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. PQQKQ Q38 R2- RIG ROL RPZ SBC SDF SDG SDP SES SEW SPC SPCBC SSV SSZ T5K TWZ UHS UNMZH WH7 WUQ XFK ZY4 ~G- 77I 9DU AATTM AAXKI AAYWO AAYXX ABDPE ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD 7SC 8FD AFXIZ AGCQF AGRNS JQ2 L7M L~C L~D SSH |
| ID | FETCH-LOGICAL-c331t-76945e306f4b6e5e52653d8fe82bbe15400093632d6e17f27acffaad3864eb7b3 |
| ISICitedReferencesCount | 2 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000189229500008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0950-5849 |
| IngestDate | Fri Jul 25 07:05:47 EDT 2025 Sat Nov 29 03:40:31 EST 2025 Tue Nov 18 22:19:50 EST 2025 Fri Feb 23 02:23:55 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 5 |
| Keywords | Software testing Test data adequacy Test progress Operational distribution |
| Language | English |
| License | https://www.elsevier.com/tdm/userlicense/1.0 |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c331t-76945e306f4b6e5e52653d8fe82bbe15400093632d6e17f27acffaad3864eb7b3 |
| Notes | SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14 |
| PQID | 196421350 |
| PQPubID | 41979 |
| PageCount | 6 |
| ParticipantIDs | proquest_journals_196421350 crossref_citationtrail_10_1016_j_infsof_2003_09_008 crossref_primary_10_1016_j_infsof_2003_09_008 elsevier_sciencedirect_doi_10_1016_j_infsof_2003_09_008 |
| PublicationCentury | 2000 |
| PublicationDate | 2004-04-15 |
| PublicationDateYYYYMMDD | 2004-04-15 |
| PublicationDate_xml | – month: 04 year: 2004 text: 2004-04-15 day: 15 |
| PublicationDecade | 2000 |
| PublicationPlace | Amsterdam |
| PublicationPlace_xml | – name: Amsterdam |
| PublicationTitle | Information and software technology |
| PublicationYear | 2004 |
| Publisher | Elsevier B.V Elsevier Science Ltd |
| Publisher_xml | – name: Elsevier B.V – name: Elsevier Science Ltd |
| References | Weyuker, Vokolos (BIB9) 2000; 26 Weyuker (BIB6) 1996; 5 Avritzer, Weyuker (BIB2) 1995; September Avritzer, Kondek, Liu, Weyuker (BIB4) 2002 Whittaker, Poore (BIB10) 1993; 2 Avritzer, Ros, Weyuker (BIB1) 1996; September Musa (BIB5) 1993; 10 Weyuker, Vokolos (BIB8) 1998 Weyuker, Avritzer (BIB7) 2002; June Avritzer, Weyuker (BIB3) 1996; 26 Avritzer (10.1016/j.infsof.2003.09.008_BIB4) 2002 Weyuker (10.1016/j.infsof.2003.09.008_BIB9) 2000; 26 Avritzer (10.1016/j.infsof.2003.09.008_BIB1) 1996; September Avritzer (10.1016/j.infsof.2003.09.008_BIB2) 1995; September Avritzer (10.1016/j.infsof.2003.09.008_BIB3) 1996; 26 Whittaker (10.1016/j.infsof.2003.09.008_BIB10) 1993; 2 Musa (10.1016/j.infsof.2003.09.008_BIB5) 1993; 10 Weyuker (10.1016/j.infsof.2003.09.008_BIB6) 1996; 5 Weyuker (10.1016/j.infsof.2003.09.008_BIB7) 2002; June Weyuker (10.1016/j.infsof.2003.09.008_BIB8) 1998 |
| References_xml | – start-page: 80 year: 1998 end-page: 87 ident: BIB8 article-title: Performance testing of software systems publication-title: Proceedings of ACM Workshop on Software and Performance (WOSP-98) – volume: September start-page: 705 year: 1995 end-page: 716 ident: BIB2 article-title: The automatic generation of load test suites and the assessment of the resulting software publication-title: IEEE Transactions of Software Engineering – volume: June start-page: 152 year: 2002 end-page: 158 ident: BIB7 article-title: A metric to predict software scalability publication-title: Proceedings of IEEE/Eighth International Symposium on Software Metrics (Metrics 2002) Ottawa, Canada – volume: September start-page: 76 year: 1996 end-page: 82 ident: BIB1 article-title: Reliability testing of rule-based systems publication-title: IEEE Software – year: 2002 ident: BIB4 article-title: Software performance testing based on workload characterization publication-title: Proceedings of ACM/Third International Workshop on Software and Performance (WOSP 2002) Rome, Italy, July – volume: 10 start-page: 14 year: 1993 end-page: 32 ident: BIB5 article-title: Operational profiles in software reliability engineering publication-title: IEEE Software – volume: 26 start-page: 1147 year: 2000 end-page: 1156 ident: BIB9 article-title: Experience with performance testing of software systems-issues, an approach, and case study publication-title: IEEE Transactions on Software Engineering – volume: 5 start-page: 87 year: 1996 end-page: 98 ident: BIB6 article-title: Using failure cost information for testing and reliability assessment publication-title: ACM Transactions of Software Engineering and Methodology – volume: 2 start-page: 93 year: 1993 end-page: 106 ident: BIB10 article-title: Markov analysis of software specifications publication-title: ACM Transactions on Software Engineering and Methodology – volume: 26 year: 1996 ident: BIB3 article-title: Deriving workloads for performance testing publication-title: Software Practice and Experience – volume: 5 start-page: 87 issue: 2 year: 1996 ident: 10.1016/j.infsof.2003.09.008_BIB6 article-title: Using failure cost information for testing and reliability assessment publication-title: ACM Transactions of Software Engineering and Methodology doi: 10.1145/227607.227608 – volume: 10 start-page: 14 issue: 2 year: 1993 ident: 10.1016/j.infsof.2003.09.008_BIB5 article-title: Operational profiles in software reliability engineering publication-title: IEEE Software doi: 10.1109/52.199724 – volume: 26 start-page: 1147 issue: 12 year: 2000 ident: 10.1016/j.infsof.2003.09.008_BIB9 article-title: Experience with performance testing of software systems-issues, an approach, and case study publication-title: IEEE Transactions on Software Engineering doi: 10.1109/32.888628 – volume: 2 start-page: 93 issue: 1 year: 1993 ident: 10.1016/j.infsof.2003.09.008_BIB10 article-title: Markov analysis of software specifications publication-title: ACM Transactions on Software Engineering and Methodology doi: 10.1145/151299.151326 – year: 2002 ident: 10.1016/j.infsof.2003.09.008_BIB4 article-title: Software performance testing based on workload characterization publication-title: Proceedings of ACM/Third International Workshop on Software and Performance (WOSP 2002) Rome, Italy, July doi: 10.1145/584369.584373 – volume: September start-page: 76 year: 1996 ident: 10.1016/j.infsof.2003.09.008_BIB1 article-title: Reliability testing of rule-based systems publication-title: IEEE Software doi: 10.1109/52.536461 – volume: September start-page: 705 year: 1995 ident: 10.1016/j.infsof.2003.09.008_BIB2 article-title: The automatic generation of load test suites and the assessment of the resulting software publication-title: IEEE Transactions of Software Engineering doi: 10.1109/32.464549 – volume: 26 issue: 6 year: 1996 ident: 10.1016/j.infsof.2003.09.008_BIB3 article-title: Deriving workloads for performance testing publication-title: Software Practice and Experience doi: 10.1002/(SICI)1097-024X(199606)26:6<613::AID-SPE23>3.0.CO;2-5 – start-page: 80 year: 1998 ident: 10.1016/j.infsof.2003.09.008_BIB8 article-title: Performance testing of software systems publication-title: Proceedings of ACM Workshop on Software and Performance (WOSP-98) – volume: June start-page: 152 year: 2002 ident: 10.1016/j.infsof.2003.09.008_BIB7 article-title: A metric to predict software scalability publication-title: Proceedings of IEEE/Eighth International Symposium on Software Metrics (Metrics 2002) Ottawa, Canada |
| SSID | ssj0017030 |
| Score | 1.6980842 |
| Snippet | It is usual to base the assessment of software testing progress on a coverage measure such as code coverage or specification coverage, or on the percentage of... |
| SourceID | proquest crossref elsevier |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 323 |
| SubjectTerms | Computer programming Distributed processing Operational distribution Software Software testing Studies Test data adequacy Test progress Testing |
| Title | How to judge testing progress |
| URI | https://dx.doi.org/10.1016/j.infsof.2003.09.008 https://www.proquest.com/docview/196421350 |
| Volume | 46 |
| WOSCitedRecordID | wos000189229500008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1873-6025 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0017030 issn: 0950-5849 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1LT-MwELYQRSsuCJZdUQooh72hoMROHOeIUBHLAXEAqTcrdicrKpQimvL494xf4VEBy4FLFFmJ5fhzxjPj-WYI-cNSxgVNVUwzzdFAqXhcghZxbixmqlmhmLLFJoqzMzEalee-pN_MlhMomkY8PJQ33wo1tiHYhjr7Bbi7TrEB7xF0vCLseP0v4G2RuOn-ZG6IWK1JomH55tN_XbDFJESvd8zFfec-r9t7EwjWLrrb4XHuwy-G15VRTE8PXvkLbJiJY0x2jr8kRrWjfCkDvRvw6uUpsxVozLGB_d7IHJF7Qew6D8DE2Ao4Vptl1WaPTcTzNhOO1t_sPl1MYAg3m0jXiymQyWRSSsvl7tEiL1Fq9Q7_Dken3TmRkVcum6L7pkCOtBF8i6N5T_l4sw1b3eJinax5oyA6dGBukCVofpIfgZOwSXYR06idRhbTyGMaBUx_kcvj4cXRSezrWsSasbSNC15mOaCtVmeKQw6mQgEbixoEVQpQp7V-Js7omENa1LSodF1X1ZgJnoHCv-c3WW6mDWyRCITgSqs8q0FlqHmppAINWaYEBQq17hMWPlhqn_Td1B65lh9Nd5_E3Vs3LunJJ88XYS6lV9ycQiZxgXzy5iBMvfT_0EyaHHE0ZXmy_cVxDMjq84rfIcvt7Rx2yYq-a69mt3t-7TwBblZt8g |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=How+to+judge+testing+progress&rft.jtitle=Information+and+software+technology&rft.au=Weyuker%2C+Elaine+J.&rft.date=2004-04-15&rft.issn=0950-5849&rft.volume=46&rft.issue=5&rft.spage=323&rft.epage=328&rft_id=info:doi/10.1016%2Fj.infsof.2003.09.008&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_infsof_2003_09_008 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0950-5849&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0950-5849&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0950-5849&client=summon |