TISSEA: A Framework for Testing IoT Systems Based on Technical Software Engineering Aspects

Internet of Things (IoT) systems refer to interconnected systems of devices that collect, process, and exchange data. As IoT adoption continues to grow, ensuring effective testing is of paramount importance. However, testing IoT systems remains a challenge, particularly for software engineers, due t...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE internet of things journal Ročník 12; číslo 22; s. 48787 - 48802
Hlavní autoři: Minani, Jean Baptiste, Sabir, Fatima, Moha, Naouel, Gueheneuc, Yann-Gael, Masuda, Tomoaki
Médium: Journal Article
Jazyk:angličtina
Vydáno: Piscataway IEEE 15.11.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Case studies
Cloud computing
Computer architecture
Data exchange
Data integrity
Data integrity testing
Engineers
Event handling
event handling testing
event logging testing
Focusing
Integrity
Internet of Things
Internet of Things (IoT) system testing
Security
Software engineering
software engineering (SE)
System testing
Taxonomy
Testing
testing framework
ISSN:2327-4662, 2327-4662
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract Internet of Things (IoT) systems refer to interconnected systems of devices that collect, process, and exchange data. As IoT adoption continues to grow, ensuring effective testing is of paramount importance. However, testing IoT systems remains a challenge, particularly for software engineers, due to the need to test aspects beyond their primary area of expertise (e.g., security, sensor calibration, and connectivity). Testing aspects refer to any concept or concern that should be considered when testing a given system. While several frameworks for testing exist that focus on generic aspects of IoT systems, there is no dedicated framework for testing technical software engineering (SE) aspects of IoT systems. To address this gap, we propose and evaluate TISSEA, a framework to guide software engineers to test the technical SE aspects of IoT systems. We constructed TISSEA by identifying all possible technical SE aspects from published taxonomies for IoT systems testing. Further, we mapped each aspect to the granularity of testing at each layer of the IoT system. We finally mapped each aspect with test orchestration strategies, test input artifacts, and execution strategies. We evaluated the TISSEA by surveying 22 professionals and conducting two case studies: 1) event logging and handling testing and 2) data integrity testing. The survey results show that professionals agreed with the proposed technical SE aspects for testing the device and application layers. However, the aspects proposed for testing the gateway and cloud layers still require further investigation. Results of the case studies indicate a gap between expected and captured log events. Regarding event handling, we found that some of the events reported by the system as successfully handled may include unhandled events that cannot be identified when relying on a single orchestration strategy. Regarding data integrity testing, we found that data can be altered at any node at any layer of the IoT system. However, accessing the original data allows the detection of modifications made to it at each node. Overall evaluation of TISSEA shows strong agreement with practitioners, and it could be useful to test technical SE aspects of IoT systems.
AbstractList Internet of Things (IoT) systems refer to interconnected systems of devices that collect, process, and exchange data. As IoT adoption continues to grow, ensuring effective testing is of paramount importance. However, testing IoT systems remains a challenge, particularly for software engineers, due to the need to test aspects beyond their primary area of expertise (e.g., security, sensor calibration, and connectivity). Testing aspects refer to any concept or concern that should be considered when testing a given system. While several frameworks for testing exist that focus on generic aspects of IoT systems, there is no dedicated framework for testing technical software engineering (SE) aspects of IoT systems. To address this gap, we propose and evaluate TISSEA, a framework to guide software engineers to test the technical SE aspects of IoT systems. We constructed TISSEA by identifying all possible technical SE aspects from published taxonomies for IoT systems testing. Further, we mapped each aspect to the granularity of testing at each layer of the IoT system. We finally mapped each aspect with test orchestration strategies, test input artifacts, and execution strategies. We evaluated the TISSEA by surveying 22 professionals and conducting two case studies: 1) event logging and handling testing and 2) data integrity testing. The survey results show that professionals agreed with the proposed technical SE aspects for testing the device and application layers. However, the aspects proposed for testing the gateway and cloud layers still require further investigation. Results of the case studies indicate a gap between expected and captured log events. Regarding event handling, we found that some of the events reported by the system as successfully handled may include unhandled events that cannot be identified when relying on a single orchestration strategy. Regarding data integrity testing, we found that data can be altered at any node at any layer of the IoT system. However, accessing the original data allows the detection of modifications made to it at each node. Overall evaluation of TISSEA shows strong agreement with practitioners, and it could be useful to test technical SE aspects of IoT systems.
Author Gueheneuc, Yann-Gael
Moha, Naouel
Masuda, Tomoaki
Sabir, Fatima
Minani, Jean Baptiste
Author_xml – sequence: 1
  givenname: Jean Baptiste
  orcidid: 0000-0002-9164-6645
  surname: Minani
  fullname: Minani, Jean Baptiste
  email: jeanbaptiste.minani@concordia.ca
  organization: Department of Computer Science and Software Engineering, Concordia University, Montreal, QC, Canada
– sequence: 2
  givenname: Fatima
  orcidid: 0000-0001-8707-6603
  surname: Sabir
  fullname: Sabir, Fatima
  email: fatima.sabir@pucit.edu.pk
  organization: Department of Computer Sciences, University of the Punjab, Lahore, Pakistan
– sequence: 3
  givenname: Naouel
  orcidid: 0000-0001-9252-9937
  surname: Moha
  fullname: Moha, Naouel
  email: Naouel.Moha@etsmtl.ca
  organization: Département de génie logiciel et des TI, École de technologie supérieure, Montreal, QC, Canada
– sequence: 4
  givenname: Yann-Gael
  orcidid: 0000-0002-4361-2563
  surname: Gueheneuc
  fullname: Gueheneuc, Yann-Gael
  email: yanngael.gueheneuc@concordia.ca
  organization: Department of Computer Science and Software Engineering, Concordia University, Montreal, QC, Canada
– sequence: 5
  givenname: Tomoaki
  orcidid: 0009-0007-6722-9757
  surname: Masuda
  fullname: Masuda, Tomoaki
  email: tomoaki.masuda@sloan.mit.edu
  organization: NTT Communications, Tokyo, Japan
BookMark eNpNkE1Lw0AQhhepYK39AYKHBc-p-9F8rLdYWo0Uekg8eQib3UlNbXfrbkrpvzehBT3NwLzPzPDcooGxBhC6p2RCKRFP79mqmDDCwgmPiGBTcoWGjLM4mEYRG_zrb9DY-w0hpMNCKqIh-iyyPJ-nzzjFCyd3cLTuG9fW4QJ825g1zmyB85NvYefxi_SgsTXdUH2ZRsktzm3dHqUDPDfrxgC4nkn9HlTr79B1Lbcexpc6Qh-LeTF7C5ar12yWLgPFpkkbAE0k0TqUgouQUBYSUcmIQ8KAVjSWUSVrzWKdiJpKpUJdq6imRLGKJommmo_Q43nv3tmfQ_d3ubEHZ7qTJWeRiFncLe5S9JxSznrvoC73rtlJdyopKXuNZa-x7DWWF40d83BmGgD4y1MaCk4Y_wUrxG88
CODEN IITJAU
Cites_doi 10.1007/978-3-030-97196-0_25
10.1145/2753476.2753481
10.1109/ACCESS.2022.3208146
10.1109/ICSTW.2018.00057
10.1007/978-981-13-9330-3_30
10.1007/s11227-021-03825-1
10.1007/s10664-021-10017-1
10.3390/s18092796
10.3390/fi14100276
10.1145/3368235.3368832
10.1007/978-981-13-1056-0_61
10.1109/ICST62969.2025.10988996
10.1109/SSD.2019.8893170
10.1007/978-3-031-50583-6_7
10.1109/KSE.2019.8919324
10.1145/3551349.3559545
10.5120/ijca2018916607
10.1109/TSE.2024.3402157
10.1109/ICOIN48656.2020.9016445
10.1109/JSEN.2024.3451349
10.1109/TSE.2021.3070692
10.1088/1742-6596/2493/1/012023
10.47277/ijcncs/1(6)2
10.1016/j.jss.2025.112408
10.1109/ICST49551.2021.00059
10.1007/978-981-16-0666-3_9
10.1007/s11219-021-09565-y
10.1145/3236454.3236511
10.1109/TSE.2024.3363611
10.1007/978-3-031-29860-8_26
10.1016/j.iotcps.2021.07.001
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
8FD
JQ2
L7M
L~C
L~D
DOI 10.1109/JIOT.2025.3609240
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005-present
IEEE All-Society Periodicals Package (ASPP) 1998-Present
IEEE Electronic Library (IEL)
CrossRef
Computer and Information Systems Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Computer and Information Systems Abstracts
Technology Research Database
Computer and Information Systems Abstracts – Academic
Advanced Technologies Database with Aerospace
ProQuest Computer Science Collection
Computer and Information Systems Abstracts Professional
DatabaseTitleList Computer and Information Systems Abstracts
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 2327-4662
EndPage 48802
ExternalDocumentID 10_1109_JIOT_2025_3609240
11159302
Genre orig-research
GroupedDBID 0R~
6IK
97E
AAJGR
AASAJ
AAWTH
ABAZT
ABJNI
ABQJQ
ABVLG
AGQYO
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
EBS
IFIPE
IPLJI
JAVBF
M43
OCL
PQQKQ
RIA
RIE
AAYXX
CITATION
7SC
8FD
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-c248t-e18a0dd5a9395012509ba63e82e1b17a6bafd27d89f1acc5dfc6f10c2b188d1d3
IEDL.DBID RIE
ISICitedReferencesCount 1
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001611085100038&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 2327-4662
IngestDate Thu Nov 20 16:41:36 EST 2025
Sat Nov 29 06:51:42 EST 2025
Wed Nov 19 08:27:19 EST 2025
IsPeerReviewed false
IsScholarly true
Issue 22
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c248t-e18a0dd5a9395012509ba63e82e1b17a6bafd27d89f1acc5dfc6f10c2b188d1d3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0001-9252-9937
0009-0007-6722-9757
0000-0002-4361-2563
0000-0001-8707-6603
0000-0002-9164-6645
PQID 3269727395
PQPubID 2040421
PageCount 16
ParticipantIDs crossref_primary_10_1109_JIOT_2025_3609240
proquest_journals_3269727395
ieee_primary_11159302
PublicationCentury 2000
PublicationDate 2025-11-15
PublicationDateYYYYMMDD 2025-11-15
PublicationDate_xml – month: 11
  year: 2025
  text: 2025-11-15
  day: 15
PublicationDecade 2020
PublicationPlace Piscataway
PublicationPlace_xml – name: Piscataway
PublicationTitle IEEE internet of things journal
PublicationTitleAbbrev JIoT
PublicationYear 2025
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
ref15
ref14
ref30
ref11
ref10
ref32
ref1
ref17
ref16
ref19
ref18
Minani (ref31) 2025; 226
Pessoa (ref21) 2024
ref24
ref23
ref26
ref25
ref20
Burhan (ref2) 2018; 18
ref22
ref28
ref27
ref29
ref8
ref7
ref9
ref4
ref3
ref6
ref5
References_xml – ident: ref22
  doi: 10.1007/978-3-030-97196-0_25
– ident: ref11
  doi: 10.1145/2753476.2753481
– ident: ref28
  doi: 10.1109/ACCESS.2022.3208146
– ident: ref10
  doi: 10.1109/ICSTW.2018.00057
– ident: ref3
  doi: 10.1007/978-981-13-9330-3_30
– ident: ref5
  doi: 10.1007/s11227-021-03825-1
– ident: ref16
  doi: 10.1007/s10664-021-10017-1
– volume: 18
  start-page: 2796
  issue: 9
  year: 2018
  ident: ref2
  article-title: IoT elements, layered architectures and security issues: A comprehensive survey
  publication-title: Sensors
  doi: 10.3390/s18092796
– ident: ref27
  doi: 10.3390/fi14100276
– ident: ref17
  doi: 10.1145/3368235.3368832
– ident: ref13
  doi: 10.1007/978-981-13-1056-0_61
– ident: ref32
  doi: 10.1109/ICST62969.2025.10988996
– ident: ref8
  doi: 10.1109/SSD.2019.8893170
– ident: ref25
  doi: 10.1007/978-3-031-50583-6_7
– ident: ref9
  doi: 10.1109/KSE.2019.8919324
– ident: ref12
  doi: 10.1145/3551349.3559545
– year: 2024
  ident: ref21
  article-title: RITA: Automatic framework for designing of resilient IoT applications
  publication-title: arXiv:2411.18324
– ident: ref4
  doi: 10.5120/ijca2018916607
– ident: ref24
  doi: 10.1109/TSE.2024.3402157
– ident: ref30
  doi: 10.1109/ICOIN48656.2020.9016445
– ident: ref20
  doi: 10.1109/JSEN.2024.3451349
– ident: ref7
  doi: 10.1109/TSE.2021.3070692
– ident: ref15
  doi: 10.1088/1742-6596/2493/1/012023
– ident: ref6
  doi: 10.47277/ijcncs/1(6)2
– volume: 226
  year: 2025
  ident: ref31
  article-title: IoT systems testing: Taxonomy, empirical findings, and recommendations
  publication-title: J. Syst. Softw.
  doi: 10.1016/j.jss.2025.112408
– ident: ref18
  doi: 10.1109/ICST49551.2021.00059
– ident: ref26
  doi: 10.1007/978-981-16-0666-3_9
– ident: ref1
  doi: 10.1007/s11219-021-09565-y
– ident: ref19
  doi: 10.1145/3236454.3236511
– ident: ref14
  doi: 10.1109/TSE.2024.3363611
– ident: ref29
  doi: 10.1007/978-3-031-29860-8_26
– ident: ref23
  doi: 10.1016/j.iotcps.2021.07.001
SSID ssj0001105196
Score 2.3640296
Snippet Internet of Things (IoT) systems refer to interconnected systems of devices that collect, process, and exchange data. As IoT adoption continues to grow,...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Index Database
Publisher
StartPage 48787
SubjectTerms Case studies
Cloud computing
Computer architecture
Data exchange
Data integrity
Data integrity testing
Engineers
Event handling
event handling testing
event logging testing
Focusing
Integrity
Internet of Things
Internet of Things (IoT) system testing
Security
Software engineering
software engineering (SE)
System testing
Taxonomy
Testing
testing framework
Title TISSEA: A Framework for Testing IoT Systems Based on Technical Software Engineering Aspects
URI https://ieeexplore.ieee.org/document/11159302
https://www.proquest.com/docview/3269727395
Volume 12
WOSCitedRecordID wos001611085100038&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 2327-4662
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0001105196
  issn: 2327-4662
  databaseCode: RIE
  dateStart: 20140101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELagYmChPEWhIA9MSGntpHZstoJaUYQKUgOqxBD5iVhS1Af8fWzXVUGIgS1DEkX35e6-O_s7A3BhsXJBjtjEOr6bdKQkCadSJlgQbDNLFQsq_uf7fDhk4zF_jGL1oIUxxoTNZ6blL8Navp6ohW-VtZ1fEp750ZGbeZ4vxVrrhgr2bITGlUuMePtu8FC4CjAlrYwiV2egH7knHKbyKwKHtNKv__ODdsFO5I-wuwR8D2yYah_UV2czwOiqB-ClGIxGve4V7ML-av8VdAQVFn6sRvUKB5MCxnHl8NqlMg0nFQyNdg8bHLnw_CmmBn4bWAi7QZc5OwRP_V5xc5vEgxQSlXbYPDGYCaQ1ETzjxGUkRxKkoJlhqcES54JKYXWaa8YtFkoRbRW1GKlUYsY01tkRqFWTyhwDmDJtcoOIQsI47mGkyFwFSj3toqlFqAEuVyYu35fzMspQZyBeejxKj0cZ8WiAQ2_T9Y3RnA3QXKFSRpealY5nck-2ODn547FTsO3f7pWCmDRBbT5dmDOwpT7mb7PpefhbvgABK73C
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA6igl6sT6zPHDwJ2ya7TZp4q9LSaq1CVxE8LHmKl620Vf--SZpSRTx428Muu8y3M_PNJN8EgDOLlQtyxCbW8d2kISVJOJUywYJgm1mqWFDxP_abgwF7euL3UawetDDGmLD5zNT8ZVjL1yP17ltldeeXhGd-dOQKaTRSPJNrLVoq2PMRGtcuMeL1695d7mrAlNQyilylgX5kn3Ccyq8YHBJLp_LPT9oEG5FBwtYM8i2wZMptUJmfzgCjs-6A57w3HLZbF7AFO_MdWNBRVJj7wRrlC-yNchgHlsNLl8w0HJUwtNo9cHDoAvSnGBv4bWQhbAVl5mQXPHTa-VU3iUcpJCptsGliMBNIayJ4xonLSY4mSEEzw1KDJW4KKoXVaVMzbrFQimirqMVIpRIzprHO9sByOSrNPoAp06ZpEFFIGMc-jBSZq0GpJ140tQhVwfncxMXbbGJGESoNxAuPR-HxKCIeVbDrbbq4MZqzCo7mqBTRqSaFY5rc0y1ODv547BSsdfPbftHvDW4Owbp_k9cNYnIElqfjd3MMVtXH9HUyPgl_zhcMA8EJ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=TISSEA%3A+A+Framework+for+Testing+IoT+Systems+Based+on+Technical+Software+Engineering+Aspects&rft.jtitle=IEEE+internet+of+things+journal&rft.au=Minani%2C+Jean+Baptiste&rft.au=Sabir%2C+Fatima&rft.au=Moha%2C+Naouel&rft.au=Yann-Gael+Gueheneuc&rft.date=2025-11-15&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.eissn=2327-4662&rft.volume=12&rft.issue=23&rft.spage=48787&rft.epage=48802&rft_id=info:doi/10.1109%2FJIOT.2025.3609240&rft.externalDBID=NO_FULL_TEXT
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2327-4662&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2327-4662&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2327-4662&client=summon