E-RXAI-IoT: A Systematic Evaluation Framework of Rule-Based XAI Methods for Anomaly Detection in IoT Systems

The increasing complexity of Internet of Things (IoT) systems necessitates the development of interpretable and robust artificial intelligence (AI) techniques for anomaly detection. While prior work has explored deep learning-based explainable AI (XAI) methods, systematic studies evaluating rule-bas...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access Vol. 13; pp. 188730 - 188754
Main Authors: Namrita Gummadi, Anna, Abdelrahim, Elsaid Md, Gad, Ibrahim, Abdallah, Mustafa
Format: Journal Article
Language:English
Published: Piscataway IEEE 01.01.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Accuracy
anchor
Anomalies
Anomaly detection
Artificial intelligence
Artificial neural networks
coverage
Datasets
Decision trees
Deep learning
Explainable AI
Explainable artificial intelligence
fidelity
Internet of Things
IoT security
Machine learning
Measurement
MEMS
Microelectromechanical systems
N-BaIoT
precision
Predictive models
RuleFit
Security
Source code
sparsity
stability
Support vector machines
Systematics
XAI evaluation metrics
ISSN:2169-3536, 2169-3536
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The increasing complexity of Internet of Things (IoT) systems necessitates the development of interpretable and robust artificial intelligence (AI) techniques for anomaly detection. While prior work has explored deep learning-based explainable AI (XAI) methods, systematic studies evaluating rule-based XAI approaches for IoT anomaly detection remain limited. In this paper, we fill this gap by investigating two state-of-the-art rule-based XAI methods-Anchor and RuleFit-across multiple machine learning models: Random Forest (RF), Decision Tree (DT), Deep Neural Network (DNN), and Support Vector Machine (SVM). We introduce a comprehensive evaluation framework that assesses these XAI methods using seven critical metrics mapped to AI and IoT security domains. These metrics are descriptive accuracy, sparsity, efficiency, stability, fidelity, coverage, and precision. We apply our framework to two distinct IoT datasets. The first one is the N-BaIoT dataset, which captures IoT botnet attacks (Mirai and Gafgyt) on commercial devices. The second is the MEMS dataset, which focuses on detecting anomalies in sensor readings from smart manufacturing processes. We perform a comprehensive evaluation of the seven XAI evaluation metrics for the two rule-based XAI methods for these two datasets. Through this study, we systematically highlight the strengths and limitations of rule-based XAI techniques in the context of IoT anomaly detection, providing practical insights into their interpretability and robustness. We have made the source code for our XAI evaluation framework publicly available for the community to foster reproducibility and encourage further research in this domain with more methods, metrics, and datasets.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2025.3627529