TAP with ease: a generic recommendation system for trigger-action programming based on multi-modal representation learning

The escalating popularity of smart devices has given rise to an increasing trend wherein users leverage customized trigger-action programming (TAP) rules within the Internet of Things (IoT) to automate various aspects of their lives. This article addresses the challenge of effectively combining func...

Full description

Saved in:

Bibliographic Details
Published in:	Applied soft computing Vol. 166; p. 112163
Main Authors:	Wu, Gang, Wang, Ming, Wang, Feng
Format:	Journal Article
Language:	English
Published:	Elsevier B.V 01.11.2024
Subjects:	Knowledge graph embedding Multi-modal representation learning Natural language processing Trigger-action programming Knowledge graph embedding Multi-modal representation learning Natural language processing Trigger-action programming
ISSN:	1568-4946
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	The escalating popularity of smart devices has given rise to an increasing trend wherein users leverage customized trigger-action programming (TAP) rules within the Internet of Things (IoT) to automate various aspects of their lives. This article addresses the challenge of effectively combining functions provided by many smart devices and online services by introducing a novel multi-modal representation learning model called TAP-TAG. This model integrates both textual and graph structures inherent in TAP rules, offering a holistic method to rule recommendation. TAP-TAG comprises two branches: the Knowledge Graph Embedding model, which projects triplets extracted from the TAP dataset into embeddings, and convolution neural networks that extract semantic features from the textual content of TAP rules. Extensive experiments are conducted on real-world TAP datasets to evaluate our model’s ability to recommend relevant rules based on user preferences. The experimental results show that TAP-TAG can outperform the state-of-the-art method by 5% in Precision@5, indicating that TAP-TAG is highly effective in providing accurate and diverse recommendations for TAP rules. •Present a multimodal data representation framework for recommending TAP rules.•TAP rules are divided into semantic and relation levels based on their modality.•Improve the representation quality by training text and graph modalities jointly.•Extensive experiments verify the effectiveness of the proposed framework.
ISSN:	1568-4946
DOI:	10.1016/j.asoc.2024.112163