Learning deep networks with crowdsourcing for relevance evaluation

In this paper, we propose a novel relevance evaluation method using labels collected from crowdsourcing. The proposed method not only predicts the relevance between query texts and responses in information retrieval systems but also performs the label aggregation tasks simultaneously. It first merge...

Full description

Saved in:
Bibliographic Details
Published in:EURASIP journal on wireless communications and networking Vol. 2020; no. 1; pp. 1 - 11
Main Authors: Wu, Ming, Yin, Xiaochun, Li, Qianmu, Zhang, Jing, Feng, Xinqi, Cao, Qi, Shen, Haiyuan
Format: Journal Article
Language:English
Published: Cham Springer International Publishing 25.04.2020
Springer Nature B.V
SpringerOpen
Subjects:
Artificial neural networks
Communications Engineering
Crowdsourcing
Deep learning
Engineering
Evaluation
Information retrieval
Information Systems Applications (incl.Internet)
Labels
Machine learning
Multi-modal Sensor Data Fusion in Internet of Things
Networks
Relevance evaluation
Signal,Image and Speech Processing
Crowdsourcing
Deep learning
Information retrieval
Relevance evaluation
ISSN:1687-1499, 1687-1472, 1687-1499
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we propose a novel relevance evaluation method using labels collected from crowdsourcing. The proposed method not only predicts the relevance between query texts and responses in information retrieval systems but also performs the label aggregation tasks simultaneously. It first merges two kinds of heterogeneous data (i.e., image and query text) and constructs a CNN-like deep neural network. Then, on the top of its softmax layer, an additional layer was built to model the crowd workers. Finally, classification models for relevance prediction and aggregated labels for training examples can be simultaneously learned from noisy labels. Experimental results show that the proposed method significantly outperforms other state-of-the-art methods on a real-world dataset.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1687-1499
1687-1472
1687-1499
DOI:10.1186/s13638-020-01697-2