FNNWV: farthest-nearest neighbor-based weighted voting for class-imbalanced crowdsourcing - Zhang, Wenjun; Jiang, Liangxiao; Chen, Ziqi; Li, Chaoqun (Science China. Information sciences, 2024)
In crowdsourcing scenarios, we can hire crowd workers to label crowdsourced tasks and then use label integration algorithms to infer the integrated label for each instance in the tasks. As more and more label integration algorithms are proposed, the performance of inference based only on the information of the inferred instance gradually converges. Recent algorithms attempt to exploit the information of the inferred instance’s nearest neighbors to infer and achieve good performance. However, when crowdsourced tasks are class-imbalanced, negative instances are more easily to occur in the nearest neighbors because negative instances are the majority, and thus recent algorithms are more easily biased toward the negative class. To this end, in this paper, we propose a novel label integration algorithm called farthest-nearest neighbor-based weighted voting (FNNWV) for class-imbalanced crowdsourcing. Specifically, FNNWV considers the nearest neighbors to be more similar to the inferred instance and thus uses them to vote ayes in weighted voting. Yet at the same time, FNNWV considers the farthest neighbors to be more different from the inferred instance and thus uses them to vote nays in weighted voting. Since negative instances are easier to occur in both the nearest neighbors and the farthest neighbors, FNNWV weakens the effect of negative instances by voting ayes and nays. The experimental results on 22 simulated and one real-world crowdsourced datasets show that FNNWV significantly outperforms all the other state-of-the-art competitors.