Entropy-Based Scheduling Policy for Cross Aggregate Ranking Workloads

Many data exploration applications require the ability to identify the top-k results according to a scoring function. We study a class of top-k ranking problems where top-k candidates in a dataset are scored with the assistance of another set. We call this class of workloads cross aggregate ranking....

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on services computing Vol. 11; no. 3; pp. 507 - 520
Main Authors: Chengcheng Dai, Nutanong, Sarana, Chi-Yin Chow, Cheng, Reynold
Format: Journal Article
Language:English
Published: IEEE 01.05.2018
Subjects:
Aggregates
Entropy
Indexes
knowledge and data engineering tools and techniques
Processor scheduling
Query processing
Scheduling
Uncertainty
ISSN:1939-1374, 2372-0204
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Many data exploration applications require the ability to identify the top-k results according to a scoring function. We study a class of top-k ranking problems where top-k candidates in a dataset are scored with the assistance of another set. We call this class of workloads cross aggregate ranking. Example computation problems include evaluating the Hausdorff distance between two datasets, finding the medoid or radius within one dataset, and finding the closest or farthest pair between two datasets. In this paper, we propose a parallel and distributed solution to process cross aggregate ranking workloads. Our solution subdivides the aggregate score computation of each candidate into tasks while constantly maintains the tentative top-k results as an uncertain top-k result set. The crux of our proposed approach lies in our entropy-based scheduling technique to determine result-yielding tasks based on their abilities to reduce the uncertainty of the tentative result set. Experimental results show that our proposed approach consistently outperforms the best existing one in two different types of cross aggregate rank workloads using real datasets.
ISSN:1939-1374
2372-0204
DOI:10.1109/TSC.2016.2586062