Classes are Not Clusters: Improving Label-Based Evaluation of Dimensionality Reduction

A common way to evaluate the reliability of dimensionality reduction (DR) embeddings is to quantify how well labeled classes form compact, mutually separated clusters in the embeddings. This approach is based on the assumption that the classes stay as clear clusters in the original high-dimensional...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on visualization and computer graphics Vol. 30; no. 1; pp. 781 - 791
Main Authors: Jeon, Hyeon, Kuo, Yun-Hsin, Aupetit, Michael, Ma, Kwan-Liu, Seo, Jinwook
Format: Journal Article
Language:English
Published: IEEE 01.01.2024
Subjects:
Clustering
Clustering Validation Measures
Degradation
Dimensionality reduction
Dimensionality Reduction Evaluation
Distortion measurement
Extraterrestrial measurements
Nonlinear distortion
Reliability
Scalability
ISSN:1077-2626, 1941-0506, 1941-0506
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A common way to evaluate the reliability of dimensionality reduction (DR) embeddings is to quantify how well labeled classes form compact, mutually separated clusters in the embeddings. This approach is based on the assumption that the classes stay as clear clusters in the original high-dimensional space. However, in reality, this assumption can be violated; a single class can be fragmented into multiple separated clusters, and multiple classes can be merged into a single cluster. We thus cannot always assure the credibility of the evaluation using class labels. In this paper, we introduce two novel quality measures-Label-Trustworthiness and Label-Continuity (Label-T&C)-advancing the process of DR evaluation based on class labels. Instead of assuming that classes are well-clustered in the original space, Label-T&C work by (1) estimating the extent to which classes form clusters in the original and embedded spaces and (2) evaluating the difference between the two. A quantitative evaluation showed that Label-T &C outperform widely used DR evaluation measures (e.g., Trustworthiness and Continuity, Kullback-Leibler divergence) in terms of the accuracy in assessing how well DR embeddings preserve the cluster structure, and are also scalable. Moreover, we present case studies demonstrating that Label-T &C can be successfully used for revealing the intrinsic characteristics of DR techniques and their hyperparameters.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:1077-2626
1941-0506
1941-0506
DOI:10.1109/TVCG.2023.3327187