Large-Scale Evaluation of Topic Models and Dimensionality Reductions for 2D Text Spatialization

Daniel Atzberger, Tim Cech, Rico Richter, Matthias Trapp, Willy Scheibel, Jürgen Döllner, Tobias Schreck

DOI: 10.1109/TVCG.2023.3326569

Room: 109

2023-10-25T23:00:00ZGMT-0600Change your timezone on the schedule page
2023-10-25T23:00:00Z
Exemplar figure, described by caption below
Two-dimensional layout for the 20 Newsgroups dataset. Each point represents a document within the corpus and the color its class. The layout originates from applying Latent Semantic Indexing to the term-document matrix and a subsequent application of t-SNE on the topics, which are then aggregated to the document positions.
Fast forward
Full Video
Keywords

Text visualization, spatialization, dimensionality reduction algorithms, topic modeling

Abstract

Topic models are a class of unsupervised learning algorithms for detecting the semantic structure within a text corpus. Together with a subsequent dimensionality reduction algorithm, topic models can be used for deriving spatializations for text corpora as two-dimensional scatter plots, reflecting semantic similarity between the documents and supporting corpus analysis. Although the choice of the topic model, the dimensionality reduction, and their underlying hyperparameters significantly impact the resulting layout, it is unknown which particular combinations result in high-quality layouts with respect to accuracy and perception metrics. To investigate the effectiveness of topic models and dimensionality reduction methods for the spatialization of corpora as two-dimensional scatter plots (or basis for landscape-type visualizations), we present a large-scale, benchmark-based computational evaluation. Our evaluation consists of (1) a set of corpora, (2) a set of layout algorithms that are combinations of topic models and dimensionality reductions, and (3) quality metrics for quantifying the resulting layout. The corpora are given as document-term matrices, and each document is assigned to a thematic class. The chosen metrics quantify the preservation of local and global properties and the perceptual effectiveness of the two-dimensional scatter plots. By evaluating the benchmark on a computing cluster, we derived a multivariate dataset with over 45000 individual layouts and corresponding quality metrics. Based on the results, we propose guidelines for the effective design of text spatializations that are based on topic models and dimensionality reductions. As a main result, we show that interpretable topic models are beneficial for capturing the structure of text corpora. We furthermore recommend the use of t-SNE as a subsequent dimensionality reduction.