A Scanner Deeply: Predicting Gaze Heatmaps on Visualizations Using Crowdsourced Eye Movement Data



Sungbok Shin, Sunghyo Chung, Sanghyun Hong, Niklas Elmqvist

 View presentation:2022-10-19T16:33:00ZGMT-0600Change your timezone on the schedule page
2022-10-19T16:33:00Z

Exemplar figure, described by caption below — Sungbok Shin is a Ph.D. Student at the dept. of Computer in the University of Maryland, College Park, United States. He gained his B.S. as a dual degree of Computer Science and Engineering and Mathematics and his M.Eng. in Computer Science and Engineering in Korea University, Seoul, South Korea. Sungbok is interested in understanding how existing machine learning models (e.g., neural networks) learn humans' perceptions while looking at visualizations. He is also interested in developing applications that would facilitate and can improve chart designs.

Prerecorded Talk

The live footage of the talk, including the Q&A, can be viewed on the session page, Understanding and Modeling How People Respond to Visualizations.

Fast forward

Abstract

Visual perception is a key component of data visualization. Much prior empirical work uses eye movement as a proxy to understand human visual perception. Diverse apparatus and techniques have been proposed to collect eye movements, but there is still no optimal approach. In this paper, we review 30 prior works for collecting eye movements based on three axes: (1) the tracker technology used to measure eye movements; (2) the image stimulus shown to participants; and (3) the collection methodology used to gather the data. Based on this taxonomy, we employ a webcam-based eyetracking approach using task-specific visualizations as the stimulus. The low technology requirement means that virtually anyone can participate, thus enabling us to collect data at large scale using crowdsourcing: approximately 12,000 samples in total. Choosing visualization images as stimulus means that the eye movements will be specific to perceptual tasks associated with visualization. We use these data to propose a SCANNER DEEPLY, a virtual eyetracker model that, given an image of a visualization, generates a gaze heatmap for that image. We employ a computationally efficient, yet powerful convolutional neural network for our model. We compare the results of our work with results from the DVS model and a neural network trained on the Salicon dataset. The analysis of our gaze patterns enables us to understand how users grasp the structure of visualized data. We also make our stimulus dataset of visualization images available as part of this paper’s contribution.