Exploring Interactions with Printed Data Visualizations in Augmented Reality

Wai Tong, Zhutian Chen, Meng Xia, Leo Yu-Ho Lo, Linping Yuan, Benjamin Bach, Huamin Qu

View presentation: 2022-10-19T19:00:00Z GMT-0600 Change your timezone on the schedule page
Exemplar figure, described by caption below
We investigate the possibility of interacting with printed visualizations in Augmented Reality. Suppose a student receives (a) a leaflet about university ranking and wants to analyze three universities' ranking histories of interest. Examples of interactions with (b) digital content overlaid: (c) tilt the paper to rescale the y-axis, (d) move (translate) to zoom (e) unfold to show two charts side by side and link them, (f) point to select elements and highlight them in the other chart.

Prerecorded Talk

The live footage of the talk, including the Q&A, can be viewed on the session page, Immersive Analytics and Situated Visualization.

Fast forward

This paper presents a design space of interaction techniques to engage with visualizations that are printed on paper and augmented through Augmented Reality. Paper sheets are widely used to deploy visualizations and provide a rich set of tangible affordances for interactions, such as touch, folding, tilting, or stacking. At the same time, augmented reality can dynamically update visualization content to provide commands such as pan, zoom, filter, or detail on demand. This paper is the first to provide a structured approach to mapping possible actions with the paper to interaction commands. This design space and the findings of a controlled user study have implications for future designs of augmented reality systems involving paper sheets and visualizations. Through workshops (N=20) and ideation, we identified 81 interactions that we classify in three dimensions: 1) commands that can be supported by an interaction, 2) the specific parameters provided by an (inter)action with paper, and 3) the number of paper sheets involved in an interaction. We tested user preference and viability of 11 of these interactions with a prototype implementation in a controlled study (N=12, HoloLens 2) and found that most of the interactions are intuitive and engaging to use. We summarized interactions (e.g., tilt to pan) that have strong affordance to complement “point” for data exploration, physical limitations and properties of paper as a medium, cases requiring redundancy and shortcuts, and other implications for design.