A Case Study of Using Analytic Provenance to Reconstruct User Trust in a Guided Visual Analytics System

Nadia Boukhelifa, Evelyne Lutton, Anastasia Bezerianos

External link (DOI)
View presentation:2021-10-24T13:15:00ZGMT-0600Change your timezone on the schedule page
2021-10-24T13:15:00Z
Exemplar figure, but none was provided by the authors
Fast forward

Direct link to video on YouTube: https://youtu.be/DI5zEkurlYk

Abstract

In this paper, we demonstrate how analytic provenance can be exploited to re-construct user trust in a guided Visual Analytics (VA) system, and suggest that interaction log data analysis can be a valuable tool for on-line trust monitoring. Our approach moves away from the subjective and often a-posteriori evaluations of user trust, towards more objective measures that are not only continuously tracked and updated, but also reflect both the confidence of the user in system suggestions, and the uncertainty of the system with regards to user goals. We argue that this approach is more suitable for guided visual analytics systems such as ours, where user strategies, goals and even trust can evolve over time, in reaction to new system feedback and insights from the exploration. Through the analysis of log data from a past user study with twelve participants performing a guided visual analysis task, we found that the stability of user exploration strategies is a promising factor to study “trust”. However, indirect metrics based on provenance, such as user evaluation counts and disagreement rates, are alone not sufficient to study trust reliably in guided VAs. We conclude with open challenges and opportunities for exploiting analytic provenance to support trust monitoring in guided VA systems.