Evaluating Forecasting, Knowledge, and Visual Analytics

Yafeng Lu, Michael Steptoe, Verica Buchanan, Nancy Cooke, Ross Maciejewski

External link (DOI)
View presentation:2021-10-24T13:55:00ZGMT-0600Change your timezone on the schedule page
2021-10-24T13:55:00Z
Exemplar figure, but none was provided by the authors
Fast forward

Direct link to video on YouTube: https://youtu.be/OWfUNDsSSrk

Abstract

In this paper, we explore the intersection of knowledge and the forecasting accuracy of humans when supported by visual analytics. We have recruited 40 experts in machine learning and trained them in the use of a box office forecasting visual analytics system. Our goal was to explore the impact of visual analytics and knowledge in human-machine forecasting. This paper reports on how participants explore and reason with data and develop a forecast when provided with a predictive model of middling performance ($R^2 \approx .7$). We vary the knowledge base of the participants through training, compare the forecasts to the baseline model, and discuss performance in the context of previous work on algorithmic aversion and trust.