Learning Objectives, Insights, and Assessments: How Specification Formats Impact Design

Elsie Lee, Shiqing He, Eytan Adar

View presentation:2021-10-28T14:00:00ZGMT-0600Change your timezone on the schedule page
Exemplar figure, described by caption below
There are many ways to specify a communicative visualization. We can do it based on learning objectives, insights we want the viewer to have, or assessments we want them to succeed at. In this work, we find that the format of specification impacts the choice of visualization design. In this figure we show examples of the three specifications with the most preferred design identified for each specification. The seven visualization designs are ordered from most effective to least effective.

Despite the ubiquity of communicative visualizations, specifying communicative intent during design is ad hoc. Whether we are selecting from a set of visualizations, commissioning someone to produce them, or creating them ourselves, an effective way of specifying intent can help guide this process. Ideally, we would have a concise and shared specification language. In previous work, we have argued that communicative intents can be viewed as a learning/assessment problem (i.e., what should the reader learn and what test should they do well on). Learning-based specification formats are linked (e.g., assessments are derived from objectives) but some may more effectively specify communicative intent. Through a large-scale experiment, we studied three specification types: learning objectives, insights, and assessments. Participants, guided by one of these specifications, rated their preferences for a set of visualization designs. Then, we evaluated the set of visualization designs to assess which specification led participants to prefer the most effective visualizations. We find that while all specification types have benefits over no-specification, each format has its own advantages. Our results show that learning objective-based specifications helped participants the most in visualization selection. We also identify situations in which specifications may be insufficient and assessments are vital.