Humane Visual AI: Telling the stories behind a medical condition

Wonyoung So, Edyta Paulina Bogucka, Sanja Scepanovic, Sagar Joglekar, Ke Zhou, Daniele Quercia

View presentation: 2020-10-30T14:00:00Z GMT-0600 Change your timezone on the schedule page
Exemplar figure

complex problem communication, storytelling, AI, social media data, healthcare, Martini Glass structure


A biological understanding is key for managing medical conditions, yet psychological and social aspects matter too. The main problem is that these aspects are hard to quantify and inherently difficult to communicate. To quantify psychological aspects, this work mined around half a million Reddit posts in the sub-communities specialised in 14 medical conditions, and it did so with a new deep-learning framework. In so doing, it was able to associate mentions of medical conditions with those of emotions and various expressions of symptoms. To then quantify social aspects, this work designed a probabilistic approach that mines open prescription data from the National Health Service in England to compute the prevalence of drug prescriptions, and to relate such a prevalence to census data (putting health in context). To finally visually communicate each medical condition's biological, psychological, and social aspects through storytelling, we designed a narrative-style layered Martini Glass visualization. In a user study involving 52 participants, after interacting with our visualization, a considerable number of them changed their mind on previously held opinions: 10% gave more importance to the psychological aspects of medical conditions, and 27% were more favourable of the use of social media data, suggesting the importance of persuasive elements in interactive visualizations.