Where and Why is My Bot Failing? A Visual Analytics Approach for Investigating Failures in Chatbot Conversation Flows



Avi Yaeli, Sergey Zeltyn

 External link (DOI) 

 View presentation:2021-10-28T17:30:00ZGMT-0600Change your timezone on the schedule page
2021-10-28T17:30:00Z

Exemplar figure, described by caption below — Flow analysis UI. Statistical information such as visits, abandonment and trend over time is presented for each dialog node

Fast forward

Direct link to video on YouTube: https://youtu.be/G38QCVe5YfU

Keywords

Machine Learning Techniques, Other Topics and Techniques, Data Analysis, Reasoning, Problem Solving, and Decision Making, Application Motivated Visualization, Task Abstractions & Application Domains, High-dimensional Data, Text/Document Data

Abstract

The ongoing coronavirus pandemic has accelerated the adoption of AI-powered task-oriented chatbots by businesses and healthcare organizations. Despite advancements in chatbot platforms, implementing a successful and effective bot is still challenging and requires a lot of manual work. There is a strong need for tools to help conversation analysts quickly identify problem areas and, consequently, introduce changes to chatbot design. We present a visual analytics approach and tool for conversation analysts to identify and assess common patterns of failure in conversation flows. We focus on two key capabilities: path flow analysis and root cause analysis. Interim evaluation results from applying our tool in real-world customer production projects are presented.