OoDAnalyzer: Interactive Analysis of Out-of-Distribution Samples



Changjian Chen, Jun Yuan, Yafeng Lu, Yang Liu, Hang Su, Songtao Yuan, Shixia Liu

 External link (DOI) 

 View presentation:2020-10-28T14:00:00ZGMT-0600Change your timezone on the schedule page
2020-10-28T14:00:00Z

Exemplar figure — A visual analysis approach for understanding Out-of-Distribution (OoD) samples caused by the distribution difference between training and test data: (a) controls and visualization aids; (b) a grid-based visualization to illustrate OoD samples in context. The color of the grid cell and border encodes the category, and the sequential color scheme represents the OoD scores of samples; (c) image and (d) saliency map representations of the selected region.

Fast forward

Direct link to video on YouTube: https://youtu.be/L7fumWtCSAc

Keywords

Out-of-Distribution detection, grid layout, interactive visualization

Abstract

One major cause of performance degradation in predictive models is that the test samples are not well covered by the training data. Such not well-represented samples are called OoD samples. In this paper, we propose OoDAnalyzer, a visual analysis approach for interactively identifying OoD samples and explaining them in context. Our approach integrates an ensemble OoD detection method and a grid-based visualization. The detection method is improved from deep ensembles by combining more features with algorithms in the same family. To better analyze and understand the OoD samples in context, we have developed a novel kNN-based grid layout algorithm motivated by Hall's theorem. The algorithm approximates the optimal layout and has O(kN^2) time complexity, faster than the grid layout algorithm with overall best performance but O(N^3) time complexity. Quantitative evaluation and case studies were performed on several datasets to demonstrate the effectiveness and usefulness of OoDAnalyzer.