Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles

Jun Yuan, Mengchen Liu, Fengyuan Tian, Shixia Liu

View presentation:2022-10-19T15:57:00ZGMT-0600Change your timezone on the schedule page
2022-10-19T15:57:00Z
Exemplar figure, described by caption below
ArchExplorer: (a) the architecture visualization to show the architecture clusters; (b) three selected sub-clusters after zooming into cluster A; (c) a detailed comparison of the selected architectures.

Prerecorded Talk

The live footage of the talk, including the Q&A, can be viewed on the session page, VA for ML.

Fast forward
Abstract

Recent advances in artificial intelligence largely benefit from better neural network architectures. These architectures are a product of a costly process of trial-and-error. To ease this process, we develop ArchExplorer, a visual analysis method for understanding a neural architecture space and summarizing design principles. The key idea behind our method is to make the architecture space explainable by exploiting structural distances between architectures. We formulate the calculation of the pairwise distances as solving an all-pairs shortest path problem. To improve efficiency, we decompose this problem into a set of single-source shortest path problems. The time complexity is reduced from O(kn^2N) to O(knN). Architectures are hierarchically clustered according to the distances between them. A circle-packing-based architecture visualization has been developed to convey both the global relationships between clusters and local neighborhoods of the architectures in each cluster. Two case studies and a post-analysis are presented to demonstrate the effectiveness of ArchExplorer in summarizing design principles and selecting better-performing architectures.