Scope2Screen: Focus+Context Techniques for Pathology Tumor Assessment in Multivariate Image Data

Jared Jessup, Robert Krüger, Simon Warchol, John Hoffer, Jeremy Muhlich, Cecily C. Ritch, Giorgio Gaglia, Shannon Coy, Yu-An Chen, Jia-Ren Lin, Sandro Santagata, Peter Sorger, Hanspeter Pfister

External link (DOI)
View presentation:2021-10-29T15:45:00ZGMT-0600Change your timezone on the schedule page
2021-10-29T15:45:00Z
Exemplar figure, described by caption below
Scope2Screen is a scalable system for visualization, focus+context exploration and annotation of whole-slide, high-plex, tissue images, of 100GB in size, with 10^9 or more pixels per channel, and millions of individual cells. As a multidisciplinary team of visualization experts, microscopists, and pathologists we identified key image exploration and annotation tasks involving finding, magnifying, quantifying, and organizing regions of interest in an intuitive and cohesive manner. We validate our designs with domain experts and apply Scope2Screen in two case studies involving lung and colorectal cancer.
Fast forward

Direct link to video on YouTube: https://youtu.be/Y2tGl2r8v6U

Abstract

Inspection of tissues using a light microscope is the primary method of diagnosing many diseases, notably cancer. Highly multiplexed tissue imaging builds on this foundation, enabling collection of up to 60 channels of molecular information plus cell and tissue morphology using antibody staining. This provides unique insight into disease biology and promises to help with design of patient-specific therapies. However, a substantial gap remains with respect to visualizing the resulting multivariate image data and effectively supporting pathology workflows in digital environments on screen. We, therefore, developed Scope2Screen, a scalable software system for focus+context exploration and annotation of whole-slide, high-plex, tissue images. Our approach scales to analyzing 100GB images of 10^9 or more pixels per channel, containing millions of individual cells. A multidisciplinary team of visualization experts, microscopists, and pathologists identified key image exploration and annotation tasks involving finding, magnifying, quantifying, and organizing regions of interest (ROIs) in an intuitive and cohesive manner. Building on a scope-to-screen metaphor, we present interactive lensing techniques that operate at single-cell and tissue levels. Lenses are equipped with task-specific functionality and descriptive statistics, making it possible to analyze image features, cell types, and spatial arrangements (neighborhoods) across image channels and scales. A fast sliding-window search guides users to regions similar to those under the lens; these regions can be analyzed and considered either separately or as part of a larger image collection. A novel snapshot method enables linked lens configurations and image statistics to be saved, restored, and shared with these regions. We validate our designs with domain experts and apply Scope2Screen in two case studies involving lung and colorectal cancers to discover cancer-relevant image features.