A Hybrid In Situ Approach for Cost Efficient Image Database Generation



Valentin Bruder, Matthew Larsen, Thomas Ertl, Hank Childs, Steffen Frey

 View presentation:2022-10-20T21:45:00ZGMT-0600Change your timezone on the schedule page
2022-10-20T21:45:00Z

Exemplar figure, described by caption below — Our hybrid in situ approach dynamically distributes visualization tasks based on predicted render times. It combines characteristics from traditional in situ techniques: it employs dedicated visualization nodes such as in transit in situ, and simulation nodes can also take over rendering tasks like in inline in situ. Dynamic task distribution allows to address various inefficiencies (variability, scalability, rightsizing and overhead) and save precious supercomputer resources.

Prerecorded Talk

The live footage of the talk, including the Q&A, can be viewed on the session page, Text, Language, and Image Data.

Fast forward

Keywords

Visualization, High performance computing, In situ

Abstract

The visualization of results while the simulation is running is increasingly common in extreme scale computing environments. We present a novel approach for in situ generation of image databases to achieve cost savings on supercomputers. Our approach, a hybrid between traditional inline and in transit techniques, dynamically distributes visualization tasks between simulation nodes and visualization nodes, using probing as a basis to estimate rendering cost. Our hybrid design differs from previous works in that it creates opportunities to minimize idle time from four fundamental types of inefficiency: variability, limited scalability, overhead, and rightsizing. We demonstrate our results by comparing our method against both inline and in transit methods for a variety of configurations, including two simulation codes and a scaling study that goes above 19K cores. Our findings show that our approach is superior in many configurations. As in situ visualization becomes increasingly ubiquitous, we believe our technique could lead to significant amounts of reclaimed cycles on supercomputers.