Low-Cost Post Hoc Reconstruction of HPC Simulations at Full Resolution

Abstract

High performance computing has played a pivotal and ongoing role in the field of computational fluid dynamics, enabling the simulation of increasingly larger-scale models. However, this rapid growth in model and simulation size has outpaced the capabilities of input/output (I/O) operations. Consequently, the conventional approach of saving and outputting data to persistent storage for analysis has become increasingly challenging, limiting the benefits of these advanced models. To address this challenge, we present a method for effectively handling massive-scale simulation data, ensuring its persistence at full spatial and temporal resolution for flexible post hoc analysis. We employ an in situ approach that captures interprocess communicated data, compressing cached data to a fraction of the overall simulation domain. We successfully reconstruct subdomains at full spatial and temporal resolution during post-processing through communication-free rerun using the cached halos. We detail the storage requirements of the new approach and demonstrate the substantial reductions in computational resources required to precisely recapitulate data within a local region of interest.