In-situ Topological Reduction of Scientific 3D Data

ongoing

Modern scientific methods heavily rely on large-scale 3D simulations. However, current data production speeds are much higher than storage speeds (5 orders of magnitude on typical supercomputers). This imbalance constitutes a major bottleneck in the scientific computing pipeline, such that most of the data generated by a simulation is not saved to disk, and thus remains unvisualized, unexplored and unanalyzed. TORI addresses this data bottleneck by introducing the next generation data reduction tools for large-scale scientific 3D data. TORI’s angle of attack is based on original and important advances in Topological Data Analysis (TDA), a class of techniques popularized in scientific visualization. TORI addresses data reduction at two levels: (i) at the data level, by deriving an analysis framework for ensembles of topological objects that is inspired by optimal transport, and (ii) at the computation level, by entirely revisiting TDA to adapt it to the context of high-performance in-situ data analysis. To identify informative datasets (i), TORI will introduce efficient methods for distance computations, barycenter evaluations and trajectory analysis. To perform this analysis on-the-fly (ii), TORI will revisit TDA with task parallel algorithms, coarse-to-fine computations and TDA-aware lossy compressors. TORI will be implemented in open-source in the Topology ToolKit, a leading TDA package, and interfaced with standard scientific computing packages (VTK, ParaView). It will be integrated in simulation codes with Catalyst and evaluated on real-life use cases in climatology, geophysics and astrophysics. TORI will have a far reaching impact on all fields of science using large-scale 3D simulations. By bringing together optimal transport and TDA in an innovative coarse-to-fine model, TORI will establish TDA as a standard tool for the analysis of large-scale ensemble datasets and it will initiate a new line of research in high-performance in-situ data analysis.