Efficient infinite-dimensional optimization over measures

ongoing

Optimization over probability measures has become a powerful approach for solving complex problems that involve probabilistic modeling. Advanced by the PI and collaborators, it extends finite-dimensional optimization to the infinite-dimensional space of probability measures. This framework provides a princi- pled way to address in particular the task of sampling, that refers to the process of drawing samples from a complex probability distribution, either to approximate it or generate new data. In Bayesian machine learning for instance, we can model uncertainty of the predictions by sampling a model’s parameters. Similarly, in generative modeling, sampling is crucial for producing new data such as images or text. Existing methods struggle with complex measures, are difficult to evaluate, and are limited to Euclidean spaces, making them unsuitable for infinite-dimensional ones like functions or operators. Also, their lack of computational efficiency limits their use in sequential sampling tasks. My goal is to create a unified framework to design and evaluate efficient methods for sampling measures over general spaces. Central to my approach is the use of tools from optimal transport and information geometry, which will help compare measures and design optimization dynamics. OptInfinite will advance optimization over measures by addressing two key challenges: 1) developing optimization objectives and geometries suited to the space of measures over general (possibly-infinite-dimensional) spaces to design tractable schemes and metrics, and 2) learning to solve advanced optimization problems, such as sequences of optimization tasks. This framework will yield novel sampling methods, whose efficiency can be evaluated using optimization tools. It will also enable to compare measures and reveal where and how they are different. Ultimately, our work will provide a clear methodology and toolset applicable across multiple domains.