Overcoming the curse of dimensionality through nonlinear stochastic algorithms

ended

In a series of relevant real world problems it is of fundamental importance to approximatively compute evaluations of high-dimensional functions. Such high-dimensional approximation problems appear, e.g., in stochastic optimal control problems in operations research, e.g., in supervised learning problems, e.g., in financial engineering where partial differential equations (PDEs) and forward backward stochastic differential equations (FBSDEs) are used to approximatively price financial products, and, e.g., in nonlinear filtering problems where stochastic PDEs are used to approximatively describe the state of a given physical system with only partial information available. Standard approximation methods for such approximation problems suffer from the socalled curse of dimensionality in the sense that the number of computational operations of the approximation method grows at least exponentially in the problem dimension. It is the key objective of this project to design and analyze approximation algorithms which provably overcome the curse of dimensionality in the case of stochastic optimal control problem, nonlinear PDEs, nonlinear FBSDEs, certain SPDEs, and certain supervised learning problems. We intend to solve many of the above named approximation problems by combining different types of multilevel Monte Carlo approximation methods, in particular, multilevel Picard approximation methods, with stochastic gradient descent (SGD) optimization methods. Another chief objective of this project is to prove the conjecture that the SGD optimization method converges in the training of ANNs with ReLU activation. We expect that the outcome of this project will have a significant impact on the way how highdimensional PDEs, FBSDEs, and stochastic optimal control problems are solved in engineering and operations research and on the mathematical understanding of the training of ANNs by means of the SGD optimization methods.