Computational Learning Theory: compact representation, efficient computation, and societal challenges in learning MDPs

ongoing

Computational learning theory has been highly successful over the last three decades, both in providing deep mathematical theories and in influencing the practice of machine learning. One of the great recent successes of computational learning theory has been the study of online learning and multi-arm bandits. This line of research has been highly successful, both theoretically and practically, addressing many important applications. Unfortunately, the recent theoretical progress in Markov Decision Process and reinforcement learning has been slower. Based on my fundamental contributions to reinforcement learning (e.g. policy gradient, sparse sampling and trajectory trees), to online learning and machine learning in general, I propose to take the theoretical and practical success of online learning to the “next level” by making significant breakthroughs in reinforcement learning. Our main aim is to advance the state of the art in the theory of reinforcement learning, and our research will revolve around three pillars: (1) compact representation, (2) efficient computation and (3) societal challenges, including fairness and privacy. A successful project will greatly impact reinforcement learning in all its stages. Modelling: Introducing new compact representation models, will enhance our understanding how to structure complex problems, which would greatly extend the applicability of reinforcement learning. Efficient computation: New algorithmic methodologies will give new insight for overcoming computational and statistical barriers both for planning and learning. Learning: New learning paradigms would address fundamental issues of copping with uncertainties in complex control environments of reinforcement learning. Societal challenges: Allowing the community to understand, assess, address and overcome societal challenges is of the greatest importance to the acceptance of AI methodologies by the general public.