Statistical Optimality and Computational Efficiency: batch and sequential unsupervised learning under additional structure and sampling constraints

forthcoming

Unsupervised learning is a key problem of artificial intelligence, at the crossroad of statistics and machine learning. The aim is to infer patterns from unlabelled data, by providing learning algorithms that are computationally efficient - i.e. polynomial time - and statistically performant - i.e. minimising an error criterion - and by characterising the fundamental limits for learning. In the last decade, deep and important phenomena of statistical-computational trade-offs have been unveiled: for some canonical vanilla problems, it is now admitted that no algorithm is both statistically optimal and computationally efficient. However, and somewhat surprisingly, many extensions of these commonly admitted conjectures to other models that present slight variations have been recently proven wrong. The reason is that these model variations give rise to additional structure. This could be a blessing if it can be exploited by a well chosen computationally efficient algorithm, or a curse if it confuses any such algorithm. So that many fundamental unsupervised learning problems, like robust or hierarchical clustering as well as ordering models like ranking or seriation, are poorly understood. Beyond this, in modern applications like recommender systems, unsupervised learning is often done in a sequential active way, as a complement to batch learning. Yet, efficient algorithms and the understanding of their limits are also vastly lacking, in particular in the presence of additional structure. And active learning is often done under sampling constraints, which adds a layer of model variations with respect to batch learning. In SOCE, I will tackle these complex unsupervised learning problems, which are not well understood despite their importance. I will go from batch to active unsupervised learning, and study their interface through sampling constraints. I will develop new mathematical tools and algorithms that will be instrumental for a systematic study of these problems.