Beyond Low-Rank Factorizations

ongoing

Low-rank matrix factorizations (LRMFs), such as principal component analysis, nonnegative matrix factorization and sparse component analysis, are linear dimensionality reduction techniques and are powerful unsupervised models to represent and analyze high-dimensional data sets. They are used in a wide variety of areas such as machine learning, signal processing, and data mining. Many LRMFs have been proposed in the literature, in particular in the last two decades, and used extensively in many applications, such as recommender systems, blind source separation, and text mining. Although LRMFs have known and still know tremendous success, they have several limitations. Two key limitations are that they are linear models, and that they only learn one layer of features. In this project, we go beyond LRMFs, considering generalized LRMFs (G-LRMFs) that overcome these limitations, considering non-linear and deep LRMFs. These generalizations have been introduced more recently, and they have hardly been explored compared to LRMFs. In particular, eLinoR will focus on three fundamental aspects: (1) Theory: understand G-LRMFs with a focus on computational complexity and identifiability (uniqueness of the decompositions), (2) Algorithms: solve G-LRMFs via provably correct and heuristic algorithms, and (3) Models: design new G-LRMFs for specific applications. This unified approach will enable us to understand these problems better, to develop and analyze algorithms, and to then use them for applications. The novelty and ground-breaking nature of this project also lies in the methodology, exploring G-LRMFs from different but complementary perspectives, bringing together ideas from different fields. The ultimate goal of eLinoR is to provide practitioners with theoretical and algorithmic tools for G-LRMFs, allowing them to decide which model and algorithm to use in which situation and what kind of outcome to expect, leading to a widespread and reliable use of G-LRMFs.