Diversibus Viis Plurima Solvo

ongoing

"Foundation models represent a paradigm shift in AI, exhibiting remarkable capabilities across multiple tasks. Their true potential lies in generalizing across diverse domains and modalities, a largely untapped frontier. DVPS advances this frontier by focusing on multimodal foundation models (MMFM), aiming to harness their capabilities across various application domains. DVPS emphasizes three core benefits of MMFM: label efficiency, compute reusability, and engineering efficiency. However, achieving these benefits in multimodal settings presents challenges such as modality-specific architecture and cross-modal alignment. To overcome these, DVPS aims to develop generalizable methods that work across diverse modalities and domains, creating a unified framework for MMFM development and integrating new modalities into existing models. The project focuses on generating foundational knowledge, delivering tested methods, and creating algorithms to expand MMFM capabilities across domains like cardiology, geo-intelligence, and language communication. DVPS also includes two ""surprise domains"" to drive innovation by challenging initial assumptions. Key objectives include the development of AutoDVPS, a toolkit for automated MMFM design, and the creation of DVPSBench, a benchmarking suite for evaluating MMFM across tasks and domains. DVPS aims to foster a European ecosystem for MMFM research, promoting transparency, fairness, and ethical compliance in line with European values. Through collaboration and open-source contributions, DVPS seeks to standardize and advance MMFM as a scientifically rigorous discipline."