Visual Omniversal Learning from Universal Teachers

ongoing

"Volute is an ambitious project in the space of Computer Vision and Machine Learning. While these research domains have made tremendous progress in recent years and have contributed to the success of many tech companies, we are still seeing the highest impact of artificial intelligence (AI) in the natural language processing (NLP) space - for example, in products such as ChatGPT or AI-assisted search on Google and Bing. This success stems from a ""foundation model"" (e.g., GPT) that is then fine-tuned with hand-labelled instructions to become an assistive chatbot. While many models in Computer Vision are named foundation models, none of them have been able to produce the same impact. The main challenge in Computer Vision is the complexity of the input and output space. NLP tasks are text-based; in contrast, the range of representations in computer vision is endless: images, videos, depth, infrared, medical data, satellites, and more. At the same time, the outputs are similarly diverse: images, text, discrete labels, pixel annotations, bounding boxes, 3D reconstruction, audio, and more. These outputs are often not necessarily meant for human consumption but are the inputs for other systems, such as autonomous driving or robotics. It is thus unrealistic to expect to find an actual foundation model in computer vision. Volute aims to find a new universal model type: an ""omniversal trainer"", which acts as a data generator to train task-specific models. The omniversal trainer uses a generator, trained on large data collections, and thus learns priors about the world, such as physics, appearance, and visual diversity, but does not need specific task knowledge. A downstream-task model can then be trained with limited task-specific data, augmented with the world knowledge of the universal trainer. This drastically reduces the labelling effort, both in time and cost, for downstream applications and will have a significant impact as it poses a paradigm shift in Computer Vision."