Understanding and Fixing Bottlenecks in Optimization for Modern Machine Learning
The status, timeline and source of this project. A free account opens every field.
Source: EU Funding & Tenders Portal
Modern machine learning models have been successfully deployed across fields, from scientific studies to tech- nological developments in industry, but their development remains poorly understood. The training of a large language model such as GPT-3 is estimated to cost $4.6M, and public attempts to replicate the training process alone required teams of engineers to rotating on-call for months, mon
- Status
- Under Construction
- Location
- France
- Updated
- 22 Jul 2026
- Source
- EU Funding & Tenders Portal
ConstructChat
Ask ConstructChat about Understanding and Fixing Bottlenecks in Optimization for Modern Machine Learning
Pick a question to edit it, or write your own.
First answer free · no account
Project Information FAQ
Project Information
Participants
| Not available for this record. |
Status
Original status | ongoing |
Taiyo status | Under Construction |
Taiyo last update | 2026-07-22 |
Contact
| Not available for this record. |
Description
Description | Modern machine learning models have been successfully deployed across fields, from scientific studies to tech- nological developments in industry, but their development remains poorly understood. The training of a large language model such as GPT-3 is estimated to cost $4.6M, and public attempts to replicate the training process alone required teams of engineers to rotating on-call for months, monitoring various statistics and constantly tweaking the training procedure when it broke. Existing theoretical frameworks offer limited insights into this process, as they do not capture the main difficulties that arise in practice when training neural networks, leaving practitioners to rely on error-prone heuristics and expensive trial-and-error. This leads not only to a large devel- opment cost dominated by wasted resources, but also limits the possible impacts of machine learning to areas considered profitable by industries that have the resources to carry this development. The objective of this project is to build a better understanding of how recently identified bottlenecks in neural network training slow down optimization and how to adress them. The specific aims are to: (a) Understand the impact of class imbalance on the dynamics of neural networks to identify where to allocate algorithmic resources. (b) Develop a theory to capture optimization difficulties early in training to guide the development of algorithms that improve performance during this crucial phase. (c) Identify new bottlenecks that arise from applications to new data types. The project combines experimental expertise of the postdoctoral and the theoretical expertise of the host insti- tution to identify and describe the real impact of data characteristics on neural network training. Understanding these bottlenecks will help develop more efficient and reliable algorithms and guidelines on best practices that depend on properties of the data. |
Location
Country | France |
Source
Source | EU Funding & Tenders Portal |
URL |
More Details
Record type | Project |
Article published date | 2026-01-01 |
Research
Free account
View all details
You are reading the public part. A free account opens the rest:
- Every detail on this project
- News and media coverage
- Technical research
Free account · no card · lands you back on this project
More infrastructure projects in France
- Projet 18e - Porte de Clignancourt
- Projet Broussais
- 250 premiers projets pour entretenir et rénover Paris
- Protocole d'accord RATP/Ville de Paris: création de 2000 logements d'ici 2024
- Chapelle Dubois Condorcet
- Les dix lauréats de l'appel à projets de l'Arc de l'innovation
- Projet 16e - 17e - Porte Maillot
- Landivisiau Power Plant Brittany
