Continual and Sequential Learning for Artificial Intelligence

forthcoming

Machine Learning systems, while promising, lack the autonomy needed for many real-world applications beyond mere proof-of-concept stages. The key challenge lies in enabling AI to continuously adapt to shifting data distributions and proactively seek information under high uncertainty. Fields such as drug discovery and micro-chemistry are expecting breakthroughs from AI, given the vast and intricate search spaces they deal with, coupled with expensive data acquisition. It is vital for algorithms to steer this search, assimilate new data, and strategically explore promising zones. Reinforcement Learning (RL) offers tools and methods for agents to autonomously learn from their actions, but its efficacy has been largely confined to stationary, single-task settings. ConSequentIAL's vision is a Continual and Sequential Learning AI that marries supervised and unsupervised learning with advanced data gathering and RL-driven discovery mechanisms. To achieve these goals, I propose to bridge the theory of constrained and non-stationary RL to build a sound and useful mathematical formulation of the problem. On these new solid grounds, I develop novel algorithmic principles that allow the agent to detect and respond to external shifts, while remaining aware of her own impact on the system she interacts with. I address the memory-versus-stability trade-off central to continual learning by enabling agents to actively plan their skill acquisition in accordance with their long-term goals. The ambition of this project is to position AI to tackle the consequential scientific challenges ahead.