The Age of AI Pre-Training Is Ending, OpenAI Co-Founder Ilya Sutskever Argued at NeurIPS 2024

OpenAI co-founder Ilya Sutskever recently gave a lecture at the Neural Information Processing Systems (NeurIPS) 2024 conference in Vancouver, Canada arguing that the age of artificial intelligence pre-training is ending and forecasted the rise of an AI superintelligence.

OpenAI co-founder Ilya Sutskever gave a lecture at the Neural Information Processing Systems (NeurIPS) 2024 conference in Vancouver, arguing that the age of artificial intelligence pre-training is coming to an end and predicting the rise of an AI superintelligence.

Sutskever’s lecture highlighted several key points:

**Computing Power Outpacing Data Availability for AI Pre-Training**

According to Sutskever, the rate at which computing power is increasing — thanks to better hardware, software, and machine-learning algorithms — outpaces the total amount of data available for AI model training. He likened data to fossil fuels, which will eventually run out.

“Computing power is increasing exponentially, but the amount of data is not,” said Sutskever. “This means that at some point, we will reach the limits of what can be achieved with pre-training.”

**Agentic AI, Synthetic Data, Inference Time Computing as Next Steps**

The OpenAI co-founder predicted that agentic AI, synthetic data, and inference time computing are the next evolutions of artificial intelligence that will eventually give rise to an AI superintelligence.

“Agentic AI is able to make decisions without human input and will be able to operate in the real world,” explained Sutskever. “Synthetic data is generated by AI models and can be used to train other AI models on a massive scale.”

**AI Agents in the Crypto World: A Closer Look**

AI agents are advancing rapidly in the crypto space, going beyond current chatbot models by being able to make decisions without human input.

This capability has made AI agents a central topic in the crypto narrative with the rise of AI memecoins and large-language models (LLMs) like Truth Terminal.

Truth Terminal, an LLM, went viral by promoting a memecoin called Goatseus Maximus (GOAT), which soared to a market capitalization of $1 billion — grabbing the attention of retail investors and venture capitalists.

“AI agents are able to interact with the world in a way that is not possible for humans,” said Sutskever. “This opens up new possibilities for how we can use AI to solve problems.”

**Google's Gemini 2.0 to Power AI Agents**

Google’s DeepMind artificial intelligence laboratory introduced Gemini 2.0, an artificial intelligence model that will power AI agents.

According to Google, agents built with the Gemini 2.0 framework will be able to assist in complex tasks such as coordinating between multiple websites and logical reasoning.

“Gemini 2.0 is a major advance in AI technology and will enable the creation of a new generation of AI agents,” said DeepMind in a statement.

Advancements in AI agents that can independently act and reason will pave the way for AI to overcome the issue of data hallucinations.

AI hallucinations, a phenomenon that occurs due to incorrect data sets, are becoming more prevalent as AI pre-training relies heavily on using older LLMs to train newer LLMs, which degrades performance over time.