A new research paper by David Silver of Google and Canadian scientist Rich Sutton has sparked renewed debate in the AI community. The paper, titled Welcome to the Era of Experience, outlines what the authors believe to be the next phase of artificial intelligence. According to them, the future lies not in training models on existing data but in allowing them to interact with the world and learn through experience. This marks a major shift from current approaches and aims to address a growing challenge: the scarcity of fresh, high-quality human data to train ever-larger AI models.(Source: Business Insider Africa)
Silver and Sutton trace modern AI development through two main stages. The first was the Simulation Era, where reinforcement learning enabled models to play games like chess and Go millions of times to learn strategies. This method helped build systems such as AlphaGo and AlphaZero. However, it was limited to tasks with clearly defined rewards and could not handle broader, real-world problems. The second stage, the Human Data Era, was launched by the 2017 paper Attention is All You Need. This approach trained AI models on vast amounts of human-created data from the internet and gave rise to generative tools like ChatGPT. While powerful, this method is now seen as hitting a ceiling, as models cannot surpass the knowledge they are trained on.
The proposed Era of Experience offers a more open-ended solution. Instead of relying solely on human data, AI systems would learn directly from their own actions and the results of those actions. This method could create new streams of experiential data that reflect real-world interactions. The authors suggest that this could lead to machines capable of reaching or even surpassing human-level understanding in areas such as education, health, and science. Examples in the paper include AI agents that adjust health recommendations based on biometric signals or environmental tools that react to real-time changes in carbon levels.
The paper also addresses what some view as a missed opportunity by Google. Though it pioneered many of the ideas that powered the Human Data Era, the real breakthroughs and commercial success came from OpenAI with ChatGPT and from Anthropic with Claude. Silver and Sutton’s proposal can be seen as both a bold new vision and a critique of the current reliance on static data. They warn that without the ability to learn independently, AI models risk becoming repetitive and stuck within the boundaries of human knowledge.
In summary, this report reflects a growing awareness within the AI community that the next big leap may come not from better data, but from better experiences. If the Era of Experience takes hold, it could push AI beyond its current limits and move it closer to the long-standing goal of artificial general intelligence.
