The quietly growing confidence in Silicon Valley could have huge implications: Breakthroughs with big AI models – which are expected to bring human-level artificial intelligence in the near future – could slow down.
Since the frenetic launch of ChatGPIT two years ago, AI believers have said that improvements in generic AI will accelerate as tech giants continue to add fuel to the fire in the form of data for training and computing muscle.
The argument was that delivering on the promise of the technology was simply a matter of resources – put in enough computing power and data, and artificial general intelligence (AGI) would emerge, capable of matching or exceeding human-level performance. Will happen.
Progress was moving at such a rapid pace that leading industry figures, including Elon Musk, called for a halt to AI research.
Yet major tech companies, including Musk’s company, moved forward and spent billions of dollars to avoid being left behind.
OpenAI, the Microsoft-backed creator of ChatGPIT, recently raised $6.6 billion to fund further progress.
According to CNBC, Musk’s AI company XAI is in the process of raising $6 billion to buy 100,000 Nvidia chips, which are cutting-edge electronic components that power large models.
However, there appear to be some problems standing in the way of AGI.
Industry insiders are beginning to acknowledge that large language models (LLMs) are not growing much at breakneck speed when pumped with more power and data.
Despite massive investment, performance is showing signs of improvement.
AI expert and frequent critic Gary Marcus said, “The overvaluation of companies like OpenAI and Microsoft is largely based on the assumption that LLMs, with continued scaling, will become artificial general intelligence.” “As I’ve always warned, it’s just a fantasy.”
‘now all’
A fundamental challenge is the limited amount of language-based data available for AI training.
Relying on language data alone for scaling is a major challenge, according to Scott Stevenson, CEO of AI legal practice firm Spellbook, which works with OpenAI and other providers.
“Some of the labs out there were focusing a lot on feeding more language, thinking it would get smarter,” Stevenson said.
Sasha Lucioni, researcher and AI lead at startup Hugging Face, argues that the halt in progress was predictable because of companies’ focus on size rather than purpose in model development.
“The pursuit of AGI has always been unrealistic, and the ‘bigger is better’ approach to AI was bound to eventually reach a limit – and I think that’s what we’re seeing here,” he told AFP.
The AI industry disputes these interpretations, saying that progress toward human-level AI is unpredictable.
“There is no wall,” OpenAI CEO Sam Altman posted on X on Thursday, without elaborating.
Dario Amodei, CEO of Anthropic, whose company develops cloud chatbots in partnership with Amazon, remains optimistic: “If you just look at the rate at which these capabilities are growing, it makes you think that We will reach there by 2026 or 2027.”
time to think
Yet, according to sources cited by The Information, OpenAI has delayed the release of the anticipated successor to GPT-4, the model powering ChatGPT, because its capacity increase fell short of expectations.
Now, the company is focusing on using its existing capabilities more efficiently.
This change in strategy is reflected in their recent O1 model, which is designed to provide more accurate answers through better reasoning rather than increased training data.
Stevenson said OpenAI had led to “radical improvements” by teaching its models to “spend more time thinking rather than reacting”.
He compared the advent of AI to the discovery of fire. It’s time to use success for specific tasks, instead of spending more fuel in the form of data and computer power.
Professor Walter de Brouwer of Stanford University has compared advanced LLMs to students moving from high school to university: “AI Baby was a chatbot that made a lot of improvements” and was prone to mistakes, he said.
“The Homo sapiens approach of thinking before you leap is coming,” he said.
(Except for the headline, this story has not been edited by NDTV staff and is published from a syndicated feed.)
