Satya Nadella, CEO of Microsoft (MS), and Professor Yoshua Bengio of the University of Montreal, a master of deep learning, jointly announced that inference-centered ‘test-time computing’ will grow to be the brand new law of expansion. It denies that the ‘scaling law’, which has been the premise of artificial intelligence (AI) development over the past 10 years, has reached its limits and that the pace of AI development will decelerate.
CEO Nadella appeared on the keynote speech at ‘Ignite 2024’ held in Chicago, USA on the 18th (local time) and said that AI will double in development every six months.
He mentioned ‘Moore’s Law’, which states that computing performance doubles every 18 months, and said that AI performance will double every 6 months, which is quicker than that.
He went on to say that over the past few weeks, there was a debate over whether AI performance has hit a wall, stating that “this will not be a law of physics, it’s just an empirical commentary.” He also said that what is very important is that innovation is going on in all areas, including model architecture in addition to data and systems.
Moreover, he said, “We’re witnessing the emergence of latest scaling laws,” and “This may be seen in how OpenAI has improved AI with a technology called test-time computing, or inference.”
Professor Bengio also introduced OpenAI’s ‘o1’ as a “recent type of computational scaling” through a column within the Financial Times on the identical day.
He explained that there have been several developments within the Large Language Model (LLM), reminiscent of the ‘Chain of Thought (CoT)’, and that OpenAI’s o series has further developed this.
“So we’re seeing recent types of computational scaling emerge,” he said. “Not only do we’ve more training data and bigger models, but we also spend more time eager about answers, in inference-driven tasks like math, computer science, and science. “It’s going to result in significantly improved capabilities,” he said.
It continued to seem that improving reasoning skills can be the important thing to future LLM development. Particularly, OpenAI CEO Sam Altman emphasized in November of last yr that “a breakthrough in artificial general intelligence (AGI) has been found” regarding ‘Q*’, the premise of o1. Also, after the discharge of o1, there was talk that Google and Antropic were also developing inference functions.
Nevertheless, it is alleged that OpenAI failed to attain significant model improvement through pre-training, and that Google and Antropic are also experiencing similar problems, and the importance of improving inference functions is emphasized as a ‘recent scaling law’ to unravel this problem. .
As well as, it was identified that current AI technology has not been properly applied to applications, and that great progress may be made just by developing optimized applications reasonably than improving models.
“We’re confident that just pure application-level work will improve model performance by at the very least 10 to twenty times,” said Anini Midha Andreessen, a partner at Horowitz. “It is sufficient to provide intelligent prompts, optimized UX, and delivering context to the model at the proper timing in order that the AI can shine,” he explains.
It has been identified before that the capabilities of AI models can’t be 100% delivered in the shape of applications. When it comes to UI, it’s questionable whether chatbots, that are currently common, are the reply, and it has also been identified that there’s a huge difference between voice and text when communicating with chatbots.
Partner Madha says that if the present LLM capabilities are utilized properly, the performance that may be experienced will greatly increase.
This corresponds to what CEO Altman recently said in a chat with Reddit users: “AGI may be achieved with current equipment alone,” and “I feel the following big breakthrough might be agents.”
In other words, the agent can completely change the best way it interacts with the user and maximize the model’s performance just through the use of the performance of the prevailing model.
CEO Nadella also said at Ignite, “Copilot will grow to be the UI of AI.”
Reporter Lim Da-jun ydj@aitimes.com
