That’s why Mirhoseini has been using AI to optimize AI chips. Back in 2021, she and her collaborators at Google built a non-LLM AI system that might resolve where to put various components on a pc chip to optimize efficiency. Although another researchers failed to duplicate the study’s results, Mirhoseini says that investigated the paper and upheld the work’s validity—and he or she notes that Google has used the system’s designs for multiple generations of its custom AI chips.
More recently, Mirhoseini has applied LLMs to the issue of writing kernels, low-level functions that control how various operations, like matrix multiplication, are carried out in chips. She’s found that even general-purpose LLMs can, in some cases, write kernels that run faster than the human-designed versions.
Elsewhere at Google, scientists built a system that they used to optimize various parts of the corporate’s LLM infrastructure. The system, called AlphaEvolve, prompts Google’s Gemini LLM to put in writing algorithms for solving some problem, evaluates those algorithms, and asks Gemini to enhance on essentially the most successful—and repeats that process several times. AlphaEvolve designed a brand new approach for running datacenters that saved 0.7% of Google’s computational resources, made further improvements to Google’s custom chip design, and designed a brand new kernel that sped up Gemini’s training by 1%.
Which may sound like a small improvement, but at an enormous company like Google it equates to enormous savings of time, money, and energy. And Matej Balog, a staff research scientist at Google DeepMind who led the AlphaEvolve project, says that he and his team tested the system on only a small component of Gemini’s overall training pipeline. Applying it more broadly, he says, could lead on to more savings.
3. Automating training
LLMs are famously data hungry, and training them is dear at every stage. In some specific domains—unusual programming languages, for instance—real-world data is simply too scarce to coach LLMs effectively. Reinforcement learning with human feedback, a way through which humans rating LLM responses to prompts and the LLMs are then trained using those scores, has been key to creating models that behave consistent with human standards and preferences, but obtaining human feedback is slow and expensive.
Increasingly, LLMs are getting used to fill within the gaps. If prompted with loads of examples, LLMs can generate plausible synthetic data in domains through which they haven’t been trained, and that synthetic data can then be used for training. LLMs can be used effectively for reinforcement learning: In an approach called “LLM as a judge,” LLMs, slightly than humans, are used to attain the outputs of models which can be being trained. That approach is essential to the influential “Constitutional AI” framework proposed by Anthropic researchers in 2022, through which one LLM is trained to be less harmful based on feedback from one other LLM.
Data scarcity is a very acute problem for AI agents. Effective agents have to give you the option to perform multistep plans to perform particular tasks, but examples of successful step-by-step task completion are scarce online, and using humans to generate latest examples can be pricey. To beat this limitation, Stanford’s Mirhoseini and her colleagues have recently piloted a technique through which an LLM agent generates a possible step-by-step approach to a given problem, an LLM judge evaluates whether each step is valid, after which a brand new LLM agent is trained on those steps. “You’re not limited by data anymore, since the model can just arbitrarily generate increasingly more experiences,” Mirhoseini says.
4. Perfecting agent design
One area where LLMs haven’t yet made major contributions is within the design of LLMs themselves. Today’s LLMs are all based on a neural-network structure called a transformer, which was proposed by human researchers in 2017, and the notable improvements which have since been made to the architecture were also human-designed.
