FunSearch (so called since it searches for mathematical functions, not since it’s fun) continues a streak of discoveries in fundamental math and computer science that DeepMind has made using AI. First AlphaTensor found a solution to speed up a calculation at the center of many various sorts of code, beating a 50-year record. Then AlphaDev found ways to make key algorithms used trillions of times a day run faster.
Yet those tools didn’t use large language models. Built on top of DeepMind’s game-playing AI AlphaZero, each solved math problems by treating them as in the event that they were puzzles in Go or chess. The difficulty is that they’re stuck of their lanes, says Bernardino Romera-Paredes, a researcher at the corporate who worked on each AlphaTensor and FunSearch: “AlphaTensor is great at matrix multiplication, but principally nothing else.”
FunSearch takes a special tack. It combines a big language model called Codey, a version of Google’s PaLM 2 that’s fine-tuned on computer code, with other systems that reject incorrect or nonsensical answers and plug good ones back in.
“To be very honest with you, we’ve got hypotheses, but we don’t know exactly why this works,” says Alhussein Fawzi, a research scientist at Google DeepMind. “At first of the project, we didn’t know whether this is able to work in any respect.”
The researchers began by sketching out the issue they wanted to resolve in Python, a preferred programming language. But they neglected the lines in this system that may specify methods to solve it. That’s where FunSearch is available in. It gets Codey to fill within the blanks—in effect, to suggest code that can solve the issue.
A second algorithm then checks and scores what Codey comes up with. The most effective suggestions—even when not yet correct—are saved and given back to Codey, which tries to finish this system again. “Many can be nonsensical, some can be sensible, and just a few can be truly inspired,” says Kohli. “You are taking those truly inspired ones and also you say, ‘Okay, take these ones and repeat.’”
After a few million suggestions and just a few dozen repetitions of the general process—which took just a few days—FunSearch was capable of provide you with code that produced an accurate and previously unknown solution to the cap set problem, which involves finding the most important size of a certain sort of set. Imagine plotting dots on graph paper. The cap set problem is like attempting to determine what number of dots you’ll be able to put down without three of them ever forming a straight line.
It’s super area of interest, but necessary. Mathematicians don’t even agree on methods to solve it, let alone what the answer is. (Additionally it is connected to matrix multiplication, the computation that AlphaTensor found a solution to speed up.) Terence Tao on the University of California, Los Angeles, who has won lots of the top awards in mathematics, including the Fields Medal, called the cap set problem “perhaps my favorite open query” in a 2007 blog post.