Artificial intelligence researchers claim to have made the world’s first scientific discovery using a large language model, a breakthrough that will revolutionize the technology behind ChatGPT and similar programs can generate information beyond human knowledge.
The finding emerged from Google DeepMindwhere scientists investigate whether large language models, which support modern chatbots such as OpenAI’s ChatGPT and Google’s Bard, can do more than repackage information learned in training and come up with new insights.
“When we started the project, there was no indication that it would produce anything truly new,” said Pushmeet Kohli, the head of AI for science at DeepMind. “As far as we know, this is the first time a truly new scientific discovery has been made by a large language model.”
Large language models, or LLMs, are powerful neural networks that learn the patterns of language, including computer code, from large amounts of text and other data. Since the whirlwind arrival of ChatGPT last year, the technology has debugged faulty software and extracted everything from college essays and travel itineraries to poems about climate change in the style of Shakespeare.
But while the chatbots have been hugely popular, they don’t generate new knowledge and are prone to confusion, resulting in answers that, in line with the best bar drills, are fluent and plausible, but badly flawed.
To build “FunSearch”, short for “searching in the function space”, DeepMind enlisted an LLM to write solutions to problems in the form of computer programs. The LLM is paired with an “evaluator” that automatically ranks the programs according to how well they perform. The best programs are then combined and fed back to the LLM to improve. This drives the system to continuously evolve weak programs into more powerful programs that can discover new knowledge.
The researchers tackled FunSearch on two puzzles. The first was a long-standing and somewhat mysterious challenge in pure mathematics known as the pet pose problem. It is about finding the largest set of points in space where no three points form a straight line. FunSearch has sent out programs that generate new large-cap sets that go beyond the best mathematicians have come up with.
The second puzzle was the trash packaging problem, which looks for the best ways to pack items of different sizes into containers. While this applies to physical objects, such as the most efficient way to arrange boxes in a shipping container, the same math applies to other areas, such as scheduling computing jobs in data centers. The problem is typically solved by packing items either in the first bin that has room, or in the bin with the least available space where the item will still fit. FunSearch found a better approach that avoided leaving small gaps unlikely to ever be filled, according to results published in Earth.
“In the last two or three years there have been some exciting examples of human mathematicians working with AI to make progress on unsolved problems,” said Sir Tim Gowers, professor of mathematics at the University of Cambridge, who was not involved in the research. “This work potentially gives us another very interesting tool for such collaborations, enabling mathematicians to efficiently search for clever and unexpected constructions. Even better, these constructions are humanly interpretable.”
Researchers are now exploring the range of scientific problems that FunSearch can handle. A major limiting factor is that the problems must have solutions that can be automatically verified, which precludes many questions in biology, where hypotheses often need to be tested with laboratory experiments.
to newsletter promotion
The more immediate impact may be for computer programmers. For the past 50 years, coding has greatly improved as people have created ever more specialized algorithms. “It’s actually going to be transformative in how people approach computer science and algorithmic discovery,” Kohli said. “For the first time, we see that LLMs are not taking over, but definitely helping to push the boundaries of what is possible in algorithms.”
Jordan Ellenberg, professor of mathematics at the University of Wisconsin-Madison, and co-author of the paper, said: “What I find really exciting, even more than the specific results that we found, is the prospects that this has for the future of human-machine interaction in mathematics.
“Instead of generating a solution, FunSearch generates a program that finds the solution. A solution to a particular problem may not give me any insight into how to solve other related problems. But a program that finds the solution, it’s something that one can read and interpret and hopefully generate ideas for the next problem and the next and the next.”
