An AI agent that may autonomously conduct scientific research has emerged. The agent generates research ideas, designs and executes experiments, analyzes the outcomes, and writes papers. The research team said that that is “the start of a brand new era of scientific discovery,” nevertheless it stays to be seen what the actual results shall be.
VentureBeat reported on the thirteenth (local time) that startup Sakana AI is collaborating with researchers at Oxford University and the University of British Columbia to develop an AI agent called ‘AI Scientist’ that may autonomously perform end-to-end scientific research. Post your paper within the archiveIt was reported that.
Sakana AI is an organization founded in Tokyo, Japan last yr by Lillian Jones, one in all the authors of Google’s ‘Transformer’ paper. It is legendary for its ‘merged evolution model’ that enhances model performance through model crossbreeding over lots of of years. It also released a big language model (LLM) specializing in Japanese, and is gaining attention within the Japanese market.
The AI ​​Scientist they only unveiled is a totally automated pipeline that generates scientific papers using LLM.
Given a broad scientific research direction with an easy initial code base, the AI ​​then proceeds without human intervention: generating research ideas, writing the crucial code, running experiments, summarizing and visualizing the outcomes, and even presenting the research ends in a full scientific paper, all through your entire research life cycle.
Additionally it is explained that by introducing an automatic peer review process to guage the generated papers, provide feedback, and further improve the outcomes, the generated papers will be evaluated with almost human-level accuracy.
As well as, it was revealed that it boasts high efficiency when it comes to cost. In line with the research team, the computing cost for an AI scientist to write down one research paper is simply about 15 dollars (about 20,000 won).
AI scientists undergo 4 principal processes: â–²idea generation â–²repeated experiments â–²paper writing â–²paper review.
In the concept generation stage, given a starting code ‘template’ that describes the specified research topic, AI scientists brainstorm recent research directions. During this process, they search the AI ​​academic search engine ‘Semantic Scholar’ to envision whether their ideas are novel.
Given a template and an idea, the proposed experiment is run in the next iterative experimental phase and plots are generated to visualise the outcomes. Notes explaining the content of every plot are provided, so that every one the knowledge needed to write down a paper, similar to saved diagrams and experimental notes, will be provided.
In the course of the paper writing phase, AI scientists write concise and informative progress reports in the usual machine learning conference paper style, and autonomously search semantic scalars to cite relevant papers.
Within the paper review stage, peer-reviews are generated through automated LLM-based reviewers that may evaluate the generated papers with near-human accuracy. The generated reviews will be used to enhance the project or as feedback for future idea generation. This enables AI scientists to iteratively improve their research results.
In line with the researchers, AI scientists wrote papers that exceeded the acceptance criteria of top machine learning conferences, as assessed by automated reviewers.
The researchers used quite a lot of closed LLMs, similar to GPT-4o and Claude 3 Sonnet, in addition to open-source LLMs, similar to DeepSeek and Llama-3, and located that Claude 3 Sonnet produced the very best quality papers.
Nevertheless, the researchers said that AI scientists have some shortcomings, and it stays to be seen whether they’ll actually help advance science.
Initially, I heard that since there isn’t a visual function, it’s difficult to resolve the visual problems of the paper or read the plot. Due to this fact, it was revealed that multimodality is planned to be added in the longer term.
AI may misrepresent its ideas or unfairly compare them to other content, resulting in misleading results, it said. It also identified that mistakes will be made within the means of evaluating numbers, identical to a typical LLM.
Nevertheless, he stressed that “AI Scientist represents a vital step toward realizing the complete potential of AI in scientific research,” and that it “opens the door to infinite possibilities for innovation and problem solving in among the most difficult areas of science and technology.”
He continued, “It remains to be unknown whether we are going to have the opportunity to propose ideas that may ultimately change the paradigm,” adding, “We imagine that AI scientists shall be great companions to human scientists, but only time will tell what the final result shall be.”
AI scientist On GitHub Could be used.
Reporter Park Chan cpark@aitimes.com
