‘AI scientists’ that run their own research

Could this next-gen technology devise, run, and validate its own experiments?

Spotted: AI is boosting efficiencies across almost every industry, and that includes AI research itself. Sakana AI has recently introduced the AI Scientist, which allows large language models (LLMs) to independently perform machine learning research and speed up the entire process.

The AI Scientist automates the whole research lifecycle — it can generate research ideas, execute experiments, summarise experimental results, and present findings in a full scientific manuscript. The system can also automate the peer review process to review the papers generated, write feedback, and develop ways to further improve results.

The AI Scientist has four main processes. First, the system brainstorms ideas and evaluates their novelty. It then edits a codebase to implement the novel algorithms and runs experiments to gather both numerical data and visual summaries. After this, it crafts a scientific report, explaining and contextualising the results. Finally, the system generates an automated peer review based on machine learning conference standards.

While the AI Scientist demonstrates a strong ability to innovate on top of well-established ideas, such as diffusion modelling, it’s still yet to be seen whether it can also come up with novel, paradigm-shifting ideas. In addition, ethical considerations also need to be taken into account, including the potential for misuse and the system’s ability to lower the quality of the review process.

Sakana AI has recently raised $30 million in a seed funding round backed by Lux Capital and Khosla Ventures. The startup is also supported by a number of Japanese companies, including Sony Group, and has announced a research partnership with NTT (Nippon Telegraph and Telephone).

Written By: Lisa Magloff