In a primary, a scientific convention welcomed paper submissions from any space of science, however with one catch: AI needed to do a lot of the work. Known as Agents4Science 2025, the Oct. 22 digital occasion targeted on the work of synthetic intelligence brokers — methods that pair giant language fashions with different instruments or databases to carry out multistep duties.
From formulating hypotheses to analyzing information and offering the primary spherical of peer critiques, AI brokers took the lead. Human reviewers then stepped in to evaluate the highest submissions. In all, 48 papers out of 314 made the minimize. Every needed to element how individuals and AI collaborated on each stage of the analysis and writing course of.
“We’re seeing this fascinating paradigm shift,” mentioned James Zou, a pc scientist at Stanford College who co-organized the convention. “Persons are beginning to discover utilizing AI as a co-scientist.”
Most scientific journals and conferences at the moment ban AI coauthors and prohibit peer reviewers from counting on AI. These insurance policies intention to keep away from hallucinations and different points associated to AI use. Nonetheless, this method makes it robust to study how good AI is at science. That’s what Agents4Science aimed to discover, Zou mentioned, calling the convention an experiment, with all of the supplies publicly accessible for anybody to check.
On the digital assembly, people offered AI-assisted work spanning fields resembling economics, biology and engineering. Min Min Fong, an economist on the College of California, Berkeley, and her crew collaborated with AI to check car-towing information from San Francisco. Their examine discovered that waiving excessive towing charges helped low-income individuals maintain their autos.
“AI was actually nice at serving to us with computational acceleration,” Fong mentioned. However, she discovered, “you must be actually cautious when working with AI.”
For example, the AI saved citing the incorrect date for when San Francisco’s rule waiving towing charges went into impact. Fong needed to verify this within the authentic supply to find the error. “The core scientific work nonetheless stays human-driven,” she mentioned.
For Risa Wechsler, a computational astrophysicist at Stanford who helped overview submissions, the outcomes had been combined. The papers she noticed had been technically right, she mentioned, “however they had been neither fascinating nor essential.” She was excited in regards to the potential of AI for analysis however remained unconvinced that right this moment’s brokers can “design strong scientific questions.” And, she added, the technical talent of AI can “masks poor scientific judgment.”
Nonetheless, the occasion included some glimmers of hope for the way forward for AI in science. Silvia Terragni, a machine studying engineer on the firm Upwork in San Francisco, mentioned that she gave ChatGPT some context in regards to the sorts of issues her firm offers with and requested the bot to suggest paper concepts. “One in all these was the winner,” she mentioned, chosen as one of many three prime papers within the convention. It was a examine about utilizing AI reasoning in a job market. “I believe [AI] can truly give you novel concepts,” she mentioned.
