First-Ever Research Conference Led By AI – Expert Reaction

How does AI perform at a scholarly conference? Researchers will find out this week, when an online conference will host papers where AI authorship is not only allowed—but required.

Organised by a group of Stanford University researchers, the 1st Open Conference of AI Agents for Science aims to explore “if and how AI can independently generate novel scientific insights, hypotheses, and methodologies while maintaining quality through AI-driven peer review.”

The half-day conference kicks off at 4:45 am NZ time on Thursday, 23 October. Accepted papers are listed here.

The Science Media Centre asked experts to respond.

Te Taka Keegan, Associate Professor, AI Institute, Computer Science Department, University of Waikato, comments:

“I’ve had a quick look at this conference and find it quite interesting. I believe AI will become increasingly utilised for academic research and publishing, but significant care must be taken to verify AI generated outputs.

“Looking through both the submitted and rejected papers, it appears that AI has NOT been used to offer indigenous perspectives—something it should not do without indigenous governance over the AI system. From this initial review, it’s difficult to determine whether inherent data biases have proliferated through the AI decision-making and AI-generated content.

“This is something that would need to be carefully monitored before any validity could be given to this type of activity.”

No conflicts of interest.

Ali Knott, Professor in Artificial Intelligence, Victoria University of Wellington, comments:

“I think the important thing is that this conference is recognised as an experiment. Its purpose (as I understand it) is to evaluate the possibility of AI authors and AI reviewers, rather than to advocate for AI systems in these roles. It is far too early for that kind of advocacy – I’m sure most researchers would agree. But evaluations and experiments are fine. It’s in the nature of science to run experiments and evaluations. My main worry is that the conference is understood (by journalists, or the public) as a substantive research conference, rather than an experiment. That would be a misconception.

“I’d like to point out Royal Society Te Apārangi’s guidelines on use of Generative AI in research, which I helped to develop. These guidelines are rather high-level – but they include a general principle which basically rules out the use of autonomous AIs as authors or reviewers. It’s Principle 3.2.2: [human researchers] should ‘be responsible for research outputs’. Specifically, ‘GenAI systems are neither authors nor co-authors. Authorship implies agency and responsibility, and therefore lies with human researchers’ (my highlights).

“This principle doesn’t preclude a conference of the kind being run, provided it’s understood as experimental in purpose. If the conference is understood as presenting and reviewing actual substantive research, it would contravene the guidelines we laid down.”

No conflicts of interest.

Dr Andrew Lensen, Senior Lecturer/Programme Director of Artificial Intelligence, Victoria University of Wellington, comments:

“This is an interesting experiment, where different large language models (LLMs) have been used as part of the research process, including in both research and review. The use of AI in academic research is a very polarising topic that often only makes the news when someone uses AI inappropriately to replace human review or to automate academic writing without any oversight. So, it is good to see this being framed as an ‘experimental conference’, where the pros and cons of using AI in this manner can be safely understood without affecting existing conferences and processes.

“I think the most interesting findings will come from the post-conference analyses that the conference organisers will later publish. Looking at some of the accepted papers, there is a large amount of variance in AI review quality and consistency — often, the AI reviewer agents disagree with each other! But then, so do human reviewers, often. While the reviews themselves look ‘good’, there is an inherent risk that the reviews are not entirely accurate or do not capture the most important critiques of the papers — after all, LLMs are engineered to give good-looking answers, not necessarily the correct answers.

“There is also a large variance in how different authors have utilised AI in their papers. While the conference FAQs say ‘The AI agent should be the primary contributor’, in some of the accepted papers, the authors state that AI was only used to assist in experiment design and for grammar checking of the paper. This is a bit disingenuous and risks making AI agents look much more impressive than they really are at first glance.

“The use of AI in research is a rapidly evolving topic. Many academics believe that AI can never replace human researchers, as current AI models struggle to work ‘outside of distribution’, i.e. it is hard for them to propose new ideas that extend human knowledge in innovative ways. Other researchers believe AI has a lot to offer, as there are still a lot of unexplored research ideas within our current knowledge paradigms. Either way, it is important to remember that research happens within a broader human context — even in fundamental research, we must keep in mind why we are doing the research and for whom. I can’t see how AI will replace this role anytime soon.”

Conflict of interest statement: “In addition to my substantive academic role, I am co-director of LensenMcGavin AI.”

Dr Simon McCallum, Senior Lecturer in Software Engineering, Victoria University of Wellington, comments:

“There is a steady increase of AI generated content on the internet. Recently we have moved to more than 50% of the articles on websites being over 50% AI generated according to a white paper from an SEO agency (also, it’s research only made possible by using an AI detector, which uses AI to detect AI). This new conference with explicitly AI generated content is at least honest. Whenever you add incentives to create an artifact, people will find ways to have the result without putting in the work. Academia has been awash with incentives and unintended consequences for decades.

“If we only care about the knowledge being created and tested, then AI has indeed contributed to Maths and Science significantly, however, if we care both about the outcome and the people who are part of the process, then knowledge development untethered to human development is problematic. Once we no longer need humans as part of the knowledge development process, why would humans need to read or understand the content created? We write academic papers as a means of communication. It is a link between researchers, a sharing of both knowledge and inspiration. Watching a weight-lifter can be inspirational, watching a forklift move the same weight is expected. AI is already developing knowledge and contributing to the rapid development of science. If we merely care about getting the answers right then the ability for AI to evaluate and fact check a statement is now well beyond the capability of individuals. AIs are better at predicting outcomes than humans and have the unblinking ability to monitor systems.

“What this suggests is that we should not argue that humans are important because of our concerns over quality or capability. That will be a losing battle. We need to value conferences not because they are the best way to transmit information, (I would actually use an em-dash here but that would make people think this was AI written) they have not been that since the internet was created, but because they are a format that allows human connection. We need to lean into the human aspect of knowledge and connection. We need to value conferences as places to inspire and hold accountable researchers.

“My daughter is among the Gen Zee that hate AI Art. In fact she, and the fan groups she is part of, do not even acknowledge AI content as art. AI images exist, but art is a connection between the viewer and the creator. Modern media has broken the fourth wall, and the creators of shows like Arcane, engage with their community. Game developers actively build a community as they develop their games. If all we care about is the quality of the artifact then my grandmother’s knitted jersey would be thrown away as the machines can make a better artifact. It is our connections that are critical to our retaining value in a world where any artifact can be made faster, better, and stronger by AI.

“I have read some of the articles, and indeed there is interesting content in some of them, but calling it a conference is a bit like playing cards against the computer, I am missing the conversations.”

Conflict of Interest statement: “I am lecturing at Victoria University of Wellington and co-founder of an AI in Healthcare company. I am helping with AI policy for the Labour Party.”

© Scoop Media