AI (or rather LLM’s) use is becoming more & more common and one of the issues with it (amongst many) is that when you ask it to summarize/process/or whatever something you have to upload that content to servers that are outside your control, then to make things worse a lot of these services use the data uploaded to further train their models. It is not illegal because they have put it in the user agreement that you allow them to use the data in anyway they want. Once the model is trained on data you have uploaded then it is possible for others to extract this data with a specific prompt. This happened to Samsung back in 2023 where an engineer uploaded internal source code to ChatGPT and another user was able to download it.
So the stakes are quite high when scientists are asked to peer-review papers that can contain significant breakthrough’s. Secondly a good peer review evaluates a paper and a lot of times identify issues that the author needs to address before publishing. Using AI for this can significantly degrade the quality of the review and this was identified as an issue by multiple publications.
The 40th Annual Conference on Neural Information Processing Systems (NeurIPS) has strict rules banning peer reviewers from uploading papers they are reviewing to AI chatbots but they are allowed to use AI chatbots for background research. This is pretty normal nowadays but they went a step further to enforce this policy and catch illicit AI use by deliberately concealing instructions for large language models (LLMs) in papers sent out for peer review. These instructions instruct LLM’s to use specific phrases in the review report allowing the organizers to identify the reviewers who breached policy.
The instructions tell an LLM to use telltale phrases—such as “This work addresses the central challenge” and “The claims of the paper”—in a peer-review report. Some researchers have already been caught trying to sneak secret messages into their papers in a bid to game AI tools into giving them favorable referee reports. Many publishers ban the use of AI in peer review.
Most people are in favor of this action but there are some who have been pretty vocal about not liking this action. They claim that “Designing a trap that presumes bad faith corrodes the relationship the whole system depends on,” and “You do not build a healthy reviewing culture by treating your reviewers as suspects.” I have a feeling that folks in this group are the ones who were previously using LLM’s in their review and are not happy about being caught. Is this a perfect method to catch folks who are cheating? Of-course not, but it is better than what we had earlier.
Some of the justified criticism against this method is coming from reviewers who found the embedded prompts in the paper and assumed that they were put in by the paper authors (there have been multiple cases where authors put in hidden prompts in their papers to give them favorable reviews).
Source: Scientists decry conference’s use of hidden prompts to snare AI peer reviews
– Suramya