Using present day / nascent LLMs for academic peer reviewing?

Question

The current academic publication model has several problems, including unethical fees and no payment for either the author or the reviewer (see ongoing discussion at https://news.ycombinator.com/item?id=35848894). It appears that there is no good way of getting out of the trap, because: 1. Researchers rely on being recognized as published and having their papers cited for promotions.So, open platforms like arxiv exist to publish to the masses. However...2. Readers rely on quality criteria to sift through the thousands of articles out there. Papers in highly reputed journals and conference proceedings have already gone through a pass through expert reviews.What does HN think the likelihood is of us getting an LLM that might be able to assuage this issue by providing a good, robust review judging by research quality standards? Is anybody already working on this?

speedgoose · Accepted Answer

The point of the peer review process is to get your work reviewed by peers.However LLM could be used to filter garbage papers, to help designing a research study, to help writing, or to simulate a review before submitting.

wnkrshm · Answer

It could set a target to optimize against - and automatically optimize against - which may not be a good thing at all.

jruohonen · Answer

Please no.