• AI coach transforms peer reviews into more constructive, less toxic feedback. • Stanford researchers trained LLMs on curated reviews flagged as vague or unprofessional. • The Review Feedback Agent uses five LLMs to cross-check and refine responses. • Early trials before ICML 2025 involved over 10,000 conference submissions. • Authors note 12.9% of reviews were poor quality due to vague comments. • Impact on overall research quality remains to be evaluated by future studies.
Article Summaries:
- A new study shows that an artificial‑intelligence coach can help peer reviewers write more constructive, less toxic feedback. Stanford computer scientist James Zou and colleagues trained a “Review Feedback Agent” that uses five large language models to analyze and refine reviewer comments. The tool was tested on about 20,000 reviews submitted for the 2025 International Conference on Learning Representations, where it suggested more specific, actionable language and flagged vague or unprofessional remarks. While the AI improved the tone and clarity of many reviews, the researchers note that it is still unclear whether these changes strengthen the quality of the papers being evaluated.
Sources: