Can AI Revise Research Papers with Human Review Feedback? An Empirical Study and Benchmark (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are fundamentally reshaping the scientific landscape, transitioning the role of AI from passive tools to active partners within a new paradigm of Human-AI collaboration. |
| Approach: | They propose a benchmark to evaluate the ability of Large Language Models to improve papers with human feedback. |
| Outcome: | The proposed benchmark tests the skills of Large Language Models (LLMs) on paper interpretation, experimental implementation, and paper formulation, using authors’ camera-ready versions as natural human baselines. |
Similar Papers
XtraGPT: Context-Aware and Controllable Academic Paper Revision via Human-AI Collaboration (2026.acl-long)
Copied to clipboard
Nuo Chen, Andre Lin HuiKai, Jiaying Wu, Junyi Hou, Zining Zhang, Qian Wang, Xidong Wang, Bingsheng He
| Challenge: | Existing systems are designed for general-purpose scientific text generation and fail to support high-quality scientific writing beyond surface-level polishing. |
| Approach: | They propose a human-AI collaboration framework for academic paper revision based on criteria-guided intent alignment and context-aware modeling. |
| Outcome: | The proposed framework outperforms existing LLMs and rivals the quality of proprietary ones. |
DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Process (2025.acl-long)
Copied to clipboard
| Challenge: | Existing Large Language Models (LLMs) face limited domain expertise, hallucinated reasoning, and a lack of structured evaluation. |
| Approach: | They propose a multi-stage framework to emulate expert reviewers by incorporating structured analysis, literature retrieval, and evidence-based argumentation. |
| Outcome: | The proposed model outperforms CycleReviewer-70B with fewer tokens and achieves 88.21% and 80.20% win rates. |
RExBench: Can coding agents autonomously implement AI research extensions? (2026.acl-long)
Copied to clipboard
| Challenge: | Existing large language model (LLM) agents are not capable of performing research extension tasks autonomously. |
| Approach: | They propose a benchmark to evaluate LLM agents' ability to extend existing AI research . they use extensions of 12 recently published research papers accompanied by domain expert-written instructions . |
| Outcome: | The proposed benchmark evaluates 12 LLM agents implemented using aider and OpenHands. |
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)
Copied to clipboard
Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
REVISE: A Framework for Revising OCRed text in Practical Information Systems with Data Contamination Strategy (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing Document AI frameworks lack the capability to structurally organize and manage document information. |
| Approach: | They propose a framework that corrects OCR errors at the character, word, and structural levels and a synthetic data generation strategy that realistically simulates such errors to train an effective correction model. |
| Outcome: | The proposed framework improves document retrieval and question answering tasks by correcting errors introduced by OCR errors at the character, word, and structural levels. |
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (2026.acl-short)
Copied to clipboard
| Challenge: | ACL is 30+ times larger than two decades ago, and we face issues such as overwhelming participants, outdated papers, and low quality review. |
| Approach: | aaron carroll: ACL has become 30+ times larger than two decades ago . he says increasing research in LLM, AI accelerating research can help . carroll will share some of his recent work on AI review automation, paper recommendation, and AI arXiv . |
| Outcome: | aaron e. muller: ACL has become 30+ times larger than two decades ago . he says recent work on AI review automation, paper recommendation, and arXiv is promising . |
Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation (D18-1)
Copied to clipboard
| Challenge: | Recent research suggests that neural machine translation achieves parity with professional human translation on the WMT Chinese–English news translation task. |
| Approach: | They empirically test neural machine translation on a Chinese–English news translation task . they show human raters prefer human over machine translation when evaluating documents . |
| Outcome: | The proposed method shows that human translators prefer document-level evaluation over machine translation . the results highlight the need to shift towards document- level evaluation as machine translation improves . |
Position Paper: How Should We Responsibly Adopt LLMs in the Peer Review Process? (2026.findings-eacl)
Copied to clipboard
| Challenge: | a recent paper criticizes the current use of Large Language Models (LLMs) for simple review text generation. |
| Approach: | They propose to use Large Language Models to support key aspects of the review process . they argue that this approach overlooks more meaningful applications of LLMs . authors argue that the increased reviewing burden per reviewer is a factor . |
| Outcome: | The proposed approach would support reproducibility, correctness and relevance of citations and ethics review flagging. |
RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension (2026.acl-long)
Copied to clipboard
Yelin Chen, Fanjin Zhang, Suping Sun, Yunhe Pang, Yuanchun Wang, Jian Song, XiaoYan Li, Lei Hou, Shu Zhao, Jie Tang, Juanzi Li
| Challenge: | Existing benchmarks for understanding research papers offer limited fine-grained evaluation at scale. |
| Approach: | They propose a large-scale question-answering benchmark built from review–rebuttal exchanges of high-quality computer science papers. |
| Outcome: | The proposed model is based on human-verified QA pairs and contains 15K questions. |
From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systems (2025.findings-emnlp)
Copied to clipboard
Zekun Zhou, Xiaocheng Feng, Lei Huang, Xiachong Feng, Ziyun Song, Ruihan Chen, Liang Zhao, Weitao Ma, Yuxuan Gu, Baoxin Wang, Dayong Wu, Guoping Hu, Ting Liu, Bing Qin
| Challenge: | rapid development of artificial intelligence (AI) technologies has inspired researchers to explore how AI can accelerate and enhance research. |
| Approach: | They organize the relevant studies into three main categories: hypothesis formulation, hypothesis validation, and manuscript publication. |
| Outcome: | The authors summarize the current state of research in three main areas: hypothesis formulation, hypothesis validation, and manuscript publication. |