| Challenge: | Existing approaches to reviewer-paper matching are less effective to deal with the vocabulary mismatch and partial topic overlap between the submission and reviewer. |
| Approach: | They propose to combine the common topic model and abstract topic vectors to model the topics common to the submission and the reviewer's profile while relying on abstract topic vectors. |
| Outcome: | The proposed model improves on the existing model on two datasets. |
Similar Papers
What Factors Should Paper-Reviewer Assignments Rely On? Community Perspectives on Issues and Ideals in Conference Peer-Review (2022.naacl-main)
Copied to clipboard
| Challenge: | a survey of the NLP community shows that paper-reviewer matching is a problem . authors lose valuable time and opportunities by writing reviews that are arbitrarily low . |
| Approach: | They propose to use paper-reviewer matching to improve peer review . they identify common issues and perspectives on what factors should be considered . |
| Outcome: | The proposed method improves the quality of peer review and improves interpretable peer review assignments. |
Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysis (2024.findings-emnlp)
Copied to clipboard
Jianxiang Yu, Zichen Ding, Jiaqi Tan, Kangyang Luo, Zhenmin Weng, Chenghua Gong, Long Zeng, RenJing Cui, Chengcheng Han, Qiushi Sun, Zhiyong Wu, Yunshi Lan, Xiang Li
| Challenge: | Existing approaches to review scientific papers are limited by their content or quality . SEA is a framework for automated scientific review, but its contents are generic or partial. |
| Approach: | They propose a framework for automated scientific review using large language models . they propose to use a standardized review dataset to fine-tune an LLM to generate high-quality reviews. |
| Outcome: | The proposed framework can generate high-quality reviews from standardized datasets and improves on the existing feedback mechanisms. |
What Can We Do to Improve Peer Review in NLP? (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Traditionally, peer review is expected to act as a filter for high-quality, impactful work, but this does not hold in practice. |
| Approach: | They argue that peer review is becoming increasingly spurious and that it is a problem for NLP . they propose a reproducibility checklist at EMNLP 2020 that could be used to ensure that papers are reproducible. |
| Outcome: | The reproducibility checklist at EMNLP 2020 is the first step in that direction. |
From Isolated Scoring to Collaborative Ranking: A Comparison-Native Framework for LLM-Based Paper Evaluation (2026.findings-acl)
Copied to clipboard
Pujun Zheng, Jiacheng Yao, Jinquan Zheng, Chenyang Gu, Guoxiu He, Jiawei Liu, Yong Huang, Tianrui Guo, Wei Lu
| Challenge: | Large language models (LLMs) are currently used to evaluate scientific papers by assigning an absolute score to each paper independently. |
| Approach: | They propose a comparison-native framework for paper evaluation that integrates comparison into both data construction and model learning. |
| Outcome: | The proposed framework achieves an average relative improvement of 21.8% over the strong baseline DeepReview-14B, while exhibiting robust generalization to five previously unseen datasets. |
Autonomous Machine Learning-Based Peer Reviewer Selection System (2025.coling-demos)
Copied to clipboard
| Challenge: | Existing systems that match papers with experts are inefficient and often require long turnaround times. |
| Approach: | They propose an autonomous peer reviewer selection system that employs the natural language processing model to match submitted papers with expert reviewers independently of traditional journals and conferences. |
| Outcome: | The proposed system performs faster and smaller than current models while being more scalable. |
Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future (2026.acl-long)
Copied to clipboard
Sihong Wu, Owen Jiang, Yilun Zhao, Tiansheng Hu, Yiling Ma, Kaiyan Zhang, Manasi Patwardhan, Arman Cohan
| Challenge: | Recent advances in large language models (LLMs) motivated methods that assist or automate different stages of peer review pipeline. |
| Approach: | They synthesize techniques to enhance peer review generation and after-review tasks aligned to reviews. |
| Outcome: | The proposed methods improve the peer review process by fine-tuning strategies, agent-based systems, and emerging paradigms. |
Cross-referencing Using Fine-grained Topic Modeling (N19-1)
Copied to clipboard
| Challenge: | Cross-referencing is a useful study aid for facilitating comprehension of a text, but it requires extensive thematic knowledge and a focused search through the corpus to find such useful connections. |
| Approach: | They propose a system for producing candidate cross-references which can be easily verified by human annotators. |
| Outcome: | a new system can produce cross-references that can be easily verified by human annotators . the system uses fine-grained topic modeling to identify verse pairs which are topically related . |
Evaluation and Facilitation of Online Discussions in the LLM Era: A Survey (2025.emnlp-main)
Copied to clipboard
Katerina Korre, Dimitris Tsirmpas, Nikos Gkoumas, Emma Cabalé, Danai Myrtzani, Theodoros Evgeniou, Ion Androutsopoulos, John Pavlopoulos
| Challenge: | Recent advances in LLMs enable artificial facilitation agents to not only moderate content, but also actively improve the quality of interactions. |
| Approach: | They propose a taxonomy on discussion quality evaluation and a new taxonomies for intervention and facilitation strategies. |
| Outcome: | The proposed methods synthesize ideas from Natural Language Processing (NLP) and Social Sciences to provide a taxonomy on discussion quality evaluation, and a roadmap of good practices and future research directions. |
Position Paper: How Should We Responsibly Adopt LLMs in the Peer Review Process? (2026.findings-eacl)
Copied to clipboard
| Challenge: | a recent paper criticizes the current use of Large Language Models (LLMs) for simple review text generation. |
| Approach: | They propose to use Large Language Models to support key aspects of the review process . they argue that this approach overlooks more meaningful applications of LLMs . authors argue that the increased reviewing burden per reviewer is a factor . |
| Outcome: | The proposed approach would support reproducibility, correctness and relevance of citations and ethics review flagging. |
Community Topic: Topic Model Inference by Consecutive Word Community Discovery (2022.coling-1)
Copied to clipboard
| Challenge: | a new topic modelling algorithm is developed to help researchers understand large corpora . community topic can be used to find coherent topics at various scales . |
| Approach: | They propose a topic-modeling algorithm that extracts communities from term co-occurrence networks and compares it with Latent Dirichlet Allocation and top2vec. |
| Outcome: | The proposed algorithm can find coherent topics at various scales. |