Papers by Rong Zhang

Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agents (2025.findings-emnlp)

Challenge:	Existing work on adversarial attack to improve performance of NLSM tasks has not been done.
Approach:	They propose a general two-stage training framework to enhance neural models with Vulnerability via adversarial attack.
Outcome:	The proposed framework improves neural models with Vulnerability via adversarial attack on NLSM datasets.

Copied to clipboard

Long Li, Weiwen Xu, Jiayan Guo, Ruochen Zhao, Xingxuan Li, Yuqian Yuan, Boqiang Zhang, Yuming Jiang, Yifei Xin, Ronghao Dang, Yu Rong, Deli Zhao, Tian Feng, Lidong Bing

Challenge:	Existing methods for idea generation either trivially prompt LLMs or expose LLM to extensive literature without indicating useful information.
Approach:	They propose a chain-of-ideas agent that organizes literature in a chains structure . they propose evaluating idea-generation methods from different perspectives .
Outcome:	The proposed agent outperforms existing methods and matches human quality in idea generation.

Reinforcement Learning with Semantic Rewards Enables Low-Resource Language Expansion without Alignment Tax (2026.findings-acl)

Copied to clipboard

Zeli Su, Ziyin Zhang, Zhou Liu, Xuexian Song, Zhankai Xu, Longfei Zheng, Xiaolu Zhang, Rong Fu, Guixian Xu, Wentao Zhang

Responsible & Trustworthy NLP

Challenge:	Extending large language models to low-resource languages often incurs an "alignment tax" token-level fine-tuning enforces token-level surface imitation on narrow and biased data distributions.
Approach:	They propose a semantic-space alignment paradigm powered by group-level semantic rewards instead of likelihood maximization.
Outcome:	The proposed model acquires low-resource capa- bilities while mitigating alignment tax on Tibetan–Chinese machine translation and Ti- betan headline generation.

Counterfactual Adversarial Learning with Representation Interpolation (2021.findings-emnlp)

Copied to clipboard

Wei Wang, Boxin Wang, Ning Shi, Jinfeng Li, Bingyu Zhu, Xiangyu Liu, Rong Zhang

Attention Mechanism with Energy-Friendly Operations (2022.findings-acl)

Challenge:	Existing models with statistical bias are prone to memorized correlations . large pre-trained models such as BERT have revolutionized the model development paradigm in natural language processing .
Approach:	They propose a framework to tackle the problem from a causal perspective using a latent space interpolation approach.
Outcome:	Extensive experiments show that CAT achieves substantial performance improvement over SOTA across different downstream tasks, including sentence classification, natural language inference and question answering.

Copied to clipboard

Yu Wan, Baosong Yang, Dayiheng Liu, Rong Xiao, Derek Wong, Haibo Zhang, Boxing Chen, Lidia Chao

Challenge:	Empirical results show that attention mechanism can be improved from the energy consumption aspects.
Approach:	They propose to replace multiplications with either selective operations or additions to reduce energy consumption.
Outcome:	The proposed model achieves competitable accuracy while saving 99% and 66% energy during alignment calculation and the whole attention procedure.

Muted: Multilingual Targeted Offensive Speech Identification and Visualization (2023.emnlp-demo)

Copied to clipboard

Christoph Tillmann, Aashka Trivedi, Sara Rosenthal, Santosh Borse, Rong Zhang, Avirup Sil, Bishwaranjan Bhattacharjee

Multimodality

Challenge:	Existing visualizations of offensive language use only sentence level annotations, but there are few that explore spans and other languages.
Approach:	They propose a system to identify multilingual HAP content by displaying offensive arguments and their targets using heat maps to indicate their intensity.
Outcome:	The proposed model can identify toxic spans without further fine-tuning using existing models and its attention mechanism out-of-the-box.

ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning (2025.emnlp-main)

Copied to clipboard

Yu Sun, Xingyu Qian, Weiwen Xu, Hao Zhang, Chenghao Xiao, Long Li, Deli Zhao, Wenbing Huang, Tingyang Xu, Qifeng Bai, Yu Rong

Orthogonal Subspace Learning for Language Model Continual Learning (2023.findings-emnlp)

Challenge:	Existing medical reasoning datasets are limited in scale and typically rely on incomplete data.
Approach:	They propose to use ReasonMed to train medical reasoning models using a multi-agent generation, verification, and refinement pipeline.
Outcome:	The largest medical reasoning dataset to date surpasses the prior best sub-10B models by 4.17% and even exceeds LLaMA3.1-70B on PubMedQA by 4.60%.

Copied to clipboard

Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang

Flooding-X: Improving BERT’s Resistance to Adversarial Attacks via Loss-Restricted Fine-Tuning (2022.acl-long)

Challenge:	Existing methods for continual learning in language models suffer catastrophic forgetting when learning sequential tasks.
Approach:	They propose an orthogonal low-rank adaptation approach for continual learning in language models that uses orthogons to learn sequentially.
Outcome:	The proposed approach outperforms state-of-the-art methods on continual learning benchmarks and preserves generalization ability of LLMs on unseen tasks.

Copied to clipboard

Qin Liu, Rui Zheng, Bao Rong, Jingyi Liu, ZhiHua Liu, Zhanzhan Cheng, Liang Qiao, Tao Gui, Qi Zhang, Xuanjing Huang

Selecting Demonstrations for Many-Shot In-Context Learning via Gradient Matching (2025.findings-acl)

Challenge:	Existing approaches to generating adversarial perturbations scale up the cost of training computational complexity by the number of gradient steps it takes to obtain the adversarials.
Approach:	They propose a flood method which aims at better generalization and a criterion to bring hyper-parameter-dependent flooding into effect with a narrowed-down search space by measuring how the gradient steps taken within one epoch affect the loss of each batch.
Outcome:	The proposed method improves BERT’s resistance to textual adversarial attacks by a large margin and achieves state-of-the-art robust accuracy on various text classification and GLUE tasks.

Copied to clipboard

Jianfei Zhang, Bei Li, Jun Bai, Rumei Li, Yanmeng Wang, Chenghua Lin, Wenge Rong

AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios (2025.naacl-long)

Challenge:	In-Context Learning (ICL) empowers Large Language Models for rapid task adaptation without fine-tuning.
Approach:	They propose a method that aligns fine-tuning gradients between entire training set and selected examples to enable in-context learning and fine-uning.
Outcome:	The proposed method outperforms random selection on large LLMs from 4-shot to 128-shot scenarios across 9 datasets.

Copied to clipboard

Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang, Xiawei Liu, Shiyue Yang, Rong Ye, Lei Chen, Haoyu Kuang, Xuanjing Huang, Zhongyu Wei

Linguistics & Cognitive NLP

Challenge:	Large language models are increasingly employed to empower autonomous agents to simulate human behavior.
Approach:	They propose to evaluate LLM-driven agents through multi-turn interactions using a bottom-up approach to create diverse social scenarios constructed from extensive scripts.
Outcome:	The proposed model evaluates LLM-driven agents through multi-turn interactions emphasizing goal completion and implicit reasoning.

Copied to clipboard

Shiyuan Xiao, Yuanxin Ouyang, Wenge Rong, Jianxin Yang, Zhang Xiong

Information Extraction & Text Mining

Challenge:	Named entity recognition (NER) tasks are a fundamental challenge for name recognition tasks that aim to reduce the boundary error when entities become longer.
Approach:	They propose a similarity based auxiliary classifier which can distinguish entity words from non-entity words by using vectors to indicate tags.
Outcome:	Empirical results show that the proposed classifier can perform better than baseline approaches.

DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answering (2025.acl-long)

Copied to clipboard

Rong Cheng, Jinyi Liu, Yan Zheng, Fei Ni, Jiazhen Du, Hangyu Mao, Fuzheng Zhang, Bo Wang, Jianye Hao

Natural Language Interfaces

Challenge:	Existing approaches to multi-hop question answering struggle to identify and organize dynamic knowledge . et al., 2023; Liu e.t. al. 2023) suggest a dual-process framework for multi-step reasoning .
Approach:	They propose a synergistic dual-process framework that integrates reasoning and retrieval.
Outcome:	The proposed framework improves answer accuracy and coherence even in smaller-scale models.

Token-Level Self-Evolution Training for Sequence-to-Sequence Learning (2023.acl-short)

Copied to clipboard

Keqin Peng, Liang Ding, Qihuang Zhong, Yuanxin Ouyang, Wenge Rong, Zhang Xiong, Dacheng Tao

Machine Translation

Challenge:	Adaptive training approaches do not consider the variation of learning difficulty in different training steps, making the learning deterministic and sub-optimal.
Approach:	They propose a dynamic token-level self-evolution training method that reweighs the training losses of different target tokens based on priors.
Outcome:	Empirically, the proposed method yields significant improvements on three translation tasks.

PrimeQA: The Prime Repository for State-of-the-Art Multilingual Question Answering Research and Development (2023.acl-demo)

Copied to clipboard

Avi Sil, Jaydeep Sen, Bhavani Iyer, Martin Franz, Kshitij Fadnis, Mihaela Bornea, Sara Rosenthal, Scott McCarley, Rong Zhang, Vishwajeet Kumar, Yulong Li, Md Arafat Sultan, Riyaz Bhat, Juergen Bross, Radu Florian, Salim Roukos

Question Answering

Challenge:	Question Answering (QA) is a major area of research in Natural Language Processing (NLP)
Approach:	They propose a one-stop and open-source QA repository for question answering . it supports core QA functionalities like retrieval and reading comprehension . they say it will facilitate easy replication of state-of-the-art (SOTA) QA methods .
Outcome:	The proposed framework enables easy replication of state-of-the-art (SOTA) QA methods.

Analyzing LLMs’ Knowledge Boundary Cognition Across Languages Through the Lens of Internal Representations (2025.acl-long)

Copied to clipboard

Chenghao Xiao, Hou Pong Chan, Hao Zhang, Mahani Aljunied, Lidong Bing, Noura Al Moubayed, Yu Rong

SpanMlt: A Span-based Multi-Task Learning Framework for Pair-wise Aspect and Opinion Terms Extraction (2020.acl-main)

Challenge:	Understanding the knowledge boundaries of Large Language Models (LLMs) is crucial to prevent hallucination, but research on the knowledge boundary perceptions of LLMs has predominantly focused on English.
Approach:	They propose a training-free alignment method that effectively transfers knowledge boundary perception ability across languages, thereby helping reduce hallucination risk in low-resource languages.
Outcome:	The proposed method reduces hallucination risk in low-resource languages by fine-tuning on bilingual question pair translation.

Copied to clipboard

He Zhao, Longtao Huang, Rong Zhang, Quan Lu, Hui Xue

Sentiment Analysis

Challenge:	Aspect terms and opinion terms are key problems of fine-grained aspect-based sentiment analysis.
Approach:	They propose a method to extract aspect and opinion terms as pairs from a sentence . they use shared spans to extract the terms under supervision of span boundaries .
Outcome:	The proposed method outperforms state-of-the-art methods on both aspects and opinion terms extraction tasks.

Answer Span Correction in Machine Reading Comprehension (2020.findings-emnlp)

Copied to clipboard

Revanth Gangi Reddy, Md Arafat Sultan, Efsun Sarioglu Kayi, Rong Zhang, Vittorio Castelli, Avi Sil

Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment (2025.coling-main)

Challenge:	Existing MRC systems produce only partially correct answers when presented with answerable questions.
Approach:	They propose a method that yields statistically significant performance improvements over existing MRC systems.
Outcome:	The proposed method yields statistically significant performance improvements over state-of-the-art MRC systems in monolingual and multilingual evaluation.

Copied to clipboard

Jianfei Zhang, Jun Bai, Bei Li, Yanmeng Wang, Rumei Li, Chenghua Lin, Wenge Rong

EpLSA: Synergy of Expert-prefix Mixtures and Task-Oriented Latent Space Adaptation for Diverse Generative Reasoning (2024.lrec-main)

Challenge:	Human values are inherently diverse, making it insufficient to align LLMs solely with general preferences.
Approach:	They propose a flexible paradigm for individual preference alignment that disentangles preference representation from text generation in LLMs.
Outcome:	The proposed method produces aligned quality and better than PEFT-based methods while reducing training time for each new individual preference by 80% to 90%.

Copied to clipboard

Fujun Zhang, Xiangdong Su, Jiang Li, Rong Yan, Guanglai Gao

Disentangling Language and Culture for Evaluating Multilingual Large Language Models (2025.acl-long)

Challenge:	Existing models for diverse generative reasoning struggle to generate multiple unique and plausible results.
Approach:	They propose a model based on expert-prefix mixtures and task-oriented latent space adaptation for diverse generative reasoning.
Outcome:	The proposed model outperforms existing models on three types of generative reasoning tasks.

Copied to clipboard

Jiahao Ying, Wei Tang, Yiran Zhao, Yixin Cao, Yu Rong, Wenxuan Zhang

FocusLLM: Precise Understanding of Long Context by Dynamic Condensing (2025.acl-long)

Challenge:	Extensive evaluations of large language models (LLMs) are conducted on a wide range of models, revealing a notable cultural-linguistic synergy phenomenon, where models exhibit better performance when questions are culturally aligned with the language.
Approach:	They propose a Dual Evaluation Framework to comprehensively assess the multilingual capabilities of large language models by decomposing evaluation along dimensions of linguistic medium and cultural context.
Outcome:	The proposed framework allows for a nuanced analysis of LLMs’ ability to process questions within both native and cross-cultural contexts cross-lingually.

Copied to clipboard

Zhenyu Li, Yike Zhang, Tengyu Pan, Yutao Sun, Zhichao Duan, Junjie Fang, Rong Han, Zixuan Wang, Jianyong Wang

Challenge:	Existing context condensing methods cannot accurately understand the full context, as there is a considerable amount of information loss in the condensed process.
Approach:	They propose a framework to extend the fixed context length of any decoder-only LLM by distilling crucial information from long sequences.
Outcome:	The proposed framework extends the fixed context length of any decoder-only LLM, allowing it to focus on relevant information from very long sequences.

How to Determine the Most Powerful Pre-trained Language Model without Brute Force Fine-tuning? An Empirical Survey (2023.findings-emnlp)

Copied to clipboard

Jun Bai, Xiaofeng Zhang, Chen Li, Hanhua Hong, Xi Xu, Chenghua Lin, Wenge Rong

PlugAT: A Plug and Play Module to Defend against Textual Adversarial Attack (2022.coling-1)

Challenge:	Transferability estimation has been a topic of great interest in computer vision fields . a lack of a comprehensive comparison between these estimation methods is a problem .
Approach:	They conduct a thorough survey of existing methods to find the most suitable model . they also outline difficulties of consideration of training details and applicability to text generation .
Outcome:	The proposed methods perform well with superiorities in effectiveness and efficiency.

Copied to clipboard

Rui Zheng, Rong Bao, Qin Liu, Tao Gui, Qi Zhang, Xuanjing Huang, Rui Xie, Wei Wu

Hi-ArG: Exploring the Integration of Hierarchical Argumentation Graphs in Language Pretraining (2023.emnlp-main)

Challenge:	Recent studies have shown that adversarial examples can be easily fooled by adversarially perturbed examples.
Approach:	They propose a pluggable defense module PlugAT to provide robust predictions by adding a few trainable parameters to the model inputs while keeping the original model frozen.
Outcome:	The proposed model improves robustness over several strong baselines whilst training only 9.1% parameters.

Copied to clipboard

Jingcong Liang, Rong Ye, Meng Han, Qi Zhang, Ruofei Lai, Xinyu Zhang, Zhao Cao, Xuanjing Huang, Zhongyu Wei

ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods (2024.emnlp-main)

Challenge:	Recent studies have discussed its capability to assist language models for various applications.
Approach:	They propose a structure to organize arguments using the Hierarchical Argumentation Graph (Hi-ArG) and propose two approaches to exploit Hi-AarG, including a text-graph multi-modal model GreaseArR and a framework augmented with graph information.
Outcome:	The proposed structure supersedes existing language models on two argumentation tasks while incorporating graph information during further training improves vanilla language models.

Copied to clipboard

Roy Xie, Junlin Wang, Ruomin Huang, Minxing Zhang, Rong Ge, Jian Pei, Neil Gong, Bhuwan Dhingra

Robust Lottery Tickets for Pre-trained Language Models (2022.acl-long)

Challenge:	ReCaLL (Relative Conditional Log-Likelihood) is a membership inference attack that can detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities.
Approach:	They propose a membership inference attack to detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities.
Outcome:	The proposed model achieves state-of-the-art performance on the WikiMIA dataset, even with random and synthetic prefixes, and can be further improved using an ensemble approach.

Copied to clipboard

Rui Zheng, Bao Rong, Yuhao Zhou, Di Liang, Sirui Wang, Wei Wu, Tao Gui, Qi Zhang, Xuanjing Huang

Focusing, Bridging and Prompting for Few-shot Nested Named Entity Recognition (2023.findings-acl)

Challenge:	Recent studies have shown that pre-trained language models contain smaller matching subnetworks that are not robust to adversarial examples.
Approach:	They propose a method to find robust tickets hidden in pre-trained language models by learning binary weight masks and an adversarial loss objective to guide the search.
Outcome:	The proposed method improves on previous work on adversarial robustness evaluation.

Copied to clipboard

Yuanyuan Xu, Zeng Yang, Linhai Zhang, Deyu Zhou, Tiandeng Wu, Rong Zhou

Low-Resource NLP

Challenge:	Existing work on few-shot named entity recognition (NER) addresses flat entities instead of nested entities.
Approach:	They propose a method based on focusing, bridging and prompting for few-shot nested NER without using source domain data.
Outcome:	The proposed method outperforms baseline models on four benchmark datasets and outperformed several competing models on F1-score by 9.33% on ACE2004, 6.17% on ace2005, 9.40% on GENIA and 5.12% on KBP2017.

CASN:Class-Aware Score Network for Textual Adversarial Detection (2023.acl-long)

Copied to clipboard

Rong Bao, Rui Zheng, Liang Ding, Qi Zhang, Dacheng Tao

ReasonRec: A Reasoning-Augmented Multimodal Agent for Unified Recommendation (2026.findings-acl)

Challenge:	Existing approaches to counteract adversarial attacks can be divided into two directions, adversarials defense and adversarially detection.
Approach:	They propose a score-based generative method to implicitly model the data distribution using a log-density distribution and supervised contrastive learning to guide the estimation using label information.
Outcome:	The proposed method improves on three text classification tasks on four advanced attack algorithms.

Copied to clipboard

Yihua Zhang, Mingfu Liang, Jiyan Yang, Rong Jin, Wen-Yen Chen, Yiping Han, Huayu Li, Buyun Zhang, Liang Luo, Luke Simon, Sijia Liu, Tianlong Chen, Xi Liu

Responsible & Trustworthy NLP

Challenge:	Recent advances in multimodal recommenders lack explicit reasoning and self-awareness of uncertainty.
Approach:	They propose a reasoning-augmented multimodal agent structured around a three-stage explicit reasoning pipeline.
Outcome:	The proposed agent improves ranking metrics and performance on four standard recommendation tasks across five real-world datasets.

FineReason: Evaluating and Improving LLMs’ Deliberate Reasoning through Reflective Puzzle Solving (2025.acl-long)

Copied to clipboard

Guizhen Chen, Weiwen Xu, Hao Zhang, Hou Pong Chan, Chaoqun Liu, Lidong Bing, Deli Zhao, Anh Tuan Luu, Yu Rong

Leveraging Estimated Transferability Over Human Intuition for Model Selection in Text Ranking (2024.emnlp-main)

Challenge:	Recent advances in large language models (LLMs) highlight an important shift from the “System 1” way of quick reactions to the “system 2” style of reflection-and-correction problem solving.
Approach:	They propose a logic-puzzle benchmark for systematic evaluation of large language models' reasoning capabilities that decomposes each puzzle into atomic steps.
Outcome:	The proposed model improves on state checking and state transition tasks and demonstrates gains in reasoning by up to 5.1%.

Copied to clipboard

Jun Bai, Zhuofan Chen, Zhenzi Li, Hanhua Hong, Jianfei Zhang, Chen Li, Chenghua Lin, Wenge Rong

Information Extraction & Text Mining

Challenge:	Existing methods for text ranking are based on intuition, but their estimated transferability may not align well with the objectives of text ranking.
Approach:	They propose to compute expected rank as transferability, explicitly reflecting the model’s ranking capability.
Outcome:	The proposed method shows significant improvements over previous classification-oriented TE methods, human intuition, and ChatGPT with minor time consumption.

ProCQA: A Large-scale Community-based Programming Question Answering Dataset for Code Search (2024.lrec-main)

Copied to clipboard

Zehan Li, Jianfei Zhang, Chuantao Yin, Yuanxin Ouyang, Wenge Rong

DUB: Discrete Unit Back-translation for Speech Translation (2023.findings-acl)

Challenge:	Existing approaches to code question answering use bi-modal and unimodal pretraining to align text and code representations.
Approach:	They propose a modality-agnostic contrastive pre-training approach to improve alignment of text and code representations of current code language models.
Outcome:	The proposed model exhibits significant performance improvements across a wide range of code retrieval benchmarks.

Copied to clipboard

Dong Zhang, Rong Ye, Tom Ko, Mingxuan Wang, Yaqian Zhou

Speech & Audio in NLP

Challenge:	Discrete unit back-translation (DUB) is a back-translated speech-to-text translation (ST) technique that can be applied to ST . a modality gap between speech and text makes it difficult to transfer these techniques to ST due to the modality of the speech-text model.
Approach:	They propose a method to represent speech with discrete units instead of continuous features in direct ST.
Outcome:	The proposed method achieves comparable performance to existing methods that rely on large-scale external data.

RoChBert: Towards Robust BERT Fine-tuning for Chinese (2022.findings-emnlp)

Copied to clipboard

Zihan Zhang, Jinfeng Li, Ning Shi, Bo Yuan, Xiangyu Liu, Rong Zhang, Hui Xue, Donghong Sun, Chao Zhang

GeoPQA: Bridging the Visual Perception Gap in MLLMs for Geometric Reasoning (2025.findings-emnlp)

Challenge:	Pre-trained language models (e.g., BERT) have been proved vulnerable to adversarial texts.
Approach:	They propose to fuse Chinese phonetic and glyph features into pre-trained models by using a more comprehensive adversarial graph.
Outcome:	The proposed framework outperforms existing methods in significant ways on a wide range of tasks while remaining accurate on benign texts.

Copied to clipboard

Guizhen Chen, Weiwen Xu, Hao Zhang, Hou Pong Chan, Deli Zhao, Anh Tuan Luu, Yu Rong

Multimodality

Challenge:	Recent advances in reinforcement learning (RL) have enhanced the reasoning abilities of large language models, but the impact on multimodal LLMs is limited.
Approach:	They propose a two-stage RL framework that enhances visual perception and fosters reasoning capabilities.
Outcome:	The proposed framework improves geometric reasoning by 9.7% and problem-solving by 9.1% compared to direct reasoning training approach.

Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLM (2024.findings-acl)

Copied to clipboard

Jingcong Liang, Rong Ye, Meng Han, Ruofei Lai, Xinyu Zhang, Xuanjing Huang, Zhongyu Wei

Challenge:	Recent studies have focused on short dialogues, but mainly on short debates.
Approach:	They propose to use Large Language Models to construct an automated debate judge to evaluate multi-turn debates.
Outcome:	The proposed system improves on the PanelBench benchmark, which compares its performance to actual debate outcomes.

Sequential Attention with Keyword Mask Model for Community-based Question Answering (N19-1)

Copied to clipboard

Jianxin Yang, Wenge Rong, Libin Shi, Zhang Xiong

Natural Language Interfaces

Challenge:	Existing methods to model answer selection(AS) are based on feature engineering and resource toolkits.
Approach:	They propose a model that captures features and information from question and answer text and repeats multiple times(hops) in a sequential fashion.
Outcome:	The proposed model performs on answer selection tasks and multi-level answer ranking tasks.

Mixture of Attention Heads: Selecting Attention Heads Per Token (2022.emnlp-main)

Copied to clipboard

Xiaofeng Zhang, Yikang Shen, Zeyu Huang, Jie Zhou, Wenge Rong, Zhang Xiong

Responsible & Trustworthy NLP

Challenge:	Mixture-of-Experts (MoE) networks have been proposed as an efficient way to scale up model capacity and implement conditional computing.
Approach:	They propose a new architecture that combines multi-head attention with the MoE mechanism and a sparsely gated architecture that allows for faster computations.
Outcome:	The proposed architecture can scale up the number of attention heads and the number parameters while preserving computational efficiency.

A Survey of Large Language Model-Based Search Agents (2026.acl-long)

Copied to clipboard

Yunjia Xi, Jianghao Lin, Yongzhao Xiao, Zheli Zhou, Rong Shan, Te Gao, Jiachen Zhu, Weiwen Liu, Yong Yu, Weinan Zhang

Enhancing Dual-Encoders with Question and Answer Cross-Embeddings for Answer Retrieval (2021.findings-emnlp)

Challenge:	Large Language Models (LLMs) have revolutionized web search, but their integration is static and cannot handle complex contexts.
Approach:	They analyze existing research and analyze existing work from the perspectives of architecture, optimization, application, and evaluation.
Outcome:	The proposed models can comprehend user intentions and context and execute multi-turn retrieval with dynamic planning, extending search capabilities far beyond the web.

Copied to clipboard

Yanmeng Wang, Jun Bai, Ye Wang, Jianfei Zhang, Wenge Rong, Zongcheng Ji, Shaojun Wang, Jing Xiao

A Comprehensive Survey of Process Reward Models: Data Generation, Model Construction, and Usage (2026.acl-long)

Challenge:	Existing approaches to solve question answering (QA) problems are limited by the need for text generation and answer retrieval.
Approach:	They propose to introduce QA interaction features in scoring function but at the cost of low efficiency in inference stage.
Outcome:	The proposed framework significantly outperforms the state-of-the-art method on multiple answer retrieval datasets.

Copied to clipboard

Congmin Zheng, Jiachen Zhu, Zhuoying Ou, Yuxiang Chen, Kangning Zhang, Rong Shan, Zeyu Zheng, Mengyue Yang, Jianghao Lin, Yong Yu, Weinan Zhang

JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models (2026.findings-acl)

Challenge:	Large Language Models (LLMs) have advanced reasoning ability, yet conventional alignment remains dominated by outcome reward models that judge only final answers.
Approach:	They summarize applications across math, code, text, multimodal reasoning, robotics, and agents . goal is to clarify design spaces, reveal open challenges, and guide future research toward fine-grained, robust reasoning alignment.
Outcome:	The proposed model enables finer credit assignment, richer diagnostics, and improved robustness.

Copied to clipboard

Qingjia Huang, Jingyu Zhang, Jianguo Wu, Yakai Li, Weijuan Zhang, Yankai Rong, Junyi Yao, Shengzhi Zhang, Xiaoqi Jia

INDUS: Effective and Efficient Language Models for Scientific Applications (2024.emnlp-industry)

Challenge:	Currently, evaluation criteria and methods used for jailbreak effectiveness are inconsistent.
Approach:	They propose a framework to measure jailbreak effectiveness using a model that filters out jailbreak noise while preserving the original malicious question.
Outcome:	The proposed framework outperforms existing evaluation methods on a challenging benchmark containing 330 human-labeled, non-rejected jailbreak instances.

Copied to clipboard

Bishwaranjan Bhattacharjee, Aashka Trivedi, Masayasu Muraoka, Muthukumaran Ramasubramanian, Takuma Udagawa, Iksha Gurung, Nishan Pantha, Rong Zhang, Bharath Dandala, Rahul Ramachandran, Manil Maskey, Kaylin Bugbee, Michael Little, Elizabeth Fancher, Irina Gerasimov, Armin Mehrabian, Lauren Sanders, Sylvain Costes, Sergi Blanco-Cuaresma, Kelly Lockhart, Thomas Allen, Felix Grezes, Megan Ansdell, Alberto Accomazzi, Yousef El-Kurdi, Davis Wertheimer, Birgit Pfitzmann, Cesar Berrospi Ramis, Michele Dolfi, Rafael Lima, Panagiotis Vagenas, S. Mukkavilli, Peter Staar, Sanaz Vahidinia, Ryan McGranaghan, Tsengdar Lee

Multi-Stage Pre-training for Low-Resource Domain Adaptation (2020.emnlp-main)

Challenge:	Large language models trained on general domain corpora showed remarkable results on natural language processing tasks.
Approach:	They develop a suite of large language models trained on general domain corpora that address NLP tasks and smaller versions of them created using knowledge distillation.
Outcome:	The proposed models outperform general-purpose and domain-specific encoders on new and existing tasks and in industrial settings.

Copied to clipboard

Rong Zhang, Revanth Gangi Reddy, Md Arafat Sultan, Vittorio Castelli, Anthony Ferritto, Radu Florian, Efsun Sarioglu Kayi, Salim Roukos, Avi Sil, Todd Ward

Scaling Law for Multimodal Large Language Model Supervised Fine-Tuning (2026.acl-long)

Challenge:	Existing approaches to transfer learning target data to in-domain text . prior work has adapted pre-trained LMs to specific domains .
Approach:	They extend the vocabulary of a pretrained language model with domain-specific terms to create synthetic tasks that help it transfer to downstream tasks.
Outcome:	The proposed approaches show significant performance gains on extractive reading comprehension, document ranking and duplicate question detection tasks.

Copied to clipboard

YiFan Zhang, Tao Yu, Feng Li, Chaoyou Fu, Yibo Hu, Kun Wang, Qingsong Wen, Zhang Zhang, Liang Wang, Rong Jin

CogAtom: From Cognitive Atoms to Olympiad-level Mathematical Reasoning in Large Language Models (2025.findings-emnlp)

Challenge:	supervised fine-tuning (SFT) is crucial for multimodal large language models, yet a comprehensive scaling law is lacking . et al.: scaling laws focus on model size, pre-training tokens, and MLLM SFT data volumes .
Approach:	They propose two scaling laws to guide optimal model-data configuration . they propose one applicable when training data volumes are well defined by researchers .
Outcome:	The proposed scaling laws provide valuable recommendations for optimal resource allocation . they show that the proposed laws are more accurate than existing models .

Copied to clipboard

Zhuofan Chen, Jiyuan He, Yichi Zhang, Xing Hu, Haoxing Wen, Jun Bai, Wenge Rong