Papers by Gong Zhang

LanguageFlow: Advancing Diffusion Language Generation with Probabilistic Flows (2024.naacl-long)

Challenge:	Unlike English letters, Chinese characters have rich and specific meanings.
Approach:	They propose to model Chinese words' internal structures as dependency trees with 11 labels for distinguishing syntactic relationships.
Outcome:	The proposed model of Chinese word-internal structures shows it can be used to parse sentences . it shows that the model can be applied to a sentence-level task with a competitive dependency parser.

Copied to clipboard

Shujian Zhang, Lemeng Wu, Chengyue Gong, Xingchao Liu

Text Generation

Challenge:	Recent work has demonstrated success in controlling sentence attributes and structure based on diffusion language models.
Approach:	They propose a language-rectified flow method that reformulates standard probabilistic flow models to learn ordinary differential equations to transport between the source and target distributions.
Outcome:	The proposed method outperforms baselines on three fine-grained control tasks and multiple high-quality text editing tasks.

HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference (2025.findings-acl)

Copied to clipboard

Ping Gong, Jiawei Yi, Shengnan Wang, Juncheng Zhang, Zewen Jin, Ouxiang Zhou, Ruibo Liu, Guanbin Xu, Youhui Bai, Bowen Ye, Kun Yuan, Tong Yang, Gong Zhang, Renhai Chen, Feng Wu, Cheng Li

Text Embedding as Treatment: A Meta Causal Approach for Robust Sentiment Classification (2026.findings-acl)

Challenge:	Existing top-k attention methods struggle to strike a balance between efficiency and accuracy.
Approach:	They propose a top-k attention approach that integrates low-overhead techniques into the Top-k Attention process to achieve 7.2 speedup compared to vanilla full attention.
Outcome:	The proposed approach achieves 7.2 speedup compared to current top-k attention methods while maintaining model accuracy.

Copied to clipboard

Fengxiang Cheng, Chuan Zhou, Xiang Li, Haoxuan Li, Wen-li Wang, Jinkun Chen, Mingming Gong, Kun Zhang

Information Extraction & Text Mining

Challenge:	Existing methods for sentiment classification use binary treatment of words . Existing approaches limit generalizability to novel words and low-frequency words if there is a word in a sentence that is not treated .
Approach:	They propose a meta-causal approach that uses a single training task to identify causal words for arbitrary words.
Outcome:	The proposed method reduces the spurious correlation between word treatment and sentiment classification by removing words with low treatment effects from a pre-trained language model.

APOLLO: An Optimized Training Approach for Long-form Numerical Reasoning (2024.lrec-main)

Copied to clipboard

Jiashuo Sun, Hang Zhang, Chen Lin, Xiangdong Su, Yeyun Gong, Jian Guo

Programming Languages in NLP

Challenge:	Existing methods to generate reasoning programs that ignore the differences between facts treated all facts equally, leading to wrong punishment of programs that differed from the ground truth.
Approach:	They propose an optimized training framework for long-form numerical reasoning that incorporates a number-aware negative sampling strategy and consistency-based reinforcement learning to increase execution accuracy.
Outcome:	The proposed method improves the performance of long-form numerical reasoning on the FinQA and ConvFinQA leaderboards.

RTQA : Recursive Thinking for Complex Temporal Knowledge Graph Question Answering with Large Language Models (2025.emnlp-main)

Copied to clipboard

Zhaoyan Gong, Juan Li, Zhiqiang Liu, Lei Liang, Huajun Chen, Wen Zhang

CULG: Commercial Universal Language Generation (2022.naacl-industry)

Challenge:	Current temporal knowledge graph question answering methods focus on implicit temporal constraints and lack the capability to handle complex temporal queries.
Approach:	They propose a temporal knowledge graph question answering framework that recursively decomposes questions into sub-problems and employs multi-path answer aggregation to improve fault tolerance.
Outcome:	The proposed framework outperforms existing methods on multiTQ and TimelineKGQA benchmarks.

Copied to clipboard

Haonan Li, Yameng Huang, Yeyun Gong, Jian Jiao, Ruofei Zhang, Timothy Baldwin, Nan Duan

Encouraging Lexical Translation Consistency for Document-Level Neural Machine Translation (2021.emnlp-main)

Challenge:	Pre-trained language models have improved performance for many NLP tasks in finance and healthcare.
Approach:	They propose a large-scale commercial universal language generation model which is pre-trained on a corpus drawn from 10 markets across 7 languages.
Outcome:	The proposed model outperforms other models on commercial generation tasks and on other markets, languages, and tasks.

Copied to clipboard

Xinglin Lyu, Junhui Li, Zhengxian Gong, Min Zhang

Multilinguality

Challenge:	Experimental results show document-level neural machine translation improves lexical consistency . inconsistent translations tend to confuse readers in some cases .
Approach:	They propose to use a word link to obtain a document word link and an auxiliary loss function to constrain that their translation should be consistent.
Outcome:	The proposed approach improves translation consistency on ChineseEnglish and EnglishFrench translation tasks.

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training (2025.acl-long)

Copied to clipboard

Zheheng Luo, Xin Zhang, Xiao Liu, Haoling Li, Yeyun Gong, Qi Chen, Peng Cheng

PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit (2022.naacl-demo)

Challenge:	Existing methods to optimise pretraining performance have not addressed the complexities of domain-adaptive continual pretraining.
Approach:	They propose a framework that dynamically assesses learning velocity and adjusts data proportions accordingly, favouring slower learning domains while de-emphasising faster learning ones.
Outcome:	The proposed framework achieves performance gains in math and code reasoning tasks and command-line generation benchmarks.

Copied to clipboard

Hui Zhang, Tian Yuan, Junkun Chen, Xintong Li, Renjie Zheng, Yuxin Huang, Xiaojie Chen, Enlei Gong, Zeyu Chen, Xiaoguang Hu, Dianhai Yu, Yanjun Ma, Liang Huang

Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling (2023.emnlp-main)

Challenge:	PaddleSpeech is an open-source speech toolkit that supports speech-to-text and text-to speech tasks.
Approach:	They describe the design philosophy and core architecture of PaddleSpeech to support several essential speech-to-text and text-to speech tasks.
Outcome:	The proposed framework achieves competitive or state-of-the-art performance on various speech datasets and implements the most popular methods.

Copied to clipboard

Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong, Jinyang Guo, Xianglong Liu

Noisy Pair Corrector for Dense Retrieval (2023.findings-emnlp)

Challenge:	asymmetric outliers in transformer language models are a challenge for post-training quantization . we propose a framework for outlier suppression that can be seamlessly migrated into subsequent modules .
Approach:	They propose a framework for post-training quantization that includes the channel-wise shifting and scaling for concentration.
Outcome:	The proposed framework can be migrated into subsequent modules while maintaining equivalence.

Copied to clipboard

Hang Zhang, Yeyun Gong, Xingwei He, Dayiheng Liu, Daya Guo, Jiancheng Lv, Jian Guo

Temp-R1: A Unified Autonomous Agent for Complex Temporal KGQA via Reverse Curriculum Reinforcement Learning (2026.acl-long)

Challenge:	Existing dense retrieval models assume that query-document pairs are exactly matched, resulting in mismatched-pair noise.
Approach:	They propose a novel approach to train an effective model with mismatched-pair noise.
Outcome:	The proposed model performs well on natural question and triviaQA, code-search benchmarks and SO-DS.

Copied to clipboard

Zhaoyan Gong, Zhiqiang Liu, Songze Li, Xiaoke Guo, Yuanxiang Liu, Xinle Deng, Zhizhen Liu, Lei Liang, Huajun Chen, Wen Zhang

Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method (2023.findings-acl)

Challenge:	Existing methods rely on fixed workflows and expensive closed-source APIs, limiting flexibility and scalability.
Approach:	They propose a temporal reasoning agent that trains on difficult questions first . they expand the action space with specialized internal actions alongside external action .
Outcome:	The proposed agent improves 19.8% over baselines on complex questions and multi-tasks.

Copied to clipboard

Shicheng Tan, Weng Lam Tam, Yuanchun Wang, Wenwen Gong, Shu Zhao, Peng Zhang, Jie Tang

Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalability (2026.acl-long)

Challenge:	Existing knowledge distillation methods rely on intermediate layer features and golden labels, which require aligned model architecture and labeled data respectively.
Approach:	They propose a general language model distillation method that performs two-stage word prediction distillation and vocabulary compression, which is simple and shows extremely strong performance.
Outcome:	The proposed method outperforms 25 state-of-the-art methods on the SuperGLUE benchmark, achieving an average score that surpasses the best method by 3%.

Copied to clipboard

Xiao Liang, Zhong-Zhi Li, Zhenghao Lin, Eric Hanchen Jiang, Hengyuan Zhang, Yelong Shen, Kai-Wei Chang, Ying Nian Wu, Yeyun Gong, Weizhu Chen

Selected Languages are All You Need for Cross-lingual Truthfulness Transfer (2025.coling-main)

Challenge:	Large language models have demonstrated strong reasoning capabilities through step-by-step chain-of-thought (CoT) reasoning, but their strictly sequential nature constrains test-time scalability.
Approach:	They propose an end-to-end reinforcement learning framework to enhance LLMs' DAC-style reasoning capacity by decomposing a problem into subproblems and solving them sequentially.
Outcome:	The proposed model surpasses CoT by 8.6% and 6.3% on competition-level benchmarks and is available at the [github.com/MasterVito/DAC-RL].

Copied to clipboard

Weihao Liu, Ning Wu, Wenbiao Ding, Shining Liang, Ming Gong, Dongmei Zhang

Multilinguality

Challenge:	Existing methods for truthfulness enhancement in English are limited to multilingual scenarios.
Approach:	They propose a method for cross-lingual truthfulness transfer that uses language bias and transfer contributions to select an optimal subset of all tested languages and employ translation instruction tuning for cross language truthfulness transfers.
Outcome:	The proposed method reduces multilingual representation disparity and boosts cross-lingual truthfulness transfer of LLMs.

GMFL: Efficient Global Masking for Federated LLM Fine-tuning (2026.acl-long)

Copied to clipboard

Xin Huang, Yan Hu, Yue-Jiao Gong, Xinglin Zhang

UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions (2026.acl-long)

Challenge:	Low-Rank Adaptation (LoRA) has emerged as a prominent solution to mitigate the communication and computation costs in federated fine-tuning of Large Language Models (LLMs).
Approach:	They propose a plug-and-play layer freezing mechanism to integrate with existing federated fine-tuning frameworks.
Outcome:	The proposed solution reduces communication overhead and lowers computational costs while preserving the performance of the underlying federated fine-tuning methods.

Copied to clipboard

Chunyu Qiang, Xiaopeng Wang, Kang Yin, Yuzhe Liang, Yuxin Guo, Teng Ma, Ziyu Zhang, Tianrui Wang, Cheng Gong, Yushen Chen, Ruibo Fu, Longbiao Wang, Jianwu Dang

Recurrent Inference in Text Editing (2020.findings-emnlp)

Challenge:	Generative audio modeling has been fragmented into specialized tasks such as text-to-speech (TTS), text- to-music (TTM), and text-ta (TTA) specialized models require reference audio for timbre cloning and strict phoneme alignment, whereas TTA models generate unstructured textures from open-ended captions.
Approach:	They propose a unified flow-matching framework capable of synthesizing speech, music, sound effects . they propose 'token injection mechanism' that projects unstructured environmental sounds into structured temporal latent space .
Outcome:	The proposed framework achieves state-of-the-art performance in instruction-based TTS and TTM while maintaining competitive fidelity in TTA.

Copied to clipboard

Ning Shi, Ziheng Zeng, Haotian Zhang, Yichen Gong

Challenge:	Existing inference methods map the unedited text to the edited text or to the editing operations, but performance is degraded by the limited source text encoding and long, varying decoding steps.
Approach:	They propose a new inference method that iteratively performs editing actions . they introduce three types of editing tasks: AOR, AES, AEC .
Outcome:	The proposed method significantly narrows the problem space by iterating editing actions.

Mitigating Posterior Salience Attenuation in Long-Context LLMs with Positional Contrastive Decoding (2025.acl-short)

Copied to clipboard

Zikai Xiao, Ziyang Wang, Wen Ma, Yan Zhang, Wei Shen, WangYan WangYan, Luqi Gong, Zuozhu Liu

Challenge:	Current solutions incur prohibitive training costs, leaving statistical behaviors and cost-effective approaches underexplored.
Approach:	They propose a positional contrast decoding technique that contrasts long-aware attention with designed local-awn attention.
Outcome:	The proposed model achieves state-of-the-art performance on long-context benchmarks.

Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis (2022.emnlp-main)

Copied to clipboard

Shuai Fan, Chen Lin, Haonan Li, Zhenghao Lin, Jinsong Su, Hang Zhang, Yeyun Gong, JIan Guo, Nan Duan

Sentiment Analysis

Challenge:	Existing pre-trained language representation models (PLMs) capture sentiment information from word-level while under-considering sentence-level information.
Approach:	They propose a Sentiment-aware pre-trained language model with combined Word-level and Sentence-level Pre-training tasks that enhance the PLM’s knowledge about sentiment words.
Outcome:	The proposed model achieves state-of-the-art on various sentence-level and aspect-level sentiment classification benchmarks.

ASTRA: Adaptive Semantic Tree Reasoning Architecture for Complex Table Question Answering (2026.acl-long)

Copied to clipboard

Xiaoke Guo, Songze Li, Zhiqiang Liu, Zhaoyan Gong, Yuanxiang Liu, Huajun Chen, Wen Zhang

Mixture-of-Modules: Reinventing Transformers as Dynamic Assemblies of Modules (2024.emnlp-main)

Challenge:	Existing serialization methods fail to capture explicit hierarchies and lack schema flexibility . Existing tree-based approaches suffer from limited semantic adaptability .
Approach:	They propose a method that leverages the global semantic awareness of LLMs to reconstruct tables into Logical Semantic Trees.
Outcome:	The proposed method achieves state-of-the-art (SOTA) performance on complex table benchmarks.

Copied to clipboard

Zhuocheng Gong, Ang Lv, Jian Guan, Wei Wu, Huishuai Zhang, Minlie Huang, Dongyan Zhao, Rui Yan

Quantifying and Improving the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Data (2026.acl-long)

Challenge:	Empirical results show that MoMs consistently outperform vanilla transformers .
Approach:	They propose an architecture that allows for a mixture-of-modules computation that uses a finite set of modules defined by multi-head attention and feed-forward networks.
Outcome:	The proposed architecture outperforms vanilla Transformers and their variants in multiple ways.

Copied to clipboard

Shiping Yang, Jie Wu, Wenbiao Ding, Ning Wu, Shining Liang, Ming Gong, Hongzhi Li, Hengyuan Zhang, Angel X. Chang, Dongmei Zhang

Beyond Logits: Aligning Feature Dynamics for Effective Knowledge Distillation (2025.acl-long)

Challenge:	Existing studies on robustness to explicit noise (e.g., document semantics) but overlook implicit noise (spurious features).
Approach:	They propose a framework to quantify the robustness of RAGs against spurious features by integrating a data synthesis pipeline and a taxonomy.
Outcome:	The proposed framework quantifies the robustness of RALMs against spurious features.

Copied to clipboard

Guoqiang Gong, Jiaxing Wang, Jin Xu, Deping Xiang, Zicheng Zhang, Leqi Shen, Yifeng Zhang, JunhuaShu JunhuaShu, ZhaolongXing ZhaolongXing, Zhen Chen, Pengzhang Liu, Ke Zhang

Search Augmented Instruction Learning (2023.findings-emnlp)

Challenge:	Knowledge distillation (KD) compresses large language models into lightweight versions called student models.
Approach:	They propose to align the entire feature dynamics between teacher and student models by using two additional loss terms to achieve this.
Outcome:	The proposed method matches the entire feature dynamics between teacher and student models rather than just the final states.

Copied to clipboard

Hongyin Luo, Tianhua Zhang, Yung-Sung Chuang, Yuan Gong, Yoon Kim, Xixin Wu, Helen Meng, James Glass

ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training (2020.findings-emnlp)

Challenge:	Large language models (LLMs) have been significantly improved by instruction fine-tuning, but still lack transparency and the ability to utilize up-to-date knowledge and information.
Approach:	They propose a search-augmented instruction learning model which grounds the language generation and instruction following abilities on complex search results generated by in-house and external search engines.
Outcome:	The proposed model outperforms plain LLMs on zero-shot language tasks and can generate both natural and programming languages following natural language guidance and requests.

Copied to clipboard

Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, Ming Zhou

Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training (2021.acl-long)

Challenge:	Existing sequence-to-sequence models are optimized for future n-gram prediction and n stream self-attention mechanism.
Approach:	They propose a self-supervised objective called future n-gram prediction and the proposed n stream self-attention mechanism to optimize the model for sequence-to-sequence learning.
Outcome:	The proposed model achieves state-of-the-art on CNN/DailyMail, Gigaword, and SQuAD 1.1 benchmarks compared to the models using the same scale pre-training corpus.

Copied to clipboard

Linqing Chen, Junhui Li, Zhengxian Gong, Boxing Chen, Weihua Luo, Min Zhang, Guodong Zhou

Machine Translation

Challenge:	Context-aware neural machine translation (NMT) remains challenging due to the lack of large-scale document-level parallel corpora.
Approach:	They propose to use large-scale parallel datasets and source-side monolingual documents to improve context-aware neural machine translation.
Outcome:	The proposed model can be used to translate both sentences and documents on four translation tasks.

Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models (2026.findings-acl)

Copied to clipboard

Teng Wang, Jiang Zhangyi, Zhenqi He, Hailei Gong, Shenyang Tong, Wenhan Yang, Zeyu Li, Yanan Zheng, Zifan He, Zewen Ye, Shengjie Ma, Jianping Zhang

What’s Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI Reasoning (2026.findings-acl)

Challenge:	Existing Process Reward Models (PRMs) are vulnerable to reward hacking and require expensive, large-scale annotation of reasoning steps.
Approach:	They propose a reward model approach which evaluates both individual and consecutive reasoning steps from fine-grained and coarse-grounded level.
Outcome:	Empirical results show that the proposed model performs better than existing PRMs and is more robust than existing models.

Copied to clipboard

Songze Li, Xiaoke Guo, Tianqi Liu, Biao Yi, Zhaoyan Gong, Zhiqiang Liu, Huajun Chen, Wen Zhang

RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging (2025.emnlp-main)

Challenge:	Existing GUI reasoning methods rely on direct screen-based decision-making, which lacks interpretability and overlooks a comprehensive understanding of UI elements, ultimately leading to task failure.
Approach:	They propose a GUI reasoning paradigm that treats the GUI reasoning task as a cyclic *Screen-UI elements-Action process.
Outcome:	The proposed paradigm achieves state-of-the-art UI understanding performance while yielding superior results in GUI reasoning tasks.

Copied to clipboard

Bowen Wang, Haiyuan Wan, Liwen Shi, Chen Yang, Peng He, Yue Ma, Haochen Han, Wenhao Li, Tiao Tan, Yongjian Li, Fangming Liu, Gong Yifan, Sheng Zhang

When Allies Turn Foes: Exploring Group Characteristics of LLM-Based Multi-Agent Collaborative Systems Under Adversarial Attacks (2025.findings-emnlp)

Challenge:	Existing models that require task labels or performance trade-offs are susceptible to catastrophic forgetting.
Approach:	They propose a representation-aware model merging framework for continual learning without access to historical data.
Outcome:	The proposed framework outperforms baselines in knowledge retention and generalization across five NLP tasks and multiple continual learning scenarios.

Copied to clipboard

Jiahao Zhang, Baoshuo Kan, Tao Gong, Fu Lee Wang, Tianyong Hao

Robustness in NLP

Challenge:	a new study examines the group characteristics of adversarial agents in multi-agent collaborative systems . collaborative agents are tasked with generating counterfactual answers to a given collaborative problem .
Approach:	They evaluate collaborative systems under adversarial attacks and propose methods to mitigate them . they also introduce a new metric to quantify the robustness of collaborative systems against such attacks .
Outcome:	The proposed method has been proven effective against adversarial attacks.

Mining Word Boundaries from Speech-Text Parallel Data for Cross-domain Chinese Word Segmentation (2025.coling-main)

Copied to clipboard

Xuebin Wang, Lei Zhang, Zhenghua Li, Shilin Zhou, Chen Gong, Yang Hou

PRAM: An End-to-end Prototype-based Representation Alignment Model for Zero-resource Cross-lingual Named Entity Recognition (2023.findings-acl)

Challenge:	Recent studies on Chinese Word Segmentation (CWS) have focused on the cross-domain scenarios, but there is a high cost of manually annotating high-quality data.
Approach:	They propose to explicitly mine word boundaries from parallel speech-text data by using the Montreal Forced Aligner toolkit to perform character-level alignment on speech- text data.
Outcome:	The proposed approach is based on character-level alignment on speech-text data and a robust complete-then-train (CTT) strategy.

Copied to clipboard

Yucheng Huang, Wenqiang Liu, Xianli Zhang, Jun Lang, Tieliang Gong, Chen Li

Representation Learning

Challenge:	Existing methods to address the named entity recognition problem are limited and lack explicit optimization specific to the task.
Approach:	They propose a prototype-based representation alignment model for a cross-lingual named entity recognition task using labeled source language data.
Outcome:	The proposed model outperforms existing state-of-the-art methods in some challenging scenarios.

MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation (2025.findings-emnlp)

Copied to clipboard

Chenghao Yang, Yinbo Luo, Zhoufutu Wen, Qi Chu, Tao Gong, Longxiang Liu, Kaiyuan Zhang, Jianpeng Jiao, Ge Zhang, Wenhao Huang, Nenghai Yu

Dialogue Systems & Conversational Agents

Challenge:	Large Language Models (LLMs) have been widely adopted in real-world dialogue applications, but their robustness is criticized all along.
Approach:	They propose to use play-by-play text commentary to build a multi-turn athletic real-world scenario dialogue benchmark to evaluate three critical aspects of multi-turned conversations: ultra multi- turn, interactive multi-twist, and cross-turn tasks.
Outcome:	The proposed benchmarks outperform open-source LLMs on three critical aspects of multi-turn conversations: ultra multi-turned, interactive multi- turn, and cross-turn tasks.

XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecs (2026.acl-long)

Copied to clipboard

Yitian Gong, Luozhijie Jin, Kuangwei Chen, Dong Zhang, Ruifan Deng, Xiaogui Yang, Xin Zhang, Zhaoye Fei, Qinyuan Cheng, Shimin Li, Xipeng Qiu

Reasoning is All You Need for Video Generalization: A Counterfactual Benchmark with Sub-question Evaluation (2025.findings-acl)

Challenge:	Existing speech codecs struggle to balance these objectives at low bitrates . XY-Tokenizer achieves stronger semantic alignment than representative semantic-distillation codec .
Approach:	They propose a low-bitrate speech codec that aligns discrete speech representations with text while preserving fine-grained acoustic details for reconstruction.
Outcome:	The proposed codec outperforms existing low-bitrate speech codecs in speech understanding and generation tasks.

Copied to clipboard

Qiji Zhou, YiFan Gong, Guangsheng Bao, Hongjie Qiu, Jinqiang Li, Xiangrong Zhu, Huajian Zhang, Yue Zhang

Robustness in NLP

Challenge:	Existing multimodal benchmarks often overlook counterfactual reasoning, which is crucial for robust video understanding.
Approach:	They propose a multidimensional multimodal benchmark that systematically evaluates MLLMs across the abstract-concrete and perception-cognition dimensions.
Outcome:	The proposed model decomposes complex queries into structured sub-questions, enabling fine-grained reasoning analysis.

Reinforcement Learning on Pre-Training Data (2026.acl-long)

Copied to clipboard

Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang

COPNER: Contrastive Learning with Prompt Guiding for Few-shot Named Entity Recognition (2022.coling-1)

Challenge:	Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings.
Approach:	They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data.
Outcome:	Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base.

Copied to clipboard

Yucheng Huang, Kai He, Yige Wang, Xianli Zhang, Tieliang Gong, Rui Mao, Chen Li

Mask Attention Networks: Rethinking and Strengthen Transformer (2021.naacl-main)

Challenge:	Existing methods for Named Entity Recognition (NER) use a similarity metric to measure semantic similarity between test samples and referents, but their performance is limited due to the label scarcity.
Approach:	They propose a novel approach to learn a similarity metric for measuring the semantic similarity between test samples and referents, where each referent represents an entity class.
Outcome:	The proposed approach outperforms state-of-the-art models with a significant margin in most cases.

Copied to clipboard

Zhihao Fan, Yeyun Gong, Dayiheng Liu, Zhongyu Wei, Siyuan Wang, Jian Jiao, Nan Duan, Ruofei Zhang, Xuanjing Huang

CMMLU: Measuring massive multitask language understanding in Chinese (2024.findings-acl)

Challenge:	Existing research explores to enhance the two sublayers separately to improve the capability of Transformer for text representation.
Approach:	They propose to combine SAN and Feed-Forward Networks to create a dynamic mask attention network with a learnable mask matrix which can model localness adaptively.
Outcome:	The proposed model outperforms the original Transformer on translation and text summarization tasks.

Copied to clipboard

Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, Timothy Baldwin

Transferring from Formal Newswire Domain with Hypernet for Twitter POS Tagging (D18-1)

Challenge:	Existing large language models struggle to achieve an accuracy of even 60%, which is the pass mark for Chinese exams.
Approach:	They propose to use CMMLU to evaluate Chinese multilingual and Chinese LLMs in a comprehensive benchmark that covers various subjects and settings.
Outcome:	The proposed benchmark covers natural sciences, social sciences, engineering, and the humanities and aims to improve on existing models.

Copied to clipboard

Tao Gui, Qi Zhang, Jingjing Gong, Minlong Peng, Di Liang, Keyu Ding, Xuanjing Huang

TopoDIM: One-shot Topology Generation of Diverse Interaction Modes for Multi-Agent Systems (2026.findings-acl)

Challenge:	Existing POS tagging methods for Twitter use labeled newswire text . however, Twitter users tend to mimic formal media expressions and develop linguistically informal styles.
Approach:	They propose to use newswire text to learn POS tagging for Twitter while twitter users are developing linguistically informal styles.
Outcome:	The proposed method achieves better performance than state-of-the-art methods on three different datasets.

Copied to clipboard

Rui Sun, Jie Ding, Chenghua Gong, Tianjun Gu, Yihang Jiang, Juyuan Zhang, Liming Pan, Linyuan Lü

Dialogue Systems & Conversational Agents

Challenge:	Existing communication topologies rely on spatio-temporal dialogues, which incur high latency and computation.
Approach:	They propose a framework for one-shot Topology generation with Diverse Interaction Modes that enables agents to construct heterogeneous communication without iterative coordination.
Outcome:	The proposed framework reduces total token consumption by 46.41% while improving average performance by 1.50% over state-of-the-art methods.

Subjective Topic meets LLMs: Unleashing Comprehensive, Reflective and Creative Thinking through the Negation of Negation (2024.emnlp-main)

Copied to clipboard

Fangrui Lv, Kaixiong Gong, Jian Liang, Xinyu Pang, Changshui Zhang

A Graph Representation of Semi-structured Data for Web Question Answering (2020.coling-main)

Challenge:	Large language models (LLMs) exhibit powerful reasoning capacity, but their evaluation still lacks comprehensiveness.
Approach:	They propose a framework grounded in the principle of the Negation of Negation (NeoN) to unleash the potential comprehensive, reflective, and creative thinking abilities of LLMs.
Outcome:	The proposed framework unleashes the potential comprehensive, reflective, and creative thinking abilities of large language models.

Copied to clipboard

Xingyao Zhang, Linjun Shou, Jian Pei, Ming Gong, Lijie Wen, Daxin Jiang

Structured Data in NLP

Challenge:	Existing studies treat semi-structured data as flat documents with pieces of text . semi-structural data is more effective to represent rich relational information . question answering is an important feature in most search engines .
Approach:	They propose a graph representation of Web tables and lists based on categorization of components and their relations . they also develop reasoning techniques on the graph model for the question answering task .
Outcome:	The proposed graph improves F1 score by 3.90 points over the state-of-the-art baselines on real datasets.

Long Chain-of-Thought Fine-tuning via Understanding-to-Reasoning Transition (2025.emnlp-main)

Copied to clipboard

Chenxin An, Zhihui Xie, Xiaonan Li, Ming Zhong, Shansan Gong, Lei Li, Jun Zhang, Jingjing Xu, Lingpeng Kong

Learning with Different Amounts of Annotation: From Zero to Many Labels (2021.emnlp-main)

Challenge:	Existing research on long-context scaling in language models has focused on managing lengthy input prompts instead of producing long outputs.
Approach:	They propose a sequence-level curriculum learning framework that shifts a model’s focus from interpreting long chain-of-thoughts to generating them.
Outcome:	Experiments on rigorous reasoning benchmarks, including AIME24 and GPQA Diamond, show that the proposed approach surpasses standard fine-tuning by over 10% while maintaining robust performance on understanding tasks.

Copied to clipboard

Shujian Zhang, Chengyue Gong, Eunsol Choi

Challenge:	a lack of annotator agreement can hinder training of NLP systems . we propose a learning algorithm that can learn from training examples with zero, one, or multiple labels.
Approach:	They propose an annotation distribution scheme that assigns multiple labels to training examples . they propose a learning algorithm that can learn from training examples with different amount of annotation .
Outcome:	The proposed method achieves consistent gains in two tasks, suggesting distributing labels unevenly among training examples can be beneficial for many NLP tasks.

Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning (2024.findings-naacl)

Copied to clipboard

Tianhua Zhang, Jiaxin Ge, Hongyin Luo, Yung-Sung Chuang, Mingye Gao, Yuan Gong, Yoon Kim, Xixin Wu, Helen Meng, James Glass

FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steering (2025.findings-acl)

Challenge:	Existing methods for surfacing symbolic reasoning capabilities are limited to narrow tasks . arithmetic computations are unnatural to perform in pure language space, and hence present difficulties for LLMs.
Approach:	They propose a natural language embedded program framework for solving symbolic reasoning tasks.
Outcome:	The proposed framework improves on strong baselines across math and symbolic reasoning, text classification, question answering, and instruction following tasks.

Copied to clipboard

Yichen Li, Zhiting Fan, Ruizhe Chen, Xiaotang Gai, Luqi Gong, Yan Zhang, Zuozhu Liu

ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation (2021.acl-demo)

Challenge:	Existing prompt-based debiasing methods exhibit instability due to sensitivity to prompt changes . fine-tuning-based techniques incur substantial computational overhead and catastrophic forgetting .
Approach:	They propose a debiasing framework that encodes fairness-related features into separable directions in the hidden activation space.
Outcome:	The proposed framework performs inference-time debiasing without requiring retraining or prompt design . it detects bias signatures in activations and then computes debiased steering vectors . the proposed framework is available to download in the u.s.

Copied to clipboard

Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang, Jiusheng Chen, Ruofei Zhang, Houqiang Li, Nan Duan

An Empirical Investigation of Implicit and Explicit Knowledge-Enhanced Methods for Ad Hoc Dataset Retrieval (2023.findings-emnlp)

Challenge:	Existing models for pre-training are not convenient for users to find and set them up.
Approach:	They propose to extend ProphetNet into other domains and languages by pre-training models . they pre-train a cross-lingual generation model ProphetNet-Multi and a Chinese generation model .
Outcome:	The proposed models achieve new state-of-the-art on 10 benchmarks.

Copied to clipboard

Weiqing Luo, Qiaosheng Chen, Zhiyang Zhang, Zixian Huang, Gong Cheng

Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarios (2024.findings-acl)

Challenge:	Existing methods for ad hoc dataset retrieval are lexical and cannot capture semantic similarity.
Approach:	They propose to implement and evaluate a set of implicit and explicit knowledge-enhancement retrieval methods on two test collections to find semantic matches for ad hoc dataset retrieval.
Outcome:	The proposed methods are compared with existing methods on two test collections and reveal the unique features of the task and suggest an interpolation of different kinds of methods as the current best practice.

Copied to clipboard

Lei Lin, Jiayi Fu, Pengli Liu, Qingyang Li, Yan Gong, Junchen Wan, Fuzheng Zhang, Zhongyuan Wang, Di Zhang, Kun Gai

AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora (2026.acl-long)

Challenge:	chain-of-thought (CoT) prompting has been shown to be effective on complex reasoning tasks, but the naive greedy decoding used in CoT prompting causes the repetitiveness and local optimality.
Approach:	They propose a generalizable ensemble-optimization method that uses a set of reasoning paths to prompt a language model one more time to determine the optimal answer.
Outcome:	The proposed method can be generalized to almost all scenarios where the type of input questions and answer format of reasoning paths may be unknown.

Copied to clipboard

Jiaxin Bai, Wei Fan, Qi Hu, Qing Zong, Chunyang Li, Hong Ting Tsang, Hongyu Luo, Yauwai Yim, Haoyu Huang, Xiao Zhou, Feng Qin, Tianshi Zheng, Xi Peng, Xin Yao, Huiwen Yang, Leijie Wu, JI Yi, Gong Zhang, Renhai Chen, Yangqiu Song

Large Language Models as an Indirect Reasoner: Contrapositive and Contradiction for Automated Reasoning (2025.coling-main)

Challenge:	Existing knowledge graph construction frameworks require predefined schemas, limiting their scalability and domain coverage.
Approach:	They propose a framework for fully autonomous knowledge graph construction that eliminates the need for predefined schemas.
Outcome:	The proposed framework outperforms state-of-the-art models on multi-hop QA tasks and enhances LLM factuality.

Copied to clipboard

Yanfang Zhang, Yiliu Sun, Yibing Zhan, Dapeng Tao, Dacheng Tao, Chen Gong

Multi-View Document Representation Learning for Open-Domain Dense Retrieval (2022.acl-long)

Challenge:	Recent studies have focused on improving the ability of Large Language Models to perform complex reasoning.
Approach:	They propose a Direct-Indirect Reasoning method that integrates DR and IR as parallel reasoning paths that are merged to derive the final answer.
Outcome:	The proposed method outperforms existing methods on four datasets related to logical reasoning and proof.

Copied to clipboard

Shunyu Zhang, Yaobo Liang, Ming Gong, Daxin Jiang, Nan Duan

Representation Learning

Challenge:	Existing methods for dense retrieval are hard to match with multiple views.
Approach:	They propose a multi-view document representation learning framework to generate multiple embeddings through viewers to represent documents and enforce them to align with different queries.
Outcome:	The proposed method outperforms recent works and achieves state-of-the-art results.

ALLSH: Active Learning Guided by Local Sensitivity and Hardness (2022.findings-naacl)

Copied to clipboard

Shujian Zhang, Chengyue Gong, Xingchao Liu, Pengcheng He, Weizhu Chen, Mingyuan Zhou

Low-Resource NLP

Challenge:	Existing studies show that labeling in crowdsourcing annotations is not an annotation artifact but rather a core linguistic phenomenon.
Approach:	They propose to retrieve unlabeled data with a local sensitivity and hardness-aware acquisition function.
Outcome:	The proposed method achieves consistent gains over the commonly used active learning strategies in various classification tasks.

ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods (2024.emnlp-main)

Copied to clipboard

Roy Xie, Junlin Wang, Ruomin Huang, Minxing Zhang, Rong Ge, Jian Pei, Neil Gong, Bhuwan Dhingra

Self-Correction Makes LLMs Better Parsers (2025.findings-emnlp)

Challenge:	ReCaLL (Relative Conditional Log-Likelihood) is a membership inference attack that can detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities.
Approach:	They propose a membership inference attack to detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities.
Outcome:	The proposed model achieves state-of-the-art performance on the WikiMIA dataset, even with random and synthetic prefixes, and can be further improved using an ensemble approach.

Copied to clipboard

Ziyan Zhang, Yang Hou, Chen Gong, Zhenghua Li

FastSeq: Make Sequence Generation Faster (2021.acl-demo)

Challenge:	Large language models (LLMs) have achieved remarkable success across various natural language processing tasks, but they still face challenges in performing fundamental NLP tasks, such as syntactic parsing.
Approach:	They propose a method that leverages grammar rules from existing treebanks to guide LLMs in correcting previous errors.
Outcome:	The proposed method significantly improves performance on in-domain and cross-domain datasets.

Copied to clipboard

Yu Yan, Fei Hu, Jiusheng Chen, Nikhil Bhendawade, Ting Ye, Yeyun Gong, Nan Duan, Desheng Cui, Bingyu Chi, Ruofei Zhang

GLGE: A New General Language Generation Evaluation Benchmark (2021.findings-acl)

Challenge:	Transformer-based models have made tremendous impact in natural language generation, but inference speed is still a bottleneck due to large model size and intensive computing involved in auto-regressive decoding process.
Approach:	They propose an attention cache optimization, an efficient algorithm for detecting repeated n-grams, and an asynchronous generation pipeline with parallel I/O to accelerate sequence generation without loss of accuracy.
Outcome:	The proposed framework can accelerate the sequence generation by 4x to 9x with a simple one-line code change for a set of widely used and diverse models.

Copied to clipboard

Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan

BMCook: A Task-agnostic Compression Toolkit for Big Models (2022.emnlp-demos)

Challenge:	Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models.
Approach:	They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks.
Outcome:	The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages.

Copied to clipboard

Zhengyan Zhang, Baitao Gong, Yingfa Chen, Xu Han, Guoyang Zeng, Weilin Zhao, Yanxu Chen, Zhiyuan Liu, Maosong Sun

CoG: Controllable Graph Reasoning via Relational Blueprints and Failure-Aware Refinement over Knowledge Graphs (2026.acl-long)

Challenge:	Existing efforts to compress medium-sized models for specific tasks have limited results.
Approach:	They propose a task-agnostic compression toolkit for big models that implements quantization, pruning, distillation and MoEfication methods.
Outcome:	The proposed tool improves performance on a model with 3 billion parameters by 12x . it also outperforms the original model on three typical NLP benchmarks.

Copied to clipboard

Yuanxiang Liu, Songze Li, Xiaoke Guo, Zhaoyan Gong, Qifei Zhang, Huajun Chen, Wen Zhang

Structured Data in NLP

Challenge:	Existing approaches to large language models often exhibit cognitive rigidity, causing reasoning stagnation.
Approach:	They propose a training-free framework that mimics the interplay between intuition and deliberation.
Outcome:	The proposed framework outperforms state-of-the-art approaches on three benchmarks.

For-Value: Efficient Forward-Only Data Valuation for finetuning LLMs and VLMs (2026.acl-long)

Copied to clipboard

Wenlong Deng, Qi Zeng, Jiaming Zhang, Minghui Chen, Zixin Ding, Christos Thrampoulidis, Boying Gong, Xiaoxiao Li

Multi-grained Chinese Word Segmentation with Weakly Labeled Data (2020.coling-main)

Challenge:	Existing methods for data valuation rely on gradient computations, making them prohibitive for billion-parameter models.
Approach:	They propose a forward-only data valuation framework that enables efficient batch-scalable value estimation while maintaining effectiveness.
Outcome:	The proposed framework matches or outperforms gradient-based baselines in detecting influential data and mislabeled data while achieving significant efficiency improvements.

Copied to clipboard

Chen Gong, Zhenghua Li, Bowei Zou, Min Zhang

Knowledge Enhanced Pre-training for Cross-lingual Dense Retrieval (2024.lrec-main)

Challenge:	Existing work on single-grained word segmentation (SWS) focuses on segmenting a sentence into multiple word sequences to preserve all words of different granularities.
Approach:	They propose to use a graph-based parser to accommodate weakly labeled data for MWS by employing a simple yet competitive graph-basic parsers with local loss.
Outcome:	The proposed model outperforms the state-of-the-art model on weakly labeled data on a high-quality dataset from canonical newswire (NEWS) and non-canonical web (BAIKE) data.

Copied to clipboard

Hang Zhang, Yeyun Gong, Dayiheng Liu, Shunyu Zhang, Xingwei He, Jiancheng Lv, Jian Guo

Adapting LLM Agents with Universal Communication Feedback (2025.findings-naacl)

Challenge:	Existing mPLMs neglect the importance of knowledge in cross-lingual dense retrieval.
Approach:	They propose a novel mPLM that leverages knowledge to learn language-agnostic semantic representations from a multilingual knowledge base and an annotation of Wiki.
Outcome:	The proposed model achieves strong multilingual and cross-lingual retrieval performance with significant improvements over existing mPLMs.

Copied to clipboard

Kuan Wang, Yadong Lu, Michael Santacroce, Yeyun Gong, Chao Zhang, Yelong Shen

Challenge:	Recent advances in large language models (LLMs) have demonstrated potential for LLM agents.
Approach:	They propose a universal buffer and iterative pipeline to store feedback and itersative pipelines to enable LLM agents to explore and update their policy in an environment.
Outcome:	The proposed approach outperforms supervised instruction fine-tuning baselines on four datasets.

NaviMaster: Learning a Unified Policy for GUI and Embodied Navigation Tasks (2026.acl-long)

Copied to clipboard

Zhihao Luo, Wentao Yan, Jingyu Gong, Min Wang, Zhizhong Zhang, Xuhong Wang, Yuan Xie, Xin Tan

Challenge:	Recent advances in Graphical User Interface (GUI) and embodied navigation have driven progress, yet these domains have largely evolved in isolation, with disparate datasets and training paradigms.
Approach:	They propose a visual-target trajectory collection pipeline that generates trajectories for GUI and embodied tasks using a single formulation.
Outcome:	The proposed agent outperforms state-of-the-art agents in GUI navigation, spatial affordance prediction, and embodied navigation.

Metric-guided Distillation: Distilling Knowledge from the Metric to Ranker and Retriever for Generative Commonsense Reasoning (2022.emnlp-main)

Copied to clipboard

Xingwei He, Yeyun Gong, A-Long Jin, Weizhen Qi, Hang Zhang, Jian Jiao, Bartuer Zhou, Biao Cheng, Sm Yiu, Nan Duan

Shorten After You’re Right: Lazy Length Penalties for Reasoning RL (2026.findings-acl)

Challenge:	Existing work on commonsense generation requires models to have relational reasoning and compositional generalization capabilities.
Approach:	They propose a metric distillation rule to distill knowledge from a standard metric to a ranker and transfer it to re-ranking a retriever.
Outcome:	The proposed method surpasses the previous SOTA.

Copied to clipboard

Danlong Yuan, Tian Xie, Shaohan Huang, Huishuai Zhang, Zhuocheng Gong, Chong Luo, Furu Wei, Dongyan Zhao

XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation (2020.emnlp-main)

Challenge:	Existing shortening methods for long reasoning models rely on additional supervision or multi-stage post-training.
Approach:	They propose a lazy length penalty that imposes length pressure on models without extra training stages.
Outcome:	The proposed method significantly reduces response length without extra training stages while maintaining or improving performance.

Copied to clipboard

Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, Ming Zhou

Multilinguality

Challenge:	XGLUE provides a benchmark dataset to train large-scale cross-lingual pre-trained models . XCLUE provides 11 diversified tasks that cover both understanding and generation scenarios .
Approach:	They introduce a new benchmark dataset to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora.
Outcome:	The proposed dataset is labeled in English and includes only natural language understanding tasks.

An Enhanced Knowledge Injection Model for Commonsense Generation (2020.coling-main)

Copied to clipboard

Zhihao Fan, Yeyun Gong, Zhongyu Wei, Siyuan Wang, Yameng Huang, Jian Jiao, Xuanjing Huang, Nan Duan, Ruofei Zhang

Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models (2022.emnlp-main)

Challenge:	a recent study shows that digging the relationship of concepts from scratch is non-trivial for commonsense generation tasks.
Approach:	They use a retrieve-and-edit framework to retrieve a prototype with these concepts . they use qt and qq to generate commonsense questions at scale .
Outcome:	The proposed method significantly improves the performance on commonsense generation tasks.

Copied to clipboard

Shujian Zhang, Chengyue Gong, Xingchao Liu

Natural Language Interfaces

Challenge:	Xu et al., 2021) find that retrieval-reader models overfit top-rank passages and fail to reason over entire retrieval passages.
Approach:	They propose a passage mask mechanism which desensitizes the impact from top-rank retrieval passages and prevents the model from overfitting.
Outcome:	Experiments on open question answering, dialogue conversation, and fact verification show that the proposed model outperforms baselines.

Controllable Text Generation via Probability Density Estimation in the Latent Space (2023.acl-long)

Copied to clipboard

Yuxuan Gu, Xiaocheng Feng, Sicheng Ma, Lingyuan Zhang, Heng Gong, Weihong Zhong, Bing Qin

Text Generation

Challenge:	Existing control approaches cannot effectively model complex space with diverse attributes, high dimensionality, and asymmetric structure, leaving subsequent controls unsatisfactory.
Approach:	They propose a control framework using probability density estimation in the latent space and an invertible transformation function that maps the complex distributions to simple Gaussian distributions in the prior space.
Outcome:	The proposed method outperforms baselines on attribute relevance and text quality, achieving a new SOTA.

Fine-Grained Features-based Code Search for Precise Query-Code Matching (2025.coling-main)

Copied to clipboard

Xinting Zhang, Mengqiu Cheng, Mengzhen Wang, Songwen Gong, Jiayuan Xie, Yi Cai, Qing Li

Revisiting Grammatical Error Correction Evaluation and Beyond (2022.emnlp-main)

Challenge:	Existing methods to locate code snippets from databases represent the semantics of code and query by averaging the features of each token and word.
Approach:	They propose a fine-grained code search model that consists of a cross-modal encoder, mapping layer and classification layer to capture fine-granular interactions between code and query.
Outcome:	The proposed model significantly outperforms existing methods across multiple programming language datasets.

Copied to clipboard

Peiyuan Gong, Xuebo Liu, Heyan Huang, Min Zhang

Vector Calligrapher: Generating Scalable Vector Graphics via Structured Linguistic Supervision (2026.acl-long)

Challenge:	Pretraining-based (PT) evaluation metrics are not effective for training grammatical error correction systems.
Approach:	They propose a pretraining-based GEC evaluation metric which only uses PT-based metrics to score the corrected parts of the system.
Outcome:	The proposed evaluation metric outperforms existing methods on a CoNLL14 evaluation task.

Copied to clipboard

Bo Zhou, Xikang Chen, Yan Gong, Yin Zhang

Improving Chinese Named Entity Recognition with Multi-grained Words and Part-of-Speech Tags via Joint Modeling (2024.lrec-main)

Challenge:	Existing approaches to generate SVG-based fonts struggle with semantic ambiguity and inefficiency . edward mcginley: generic text tokenizers fragment coordinate-dense SVG XML into excessively long sequences .
Approach:	They propose a system that treats SVG generation as a conditional language modeling task . they propose linguistic supervision framework that decomposes typographic style into interpretable linguistic dimensions .
Outcome:	The proposed system improves CLIP score by +23% while reducing geometric error by 48% and boosts generation efficiency by 18% Command-per-Token (C/T) ratio.

Copied to clipboard

Chenhui Dou, Chen Gong, Zhenghua Li, Zhefeng Wang, Baoxing Huai, Min Zhang

KFCNet: Knowledge Filtering and Contrastive Learning for Generative Commonsense Reasoning (2021.findings-emnlp)

Challenge:	Named entity recognition (CNER) is a fundamental task in natural language processing (NLP).
Approach:	They propose a tree parsing approach for jointly modeling Chinese named entity recognition (CNER) with multi-grained word segmentation (MWS) and POS tagging tasks.
Outcome:	The proposed approach achieves better or comparable performance with current methods.

Copied to clipboard

Haonan Li, Yeyun Gong, Jian Jiao, Ruofei Zhang, Timothy Baldwin, Nan Duan

Easy Dataset: A Unified and Extensible Framework for Synthesizing LLM Fine-Tuning Data from Unstructured Documents (2025.emnlp-demos)

Challenge:	Pre-trained language models have led to substantial gains over a broad range of NLP tasks, but have limitations for high-quality tasks such as commonsense generation and ad keyword generation.
Approach:	They propose a Knowledge Filtering and Contrastive learning Network which references external knowledge and achieves better generation performance.
Outcome:	The proposed model outperforms the current state of the art on the CommonGen benchmark by a large margin.

Copied to clipboard

Ziyang Miao, Qiyu Sun, Jingyuan Wang, Yuchen Gong, Yaowei Zheng, Shiqi Li, Richong Zhang

Coherent Entity Disambiguation via Modeling Topic and Categorical Dependency (2023.findings-emnlp)

Challenge:	Existing data synthesis tools struggle to extract reliable fine-tuning data from heterogeneous documents.
Approach:	They propose a framework for synthesizing fine-tuning data from unstructured documents via an intuitive graphical user interface.
Outcome:	The proposed framework can extract reliable data from unstructured documents via an intuitive graphical user interface (GUI) it leverages persona-driven prompting approach to generate diverse question-answer pairs using public-available LLMs.

Copied to clipboard

Zilin Xiao, Linjun Shou, Xingyao Zhang, Jie Wu, Ming Gong, Daxin Jiang

Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations (2024.findings-emnlp)

Challenge:	Existing entity disambiguation methods struggle to capture explicit discourse-level dependencies, resulting in incoherent predictions at the abstract level.
Approach:	They propose an unsupervised variational autoencoder to extract latent topic vectors of context sentences to enhance coherence of entity predictions.
Outcome:	The proposed system achieves state-of-the-art on popular ED benchmarks with an average improvement of 1.3 F1 points.

Copied to clipboard

Nuo Chen, Zinan Zheng, Ning Wu, Ming Gong, Dongmei Zhang, Jia Li

ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models (2023.findings-emnlp)

Challenge:	Existing research focuses on developing powerful large language models for mathematical reasoning within monolingual languages.
Approach:	They propose to use translation to build powerful multilingual math reasoning models . they propose different training strategies to build xMR LLMs that outperform open-source LLM .
Outcome:	The proposed model outperforms open-source LLMs and surpasses ChatGPT in few-shot scenarios.

Copied to clipboard

Zhipeng Chen, Kun Zhou, Beichen Zhang, Zheng Gong, Xin Zhao, Ji-Rong Wen

Simulating Classroom Education with LLM-Empowered Agents (2025.naacl-long)

Challenge:	Large language models (LLMs) have excellent performance in evaluation benchmarks, but struggle in complex reasoning tasks.
Approach:	They propose a tool-augmented chain-of-thought reasoning framework for chat-based LLMs . they model chain- of-thoughting reasoning as multi-turn conversations to utilize tools .
Outcome:	The proposed framework can outperform state-of-the-art models on complex reasoning tasks.

Copied to clipboard

Zheyuan Zhang, Daniel Zhang-Li, Jifan Yu, Linlu Gong, Jinchang Zhou, Zhanxin Hao, Jianxiao Jiang, Jie Cao, Huiqin Liu, Zhiyuan Liu, Lei Hou, Juanzi Li

A Distributional Lens for Multi-Aspect Controllable Text Generation (2022.emnlp-main)

Challenge:	Initial studies have focused on task-specific, independent LLM-empowered agents, but the potential of LLMs within a multi-agent collaborative framework for classroom simulation with real user participation remains unexplored.
Approach:	They propose a multi-agent classroom simulation teaching framework that recognizes representative class roles and introduces a novel class control mechanism for automatic classroom teaching.
Outcome:	The proposed framework can simulate dynamic learning environment for users with active teacher-student and student-studente interactions.

Copied to clipboard

Yuxuan Gu, Xiaocheng Feng, Sicheng Ma, Lingyuan Zhang, Heng Gong, Bing Qin

Text Generation

Challenge:	Existing methods for multi-aspect control suffer from attribute degeneration due to mutual interference of these controllers.
Approach:	They propose to use attribute fusion to find the intersections of multiple attributes as their combination for generation.
Outcome:	The proposed method outperforms baselines on attribute relevance and text quality and achieves the SOTA.

CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion (2023.emnlp-main)

Copied to clipboard

Xingwei He, Yeyun Gong, A-Long Jin, Hang Zhang, Anlei Dong, Jian Jiao, Siu Yiu, Nan Duan

Guangwei Zhang, Jianing Zhu, Cheng Qian, Neil Zhenqiang Gong, Rada Mihalcea, Zhaozhuo Xu, Jingrui He, Jiaqi W. Ma, Chaowei Xiao, Bo Li, Ahmed Abbasi, Dongwon Lee, Heng Ji, Denghui Zhang

Challenge:	Experimental results show that dense retrieval models are better at obtaining query-informed representations.
Approach:	They propose a dual-encoder approach that computes latent representations of query and document independently, but inference replaces the real query with a generated one.
Outcome:	The proposed approach outperforms previous dense retrieval models on in-domain and out-of-domain datasets.

Copied to clipboard

Paraphrasing

Challenge:	Copyright Detective is the first interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs.
Approach:	They propose a system that detects copyright infringements and visualizes them . they use content recall testing, paraphrase-level similarity analysis and persuasive jailbreak probing .
Outcome:	The proposed system detects, analyzes, and visualizes potential copyright risks in LLM outputs.

Data Augmentation for Cross-domain Parsing via Lightweight LLM Generation and Tree Hybridization (2025.coling-main)

Copied to clipboard

Ziyan Zhang, Yang Hou, Chen Gong, Zhenghua Li

Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding (2026.acl-long)

Challenge:	Existing approaches for constituency parsing are expensive and lack high-quality labeled data.
Approach:	They propose a data augmentation method via lightweight large language model (LLM) generation and tree hybridization to generate a large number of structurally diverse instances.
Outcome:	The proposed method achieves significant improvements on five target domains with a lightweight LLM generation cost.

Copied to clipboard

Sensen Gao, Shanshan Zhao, Xu Jiang, Lunhao Duan, Yong Xien Chng, Qing-Guo Chen, Weihua Luo, Kaifu Zhang, Jia-Wang Bian, Mingming Gong

TemplateGEC: Improving Grammatical Error Correction with Detection Template (2023.acl-long)

Challenge:	Document understanding is critical for applications from financial analysis to scientific discovery.
Approach:	They propose a taxonomy based on domain, retrieval modality, and granularity and review advances involving graph structures and agentic frameworks.
Outcome:	The proposed model enables holistic retrieval and reasoning across all modalities, unlocking comprehensive document intelligence.

Copied to clipboard

Yinghao Li, Xuebo Liu, Shuo Wang, Peiyuan Gong, Derek F. Wong, Yang Gao, Heyan Huang, Min Zhang

MuCPAD: A Multi-Domain Chinese Predicate-Argument Dataset (2022.naacl-main)

Challenge:	Existing methods for grammatical error correction (GEC) have been developed.
Approach:	They propose a method which integrates the detection labels from a Seq2Edit model to construct a template as the input.
Outcome:	The proposed method can perform human-in-the-loop error correction tasks.

Copied to clipboard

Yahui Liu, Haoping Yang, Chen Gong, Qingrong Xia, Zhenghua Li, Min Zhang

Challenge:	Recent studies show that shallow semantic role labeling (SRL) performance drops under out-of-domain setting.
Approach:	They propose to annotate a multi-domain Chinese predicate-argument dataset using a frame-free annotation methodology and strict double annotation for improving data quality.
Outcome:	The proposed dataset is compared with a dataset from six different domains.

A Survey on Open Information Extraction from Rule-based Model to Large Language Model (2024.findings-emnlp)

Copied to clipboard

Liu Pai, Wenyang Gao, Wenjie Dong, Lin Ai, Ziwei Gong, Songfang Huang, Li Zongsheng, Ehsan Hoque, Julia Hirschberg, Yue Zhang

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit (2024.emnlp-industry)

Challenge:	Open Information Extraction (OpenIE) is a key NLP task aimed at extracting structured information from unstructured text sources.
Approach:	They propose to categorize OpenIE into rule-based, neural, and pre-trained large language models and discuss each within a chronological framework.
Outcome:	The paper categorizes OpenIE approaches into rule-based, neural, and pre-trained large language models, discussing each within a chronological framework.

Copied to clipboard

Ruihao Gong, Yang Yong, Shiqiao Gu, Yushi Huang, Chengtao Lv, Yunchen Zhang, Dacheng Tao, Xianglong Liu

Multi-Modal Generative Adversarial Network for Short Product Title Generation in Mobile E-Commerce (N19-2)

Challenge:	Existing quantization techniques have been categorized as 'simple' and 'highly efficient' however, their configurations vary from each other and cannot be fairly compared .
Approach:	They propose a plug-and-play compression toolkit to explore the impact of quantization.
Outcome:	The proposed toolkit explores the impact of quantization on large language models.

Copied to clipboard

Jianguo Zhang, Pengcheng Zou, Zhao Li, Yao Wan, Xiuming Pan, Yu Gong, Philip S. Yu

Robustness in NLP

Challenge:	Existing methods for short product title generation only consider textual information from long titles . MM-GAN incorporates image information and attribute tags from product, as well as textual info from original long titles.
Approach:	They propose a multi-modal generative adversarial network for short product title generation in E-commerce . they incorporate image information and attribute tags from product, as well as textual information from original long titles .
Outcome:	The proposed model outperforms state-of-the-art methods on a large-scale E-commerce dataset.

Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing (2026.acl-long)

Copied to clipboard

Lingkun Long, Yushi Huang, Shihao Bai, Ruihao Gong, Jun Zhang, Ao Zhou, Jianlei Yang

GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model (2023.acl-industry)

Challenge:	Existing methods for estimating attention importance for tokens are ineffective . dLLMs require bidirectional attention, which limits inference efficiency .
Approach:	They propose a training-free attention sparsification framework for efficient long-context inference . they propose 'sink-aware pruning strategy' to accurately estimate and remove redundant computation .
Outcome:	The proposed approach offers 29 lossless speedup under 32K context length.

Copied to clipboard

Shicheng Tan, Weng Lam Tam, Yuanchun Wang, Wenwen Gong, Shu Zhao, Peng Zhang, Jie Tang

CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search (2022.emnlp-main)

Challenge:	Existing knowledge distillation frameworks for language models are limited by memory and the use of complex distillation methods on larger-scale PLMs.
Approach:	They propose a general knowledge distillation framework that supports distillation on larger-scale PLMs using various distillation methods.
Outcome:	The proposed framework can support distillation on larger-scale PLMs and 25 mainstream methods on 8 NVIDIA A100 (40GB) GPUs.

Copied to clipboard

Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang, Weizhu Chen, Nan Duan

GUI0: Self-Evolving Foundational GUI Agents in Super App Ecosystems (2026.acl-long)

Challenge:	Existing code pre-training approaches often adopt (masked) language modeling as the training objective which targets on learning to predict (macked) tokens in a given code context.
Approach:	They propose a code-text contrastive learning model which learns function-level code semantic representations through large-scale code corpus.
Outcome:	The proposed model achieves new state-of-the-art with significant improvement over existing pre-trained models on eleven domain/language-specific code search tasks with six programming languages in different code granularity.

Copied to clipboard

Xinyi Wang, Wei Dai, Kyle Qiao, Ke Wang, Peng Chen, Gang Cao, null Kangqin, Zhongpu Wang, Xiaode Zhang, Yanming Liu, Jihao Gu, Jingtao Xu, Gong Zhi

Cross-domain Rumor Detection via Test-Time Adaptation and Large Language Models (2025.emnlp-main)

Challenge:	Automated interaction with graphical user interfaces (GUIs) is central to general artificial intelligence, but remains challenging within Super App ecosystems.
Approach:	They propose a framework synergizing autonomous data synthesis with dual-agent co-evolution . GUI0 establishes a domain-aware foundation model via synthesized corpora and employs curriculum-driven reinforcement learning .
Outcome:	The proposed framework outperforms Gemini-2.5-Pro and Claude-4-Sonnet in the SuperAPP benchmark and has universal efficacy across base models.

Copied to clipboard

Yuxia Gong, Shuguo Hu, Huaiwen Zhang

From Bottom to Top: Extending the Potential of Parameter Efficient Fine-Tuning (2024.emnlp-main)

Challenge:	Existing approaches focus on within-domain tasks, resulting in suboptimal performance in cross-domain scenarios due to domain shifts.
Approach:	They propose a framework that incorporates both single-domain model and target graph adaptation strategies tailored to the unique requirements of cross-domain rumor detection.
Outcome:	The proposed framework surpasses existing methods in rumor detection on social media.

Copied to clipboard

Jihao Gu, Zelin Wang, Yibo Zhang, Ziji Zhang, Ping Gong

Knowing More About Questions Can Help: Improving Calibration in Question Answering (2021.findings-acl)

Challenge:	Existing methods to fine-tune large language models primarily focus on the interaction between different layers, ignoring the fact that different layers store different information.
Approach:	They propose a Parameter Efficient Fine-Tuning method which freeze pre-trained parameters and fine-tunes only a few task-specific parameters.
Outcome:	The proposed methods reduce parameter count to nearly half by omitting fine-tuning in the middle layers.

Copied to clipboard

Shujian Zhang, Chengyue Gong, Eunsol Choi

Question Answering

Challenge:	Existing work on calibration focuses on model confidence, such as the max probability of the predicted class.
Approach:	They propose a calibration method which estimates whether model correctly predicts answer for each question.
Outcome:	The proposed calibration method achieves 5-10% gains on reading comprehension benchmarks.

Iterative Formalization and Planning in Partially Observable Environments (2026.findings-acl)

Copied to clipboard

Liancheng Gong, Wang Bill Zhu, Jesse Thomason, Li Zhang

Challenge:	Existing methods to formalize an environment into the Planning Domain Definition Language (PDDL) have been shown to improve performance and control.
Approach:	They propose a framework to iteratively formalize, plan, grow, and refine PDDL representations by decomposing the environment and the goal into fully observable episodes.
Outcome:	The proposed framework improves planning success and exhibits robustness against problem complexity compared to end-to-end approaches.

Uncertainty-Aware Label Refinement for Sequence Labeling (2020.emnlp-main)

Copied to clipboard

Tao Gui, Jiacheng Ye, Qi Zhang, Zhengyan Li, Zichu Fei, Yeyun Gong, Xuanjing Huang

TRACE: Evaluating Execution Efficiency of LLM-Based Code Translation (2026.acl-long)

Challenge:	Conditional random fields (CRF) for label decoding have been a problem for many tasks.
Approach:	They propose a two-stage label decoding framework that model long-term label dependencies while being much more computationally efficient.
Outcome:	The proposed method outperforms the CRF-based methods and greatly accelerates the inference process.

Copied to clipboard

Zhihao Gong, Zeyu Sun, Dong Huang, Qingyuan Liang, Jie M. Zhang, Dan Hao

CodeV: Issue Resolving with Visual Data (2025.findings-acl)

Challenge:	Large Language Models (LLMs) have improved the functional correctness of code translation, but execution efficiency remains overlooked.
Approach:	They propose a benchmark to explicitly assess execution efficiency in LLM-translated code.
Outcome:	The proposed benchmark identifies that execution efficiency is an essential dimension of code translation . the results highlight that correctness and efficiency are often misaligned .

Copied to clipboard

Linhao Zhang, Daoguang Zan, Quanshun Yang, Zhirong Huang, Dong Chen, Bo Shen, Tianyu Liu, Yongshun Gong, Huang Pengjie, Xudong Lu, Guangtai Liang, Lizhen Cui, Qianxiang Wang

When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors (2026.acl-long)

Challenge:	Large Language Models (LLMs) have expanded to more complex repository-level tasks.
Approach:	They propose a first approach to leveraging visual data to enhance the issue-resolving capabilities of Large Language Models (LLMs) they demonstrate the effectiveness of CodeV and provide valuable insights into leveraging visualization to resolve GitHub issues.
Outcome:	The proposed approach improves the issue-resolving capabilities of Large Language Models (LLMs) by using visual data.

Copied to clipboard

Chenghao Yang, Yuning Zhang, Zhoufutu Wen, Tao Gong, Jiaheng Liu, Qi Chu, Nenghai Yu

Allies: Prompting Large Language Model with Beam Search (2023.findings-emnlp)

Challenge:	Existing metrics fail to distinguish mandatory behaviors required for task success from non-mandatory patterns that reflect a model’s autonomous preferences.
Approach:	They propose to use response pattern similarity and action graph similarity to isolate non-mandatory behaviors from mandatory behaviors.
Outcome:	Evaluating 18 models from 8 providers on -Bench and 2-Bench against Claude Sonnet 4.5, the authors find that within-family model pairs score 5.9 pp higher in response pattern similarity and action graph similarity .

Copied to clipboard

Hao Sun, Xiao Liu, Yeyun Gong, Yan Zhang, Daxin Jiang, Linjun Yang, Nan Duan

Process-based Self-Rewarding Language Models (2025.findings-acl)

Challenge:	Existing methods to build LLMs with stacking are limited by their information coverage and low fault tolerance.
Approach:	They propose a method that leverages large language models to iteratively generate new queries from an input query.
Outcome:	The proposed method outperforms baselines on open-domain question answering benchmarks.

Copied to clipboard

Shimao Zhang, Xiao Liu, Xin Zhang, Junxiao Liu, Zheheng Luo, Shujian Huang, Yeyun Gong

Instructed Language Models with Retrievers Are Powerful Entity Linkers (2023.emnlp-main)

Challenge:	Existing methods to reward LLMs' outputs are not effective in mathematical reasoning scenarios and may lead to a decline in performance.
Approach:	They propose a process-based self-rewarding pipeline that integrates long-thought reasoning, step-wise LLM-as-a-Judge, and step- wise preference optimization within the existing paradigm.
Outcome:	The proposed model improves the performance of Large Language Models on multiple mathematical reasoning benchmarks and shows that it can surpass human capabilities.

Copied to clipboard

Zilin Xiao, Ming Gong, Jie Wu, Xingyao Zhang, Linjun Shou, Daxin Jiang

Chinese Spoken Named Entity Recognition in Real-world Scenarios: Dataset and Approaches (2024.findings-acl)

Challenge:	Generative approaches powered by large language models have demonstrated emergent abilities in tasks that require complex reasoning abilities.
Approach:	They propose a sequence-to-sequence training objective with instruction-tuning that enables casual language models to perform entity linking over knowledge bases.
Outcome:	The proposed framework outperforms existing approaches with +6.8 F1 points gain on average and huge advantage in training data efficiency and compute consumption.

Copied to clipboard

Shilin Zhou, Zhenghua Li, Chen Gong, Lei Zhang, Yu Hong, Min Zhang

Speech & Audio in NLP

Challenge:	Current Chinese Spoken NER datasets are laboratory-controlled and are limited in topics.
Approach:	They propose to use Chinese Spoken NER datasets to extract entities from speech to help voice assistants better grasp the intent behind user's questions and instructions.
Outcome:	The proposed methods improve on self-training-asr and mapping then distilling, and even compared with GPT4.0, they achieve better results.

Rhombus: Incentivizing Coordination in Parallel Thinking through Reinforcement Learning (2026.findings-acl)

Copied to clipboard

Ziyuan Nan, Qi Yi, Di Huang, Yutong Wu, Guanhua Huang, Xue Gong, Kejiao Li, Yuhao Jiang, Chenchen Zhang, Zenan Xu, Xing Hu, Bo Zhou

Ensuring Safe and High-Quality Outputs: A Guideline Library Approach for Language Models (2024.naacl-long)

Challenge:	Parallel thinking is a promising avenue for scaling test-time compute in Large Language Models . however, coordinating the exploration and aggregation stages remains challenging .
Approach:	They propose a parallel thinking framework that explicitly incentivizes coordination between components via end-to-end reinforcement learning.
Outcome:	The proposed framework improves accuracy by 6.0% over long chain-of-thought baselines while reducing wall-clock latency by 39.4% under matched token budgets.

Copied to clipboard

Yi Luo, Zhenghao Lin, YuHao Zhang, Jiashuo Sun, Chen Lin, Chengjin Xu, Xiangdong Su, Yelong Shen, Jian Guo, Yeyun Gong