Papers by Cheng Shen

Neural Machine Translation with Contrastive Translation Memories (2022.emnlp-main)

Challenge:	Existing research to improve CoT efficiency falls into three categories, each with distinct limitations.
Approach:	They propose a training-free framework that addresses both dimensions of CoT reasoning by applying a progressive precision reduction strategy coupled with an entropy-based confidence mechanism for adaptive termination.
Outcome:	Empirical results show that the proposed framework achieves 11.3 efficiency gain without compromising accuracy.

Copied to clipboard

Xin Cheng, Shen Gao, Lemao Liu, Dongyan Zhao, Rui Yan

Machine Translation

Challenge:	Experimental results show that retrieval-augmented NMT model obtains substantial improvements over strong baselines in the benchmark dataset.
Approach:	They propose a retrieval-augmented NMT model that is holistically similar to the source sentence while individually contrastive to each other.
Outcome:	The proposed model improves on baselines in the translation task.

Towards Self-Evolving Agents: Enabling Autonomy through Interactive Experience Refinement (2026.findings-acl)

Copied to clipboard

Cheng Yang, Xuemeng Yang, Licheng Wen, Daocheng Fu, Jianbiao Mei, Rong Wu, Pinlong Cai, Yufan Shen, Nianchen Deng, Jia Xu, Botian Shi, Yu Qiao, Haifeng Li

Challenge:	Large Language Models struggle with complex, multi-step operational tasks because they remain static during inference and cannot learn from past experience.
Approach:	They propose a framework that organizes cross-domain insights to facilitate orchestration of long-horizon workflows.
Outcome:	The proposed framework outperforms existing methods on the TAC productivity benchmark and shows strong cross-task transferability.

Steering Away from Refusal: A Black-box Jailbreak Method Based on First-Token Distribution (2026.findings-acl)

Copied to clipboard

Shuangjie Fu, Du Su, Xin Chen, Fei Sun, Huawei Shen, Xueqi Cheng

Transductive Learning for Unsupervised Text Style Transfer (2021.emnlp-main)

Challenge:	Existing methods to analyze black-box jailbreaks lack direct optimization signals to refine adversarial prompts.
Approach:	They propose a distribution-jailbreak attack method that selects effective jailbreak templates and iteratively optimizes adversarial suffixes by maximizing the KL divergence from the standard refusal distribution.
Outcome:	The proposed method achieves state-of-the-art Attack Success Rate (ASR) on all tested open-source models and delivers over 94% ASR on GPT-4.1.

Copied to clipboard

Fei Xiao, Liang Pang, Yanyan Lan, Yan Wang, Huawei Shen, Xueqi Cheng

REALM: A Dataset of Real-World LLM Use Cases (2025.findings-acl)

Challenge:	Existing methods for style transfer are based on an inductive learning approach, which represents the style as embeddings, decoder parameters, or discriminator parameters and directly applies these general rules to the test cases.
Approach:	They propose a retrieval-based context-aware style representation that involves top-K relevant sentences in the target style in the transfer process.
Outcome:	The proposed method outperforms several strong baselines and is general and effective to the task of unsupervised style transfer.

Copied to clipboard

Jingwen Cheng, Kshitish Ghate, Wenyue Hua, William Yang Wang, Hong Shen, Fei Fang

PricingLogic: Evaluating LLMs Reasoning on Complex Tourism Pricing Tasks (2025.emnlp-main)

Challenge:	Existing studies on LLM adoption and their social implications lack empirical grounding, weakening their validity.
Approach:	They propose to integrate a dataset of over 94,000 LLM use cases collected from Reddit and news articles to provide insights into LLM adoption across different domains.
Outcome:	The proposed dataset includes over 94,000 LLM use cases collected from Reddit and news articles.

Copied to clipboard

Yunuo Liu, Dawei Zhu, Zena Al-Khalili, Dai Cheng, Yanjun Chen, Dietrich Klakow, Wei Zhang, Xiaoyu Shen

CPT-Agent: A Cognitive Process Theory-driven Framework for Student Simulation in Writing Development (2026.acl-long)

Challenge:	Recent advances in Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse domains, such as code generation, mathematical problem-solving, and general-purpose human instruction following.
Approach:	They propose to use large language models to process questions expressed in natural language to automate tourism-booking prices when multiple, overlapping farerules apply.
Outcome:	The proposed model can automate tourism-booking prices when multiple, overlapping farerules apply.

Copied to clipboard

Yuhan Chen, Zizhuo Shen, Miaomiao Cheng, Xu Han, Jiefu Gong, Shijin Wang, Wei Song

Linguistics & Cognitive NLP

Challenge:	Existing LLMs model overly capable learners who over-apply feedback, resulting in pedagogically implausible behavior.
Approach:	They propose a framework that decouples cognitive ability from writing proficiency and models their interaction during writing and revision.
Outcome:	The proposed model produces distinguishable proficiency levels and is consistent with instructional theories.

To Diff or Not to Diff? Structure-Aware and Adaptive Output Formats for Efficient LLM-based Code Editing (2026.findings-acl)

Copied to clipboard

Wei Cheng, Yongchang Cao, Chen Shen, Binhua Li, Jue Chen, Yongbin Li, Wei Hu

Programming Languages in NLP

Challenge:	Large Language Models (LLMs) are increasingly used for code editing, yet the full-code generation paradigm suffers from severe efficiency bottlenecks.
Approach:	They propose to use a structure-aware diff format to train LLMs to choose the most token-efficient format between a given diff format and full code.
Outcome:	The proposed approach matches the most token-efficient format with full-code generation while reducing latency and cost by over 30% on long-code editing tasks.

Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding (2023.findings-emnlp)

Copied to clipboard

Yu Zhang, Hao Cheng, Zhihong Shen, Xiaodong Liu, Ye-Yi Wang, Jianfeng Gao

Exploring Key Point Analysis with Pairwise Generation and Graph Partitioning (2024.naacl-long)

Challenge:	Pre-trained language models (LMs) have shown effectiveness in literature understanding tasks, especially when tuned via contrastive learning.
Approach:	They propose a multi-task contrastive learning framework that enables common knowledge sharing across different scientific literature understanding tasks while preventing task-specific skills from interfering with each other.
Outcome:	The proposed framework outperforms state-of-the-art pre-trained language models on a comprehensive dataset.

Copied to clipboard

Xiao Li, Yong Jiang, Shen Huang, Pengjun Xie, Gong Cheng, Fei Huang

Structured Data in NLP

Challenge:	Existing methods for key point analysis rely on semantic similarity instead of measuring the existence of shared key points .
Approach:	They propose a key point analysis approach with pairwise generation and graph partitioning to summarize arguments into a concise set of key points.
Outcome:	The proposed model surpasses existing models on ArgKP and QAM datasets.

SafetyQuizzer: Timely and Dynamic Evaluation on the Safety of LLMs (2025.naacl-long)

Copied to clipboard

Zhichao Shi, Shaoling Jing, Yi Cheng, Hao Zhang, Yuanzhuo Wang, Jie Zhang, Huawei Shen, Xueqi Cheng

CAN: Constrained Attention Networks for Multi-Aspect Sentiment Analysis (D19-1)

Challenge:	Large Language Models (LLMs) have been used to evaluate the safety of their users . however, evaluation questions in current benchmarks are too straightforward and difficult to update with practical relevance due to their lack of correlation with real-world events.
Approach:	They propose a question-generation framework to evaluate the safety of LLMs in the Chinese context.
Outcome:	The proposed framework reduces decline rate while maintaining similar attack success rate.

Copied to clipboard

Mengting Hu, Shiwan Zhao, Li Zhang, Keke Cai, Zhong Su, Renhong Cheng, Xiaowei Shen

Sentiment Analysis

Challenge:	Existing methods for aspect-specific sentiment classification are noisy and downgraded performance.
Approach:	They propose a constrained attention network to regularize attention for multi-aspect sentiment analysis by orthogonal regularization on multiple aspects and sparse regularization for each single aspect.
Outcome:	The proposed approach outperforms state-of-the-art methods on two public datasets and extends to multi-task settings.

The Mirage of Model Editing: Revisiting Evaluation in the Wild (2025.acl-long)

Copied to clipboard

Wanli Yang, Fei Sun, Jiajun Tan, Xinyu Ma, Qi Cao, Dawei Yin, Huawei Shen, Xueqi Cheng

Question Answering

Challenge:	despite near-perfect results, effectiveness of model editing in real-world applications remains unclear.
Approach:	They propose QAEdit and WILD to better reflect real-world use of model editing . they propose a benchmark aligned with widely used question answering datasets and a task-agnostic evaluation framework .
Outcome:	The proposed QAEdit benchmark and WILD evaluation framework show that current models perform worse than previously reported.

MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Models (2025.coling-main)

Copied to clipboard

Zihao Wei, Jingcheng Deng, Liang Pang, Hanxing Ding, Huawei Shen, Xueqi Cheng

Textual Enhanced Contrastive Learning for Solving Math Word Problems (2022.findings-emnlp)

Challenge:	Existing studies on knowledge editing focus on monolingual scenarios, neglecting the complexities presented by multilingual contexts and multi-hop reasoning.
Approach:	They propose a benchmark to evaluate the adaptability of multilingual knowledge editing methods.
Outcome:	The proposed benchmark evaluates the adaptability of multilingual knowledge editing methods across five languages.

Copied to clipboard

Yibin Shen, Qianying Liu, Zhuoyuan Mao, Fei Cheng, Sadao Kurohashi

Improving Multi-turn Dialogue Modelling with Utterance ReWriter (P19-1)

Challenge:	Recent studies show that current models rely on shallow heuristics to predict solutions . a textual Enhanced Contrastive Learning framework enforces the models to distinguish semantically similar examples while holding different mathematical logic.
Approach:	They propose a textual Enhanced Contrastive Learning framework which enforces models to distinguish semantically similar examples while holding different mathematical logic.
Outcome:	The proposed framework improves on benchmark and challenge datasets in English and Chinese.

Copied to clipboard

Hui Su, Xiaoyu Shen, Rongzhi Zhang, Fei Sun, Pengwei Hu, Cheng Niu, Jie Zhou

Paraphrasing

Challenge:	Recent research has achieved impressive results in single-turn dialogue modelling, but multi-turn models still remain challenging.
Approach:	They propose to rewrite human utterances as a pre-process to help multi-turn dialgoue modelling.
Outcome:	The proposed architecture achieves remarkably good performance on the utterance rewriting task.

MReD: A Meta-Review Dataset for Structure-Controllable Text Generation (2022.findings-acl)

Copied to clipboard

Chenhui Shen, Liying Cheng, Ran Zhou, Lidong Bing, Yang You, Luo Si

ComSearch: Equation Searching with Combinatorial Strategy for Solving Math Word Problems with Weak Supervision (2023.eacl-main)

Challenge:	a new text generation dataset is needed to controllable text summarization, but it lacks the domain knowledge.
Approach:	They propose to use existing text generation datasets to leverage input and control signals . they propose to annotate each meta-review sentence manually with a control signal .
Outcome:	The proposed method can be used to control the structure of a text generation dataset . it can be applied to a variety of tasks, including a task with a large number of meta-review sentences .

Copied to clipboard

Qianying Liu, Wenyu Guan, Jianhao Shen, Fei Cheng, Sadao Kurohashi

Numerical Reasoning

Challenge:	Existing weakly-supervised methods for solving math word problems are expensive and time-consuming.
Approach:	They propose a weakly-supervised approach to solve math word problems . they propose 'comsearch' algorithm which compresses the search space by excluding mathematically equivalent equations.
Outcome:	The proposed algorithm can compress the search space by excluding mathematically equivalent equations.

Do We Always Need Query-Level Workflows? Rethinking Agentic Workflow Generation for Multi-Agent Systems (2026.findings-acl)

Copied to clipboard

Zixu Wang, Bingbing Xu, Yige Yuan, Huawei Shen, Xueqi Cheng

Challenge:	Existing approaches generate workflows either at task level or query level, but their relative costs and benefits remain unclear.
Approach:	They propose a query-level workflow generation framework that generates tasks at task level and query level.
Outcome:	The proposed framework reduces token usage by up to 83% compared to existing approaches . it maintains competitive performance with an average degradation of just 0.61% compared with existing approaches across multiple datasets .

Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization (2023.findings-emnlp)

Copied to clipboard

Chenhui Shen, Liying Cheng, Xuan-Phi Nguyen, Yang You, Lidong Bing

ALiiCE: Evaluating Positional Fine-grained Citation Generation (2025.naacl-long)

Challenge:	ChatGPT and GPT-4 are popular as evaluation metric for complex generative tasks . however, they are not ready as human replacements due to significant limitations .
Approach:	They conduct extensive analysis to examine the stability and reliability of LLMs as automatic evaluators for abstractive summarization.
Outcome:	The proposed methods outperform the commonly used automatic metrics but are not ready for human evaluation due to significant limitations.

Copied to clipboard

Yilong Xu, Jinhua Gao, Xiaoming Yu, Baolong Bi, Huawei Shen, Xueqi Cheng

Challenge:	Existing research on citation generation is limited to sentence-level statements . positional fine-grained citations can appear anywhere within sentences .
Approach:	They propose a framework that allows LLMs to generate citations from sentences . they use dependency tree-based methods to parse sentence-level claims into atomic claims .
Outcome:	The proposed framework evaluates citation quality using three metrics including positional fine-grained citation recall, precision, and coefficient of variation of citation positions.

Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence (2020.acl-main)

Copied to clipboard

Xiaoyu Shen, Ernie Chang, Hui Su, Cheng Niu, Dietrich Klakow

Focusing Condition: Inference-Time Self-Contrastive Steering Elicits Better Conditional Text Embeddings in LLMs (2026.acl-long)

Challenge:	Recent neural attention models conflate all steps into a single end-to-end system and simplify training process.
Approach:	They propose to explicitly segment target text into fragment units and align them with their data correspondences.
Outcome:	The proposed model outperforms neural attention models on E2E and WebNLG benchmarks.

Copied to clipboard

Zifeng Cheng, Lingyun Qian, Zhiwei Jiang, Cong Wang, Yafeng Yin, Fei Shen, Ao Zhou, Qing Gu

Think Before You Speak: Cultivating Communication Skills of Large Language Models via Inner Monologue (2024.findings-naacl)

Challenge:	Existing methods for extracting conditional text embeddings from large language models (LLMs) relying on prompts often fails to produce high-quality conditional embeddables, resulting in degradation of quality.
Approach:	They propose a plug-and-play method that constructs unconditional general text embeddings and uses them to refine conditional text embeds.
Outcome:	The proposed method improves performance of prompt-based methods on clustering, Semantic Textual Similarity, and triplet alignment datasets.

Copied to clipboard

Junkai Zhou, Liang Pang, Huawei Shen, Xueqi Cheng

Seeking Diverse Reasoning Logic: Controlled Equation Expression Generation for Solving Math Word Problems (2022.aacl-short)

Challenge:	emergence of large language models (LLMs) improves capabilities of dialogue systems . but they lack communication skills, which make them more like information seeking tools .
Approach:	They propose to empower LLMs with communication skills through inner monologues . they use a benchmark to evaluate the dialogue generation ability of the model .
Outcome:	The proposed model outperforms the baselines in the evaluation of communication skills.

Copied to clipboard

Yibin Shen, Qianying Liu, Zhuoyuan Mao, Zhen Wan, Fei Cheng, Sadao Kurohashi

Evaluating Memory Capability in Continuous Lifelog Scenario (2026.findings-acl)

Challenge:	Existing methods to solve Math Word Problems rely on human annotation . empirical results suggest that our method universally improves the performance on single-unknown and multiple-un unknown benchmarks.
Approach:	They propose a controlled equation generation solver by leveraging a set of control codes to guide the model to consider certain reasoning logic and decode the corresponding equations expressions transformed from the human reference.
Outcome:	The proposed method improves performance on single-unknown and multiple-un unknown benchmarks with 13.2% accuracy on the challenging multiple-unequal datasets.

Copied to clipboard

Jianjie Zheng, Zhichen Liu, Zhanyu Shen, Jingxiang Qu, Guanhua Chen, Yile Wang, Yang Xu, Yang Liu, Sijie Cheng

Challenge:	Existing benchmarks focus on online one-on-one chatting or human-AI interactions, neglecting real-world scenarios.
Approach:	They propose a framework to curate a lifelog benchmark that combines two subsets of audio data to address temporal leakage in offline settings.
Outcome:	The proposed framework outperforms existing benchmarks on live chats and AI interactions.

TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities (2023.acl-demo)

Copied to clipboard

Zhe Zhao, Yudong Li, Cheng Hou, Jing Zhao, Rong Tian, Weijie Liu, Yiren Chen, Ningyuan Sun, Haoyan Liu, Weiquan Mao, Han Guo, Weigang Gou, Taiqiang Wu, Tao Zhu, Wenhang Shi, Chen Chen, Shan Huang, Sihong Chen, Liqun Liu, Feifei Li, Xiaoshuai Chen, Xingwu Sun, Zhanhui Kang, Xiaoyong Du, Linlin Shen, Kimmo Yan

A Set Prediction Network For Extractive Summarization (2023.findings-acl)

Challenge:	Several pre-training models of different modalities are showing a rising trend of homogeneity in their model structures.
Approach:	They propose a toolkit that supports pre-training models of different modalities.
Outcome:	The proposed toolkit can match the performance of the original implementations on text, vision, and audio benchmarks.

Copied to clipboard

Xiaoxia Cheng, Yongliang Shen, Weiming Lu

PRDetect: Perturbation-Robust LLM-generated Text Detection Based on Syntax Tree (2025.findings-naacl)

Challenge:	Recent approaches to extracting salient sentences from source document are naive and lack dependencies between sentences.
Approach:	They propose a set prediction network to detect redundancy relationship between sentences . they use a non-autoregressive decoder to predict sentences in parallel .
Outcome:	The proposed method outperforms previous state-of-the-art models on extracted summary datasets.

Copied to clipboard

Xiang Li, Zhiyi Yin, Hexiang Tan, Shaoling Jing, Du Su, Yi Cheng, Huawei Shen, Fei Sun

Syntactic Text Processing

Challenge:	Recent methods for detecting LLM-generated text have shown impressive performance, but in real-world scenarios, users often introduce perturbations to the text.
Approach:	They propose a method that detects syntactic trees that are minimally affected by perturbations and exhibit distinct differences between human-written and LLM-generated text.
Outcome:	The proposed method shows that it is significantly better against perturbations on the HC3 and GPT-3.5-mixed datasets and also has the shortest time expenditure.

System Prompt Hijacking via Permutation Triggers in LLM Supply Chains (2025.findings-acl)

Copied to clipboard

Lu Yan, Siyuan Cheng, Xuan Chen, Kaiyuan Zhang, Guangyu Shen, Xiangyu Zhang

BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrieval (2023.acl-long)

Challenge:	Large language models (LLMs) are rapidly transforming the landscape of artificial intelligence due to the substantial resources required for training.
Approach:	They propose a post-deployment attack that bypasses system prompts to compromise models . they introduce Precise Activation Guarding and Unit Deviation Sampling to protect against attack .
Outcome:	The proposed attack bypasses system prompts, enabling unrestricted model outputs and safety violations.

Copied to clipboard

Shicheng Xu, Liang Pang, Huawei Shen, Xueqi Cheng

Information Retrieval

Challenge:	Dense retrieval has shown promise in the first-stage retrieval process when trained on in-domain labeled datasets.
Approach:	They propose a method to capture matching signal to improve generalization of dense retrieval by capturing matching signal between two texts.
Outcome:	The proposed method can be combined with different training methods to improve generalization ability without additional inference overhead and target domain data.

DP3: Differentially Private Prompt Perturbation for Multi-turn LLM Inference (2026.findings-acl)

Copied to clipboard

Lele Zheng, Chao Zhang, Feiyang Yuan, Ke Cheng, Tao Zhang, Anxiao Song, Yulong Shen

Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Models (2025.emnlp-main)

Challenge:	Large language models (LLMs) are widely used for text understanding and generation . existing methods that assume single-turn interactions break down in multi-turn settings .
Approach:	They propose a differentially private prompt perturbation framework for multi-turn LLM inference . DP3 constructs a perturbation mapping table to reuse perturbations for recurring tokens .
Outcome:	The proposed framework reduces privacy costs and degrades cross-turn semantic coherence . it also provides a context-aware utility function to maintain semantic consistency across turns .

Copied to clipboard

Yilong Xu, Jinhua Gao, Xiaoming Yu, Yuanhai Xue, Baolong Bi, Huawei Shen, Xueqi Cheng

Visual Zero-Shot E-Commerce Product Attribute Value Extraction (2025.naacl-industry)

Challenge:	Utility-based retrieval has emerged as a promising topic for downstream tasks . however, capturing passage utility accurately remains unexplored due to insufficient understanding .
Approach:	They propose a framework for training utility-based retrievers in Retrieval-Augmented Language Models . it incorporates multi-task generalization and inter-passage interaction to improve performance .
Outcome:	The proposed framework improves performance on ten datasets across different tasks.

Copied to clipboard

Jiaying Gong, Ming Cheng, Hongda Shen, Pierre-Yves Vandenbussche, Janet Jenq, Hoda Eldardiry

ToolCoder: A Systematic Code-Empowered Tool Learning Framework for Large Language Models (2025.acl-long)

Challenge:	Existing zero-shot product attribute value extraction approaches require sellers to manually provide product descriptions.
Approach:	They propose a cross-modal zero-shot attribute value generation framework based on CLIP that uses product images as inputs for zero- shot inference.
Outcome:	The proposed framework significantly outperforms other vision-language models for zero-shot attribute value extraction.

Copied to clipboard

Hanxing Ding, Shuchang Tao, Liang Pang, Zihao Wei, Jinyang Gao, Bolin Ding, Huawei Shen, Xueqi Cheng

Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation (2024.acl-long)

Challenge:	Existing approaches to tool learning rely on hand-crafted prompts and natural language reasoning, making multi-step planning difficult and lacking precise error diagnosis and reflection mechanisms.
Approach:	They propose a framework that reformulates tool learning as a code generation task.
Outcome:	The proposed framework achieves superior performance in task completion accuracy and execution reliability compared to existing approaches.

Copied to clipboard

Shicheng Xu, Liang Pang, Mo Yu, Fandong Meng, Huawei Shen, Xueqi Cheng, Jie Zhou

A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models (2022.acl-long)

Challenge:	Existing studies show that LLMs face challenges in effectively using retrieved information . authors propose a method that considers LLM as "Information Refiner"
Approach:	They propose a method that considers LLMs as "Information Refiners" they propose INFO-RAG, which is low-cost and general across various tasks .
Outcome:	The proposed method improves performance of LLaMA2 by 9.39% relative points . it is low-cost and general across various tasks, and is robust and in-context learning is possible .

Copied to clipboard

Woojeong Jin, Yu Cheng, Yelong Shen, Weizhu Chen, Xiang Ren

MEIC-DT: Memory-Efficient Incremental Clustering for Long-Text Coreference Resolution with Dual-Threshold Constraints (2026.findings-acl)

Challenge:	Recent few-shot learning models such as GPT3 are expensive and slow to deploy for real-world applications.
Approach:	They propose a prompt-based low-resource learning method for VL tasks with a few examples . they pre-train a sequence-to-sequence transformer model with prefix and masked language modeling .
Outcome:	The proposed method outperforms Frozen on vision-language tasks with prompt-based learning by 18.2% point.

Copied to clipboard

Kangyang Luo, Shuzheng Si, Yuzhuo Bai, Cheng Gao, Zhitong Wang, Cheng Huang, Yingli Shen, Yufeng Han, Wenhao Li, Cunliang Kong, Maosong Sun

CODEMENV: Benchmarking Large Language Models on Code Migration (2025.findings-acl)

Challenge:	Existing supervised neural methods are underexplored for coreference resolution, especially in incremental clustering.
Approach:	They propose a dual-threshold incremental clustering approach based on a lightweight Transformer.
Outcome:	Experiments on common benchmarks show that MEIC-DT achieves highly competitive coreference performance under stringent memory constraints.

Copied to clipboard

Keyuan Cheng, Xudong Shen, Yihao Yang, TengyueWang TengyueWang, Yang Cao, Muhammad Asif Ali, Hanbin Wang, Lijie Hu, Di Wang

Qsnail: A Questionnaire Dataset for Sequential Question Generation (2024.lrec-main)

Challenge:	Large language models (LLMs) have demonstrated remarkable proficiency in handling a wide range of tasks within the software engineering domain, but their ability to perform code migration—adapting code to different environments—remains underexplored.
Approach:	They propose a benchmark to evaluate large language models’ performance in handling code migration tasks.
Outcome:	The proposed benchmark comprises 922 data points across 19 Python and Java packages and offers three tasks to systematically evaluate code migration: identifying version-incompatible functions, determining function changes, and adapting code to target environments.

Copied to clipboard

Yan Lei, Liang Pang, Yuanzhuo Wang, Huawei Shen, Xueqi Cheng

MovieChats: Chat like Humans in a Closed Domain (2020.emnlp-main)

Challenge:	Questionnaires are a professional research methodology used for qualitative and quantitative analysis of human opinions, preferences, and behaviors.
Approach:	They propose a questionnaire-based dataset that consists of 13,168 human-written questionnaires.
Outcome:	The proposed dataset contains 13,168 human-written questionnaires gathered from online platforms.

Copied to clipboard

Hui Su, Xiaoyu Shen, Zhou Xiao, Zheng Zhang, Ernie Chang, Cheng Zhang, Cheng Niu, Jie Zhou

Natural Language Interfaces

Challenge:	Currently, open-domain chatbots are far from satisfactory.
Approach:	They propose a unified, readily scalable neural approach which reconciles all subtasks like intent prediction and knowledge retrieval.
Outcome:	The proposed approach outperforms commercial systems replying on complex rules on static and interactive tests and shows that the results are remarkably good.

Dialogue Summarization with Static-Dynamic Structure Fusion Graph (2023.acl-long)

Copied to clipboard

Shen Gao, Xin Cheng, Mingzhe Li, Xiuying Chen, Jinpeng Li, Dongyan Zhao, Rui Yan

Natural-Language Policies to Executable Decisions: An Interpretable Large Language Model Framework (2026.acl-industry)

Challenge:	Dialogue summarization is a challenging task since it has dynamic interaction nature and inconsistent information flow among various speakers.
Approach:	They propose a Static-Dynamic graph-based Dialogue Summarization model which fuses prior knowledge from human expertise and adaptively learns the graph structure in an end-to-end learning fashion.
Outcome:	The proposed model can help people capture the highlights of a semi-structured and multi-participant dialogue without reviewing the complex dialogue context.

Copied to clipboard

Ziqiang Zhang, Jing Ma, Zilong Wang, Jiayuan Chen, Yi Qiao, Yu He, Wei Zhang, Dai Cheng, Xiaoyu Shen

SentBS: Sentence-level Beam Search for Controllable Summarization (2022.emnlp-main)

Challenge:	a production-grade pricing system for tourism is challenging due to unstructured nature of travel orders and ever-evolving pricing policies.
Approach:	They propose a production-grade pricing system with a strict decision boundary . they propose to combine structured extraction and bounded policy/path selection with interpretable condition trees .
Outcome:	The proposed system processed 3,960 orders in six months and reduced the order management team from 15-20 to 3 . the system reduced the per-order handling time from 10 minutes to 2 minutes.

Copied to clipboard

Chenhui Shen, Liying Cheng, Lidong Bing, Yang You, Luo Si

Diversifying Dialogue Generation with Non-Conversational Text (2020.acl-main)

Challenge:	Structure-controlled summarization is a useful and interesting research direction . current structure-controlling methods have limited effectiveness in enforcing the desired structure.
Approach:	They propose a sentence-level beam search generation method to select suitable sentences for subsequent generations.
Outcome:	The proposed method significantly reduces structural discrepancies by 68% on a structure-controlled dataset.

Copied to clipboard

Hui Su, Xiaoyu Shen, Sanqiang Zhao, Zhou Xiao, Pengwei Hu, Randy Zhong, Cheng Niu, Jie Zhou

Dialogue Systems & Conversational Agents

Challenge:	Neural network-based sequence-to-sequence models suffer from low diversity in open-domain dialogue generation.
Approach:	They propose a way to diversify dialogue generation by leveraging non-conversational text . they collect large-scale corpus from forum comments, idioms and book snippets .
Outcome:	The proposed model produces significantly more diverse responses without sacrificing relevance with context.

Multi-View Reasoning: Consistent Contrastive Learning for Math Word Problem (2022.findings-emnlp)

Copied to clipboard

Wenqi Zhang, Yongliang Shen, Yanna Ma, Xiaoxia Cheng, Zeqi Tan, Qingpeng Nong, Weiming Lu

Meta-CQG: A Meta-Learning Framework for Complex Question Generation over Knowledge Bases (2022.coling-1)

Challenge:	Existing methods for generating complex semantics and diverse equations are limited by a fixed view.
Approach:	They propose a multi-view consistent contrastive learning approach that decouples human reasoning into two independent but consistent views.
Outcome:	The proposed approach significantly outperforms existing baselines on complex problems on multiple languages.

Copied to clipboard

Kun Zhang, Yunqi Qiu, Yuanzhuo Wang, Long Bai, Wei Li, Xuhui Jiang, Huawei Shen, Xueqi Cheng

Bootstrapping Code Translation with Weighted Multilanguage Exploration (2026.acl-long)

Challenge:	Existing methods train one encoder-decoder-based model to fit all questions . however, such a one-size-fits-all strategy may not perform well for complex questions involving multiple KB relations or functional constraints.
Approach:	They propose a meta-learning framework for complex question generation over knowledge bases . they propose he meta-trained generator can acquire universal meta-knowledge .
Outcome:	The proposed framework can acquire universal and transferable meta-knowledge and quickly adapt to long-tailed samples under different dimensions.

Copied to clipboard

Yuhan Wu, Huan Zhang, Wei Cheng, Chen Shen, Jingyue Yang, Wei Hu

Multilinguality

Challenge:	Existing methods to improve code translation depend on abundant parallel code of high quality, which may not always be available.
Approach:	They propose a method that leverages functional invariance and cross-lingual portability of test suites to serve as universal verification oracles for multilingual reinforcement learning.
Outcome:	The proposed method leverages functional invariance and cross-lingual portability of test suites to serve as universal verification oracles for multilingual reinforcement learning (RL) training.

Jailbreak LLMs through Internal Stance Manipulation (2025.emnlp-main)

Copied to clipboard

Shuangjie Fu, Du Su, Beining Huang, Fei Sun, Jingang Wang, Wei Chen, Huawei Shen, Xueqi Cheng

GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion (2025.findings-acl)

Challenge:	Existing approaches to exploit LLMs' inherent safety mechanism, including GCG and AutoDAN, are ineffective for certain malicious requests.
Approach:	They propose a method that generates jailbreak prompts to suppress a refusal stance and induce affirmative responses by modifying adversarial prompts.
Outcome:	The proposed method outperforms the best baseline approach in Llama-2-7b-chat and achieves a 92.2% success rate across all models.

Copied to clipboard

Kangyang Luo, Yuzhuo Bai, Cheng Gao, Shuzheng Si, Zhu Liu, Yingli Shen, Zhitong Wang, Cunliang Kong, Wenhao Li, Yufei Huang, Ye Tian, Xuantang Xiong, Lei Han, Maosong Sun

ModelScope-Agent: Building Your Customizable Agent System with Open-source Large Language Models (2023.emnlp-demo)

Challenge:	Existing knowledge graphs lack the ability to integrate structural information into LLMs and output predictions deterministically.
Approach:	They propose a method which encodes structural information of KGs and merges it with LLMs to enhance KGC performance.
Outcome:	The proposed method improves the performance of KG Completion datasets on KGs by integrating structural information with LLMs.

Copied to clipboard

Chenliang Li, He Chen, Ming Yan, Weizhou Shen, Haiyang Xu, Zhikai Wu, Zhicheng Zhang, Wenmeng Zhou, Yingda Chen, Chen Cheng, Hongzhu Shi, Ji Zhang, Fei Huang, Jingren Zhou

Enhancing Training Data Attribution for Large Language Models with Fitting Error Consideration (2024.emnlp-main)

Challenge:	Large language models (LLMs) have demonstrated remarkable capabilities to comprehend human intentions, engage in reasoning, and design planning-like behavior.
Approach:	They propose a framework that equips large language models with tool-use capabilities . they propose LLaMA and Chat-GLM as controllers, and a model-based agent framework .
Outcome:	The proposed framework equips open-source LLMs with tool-use capabilities . it provides a user-friendly system library with a customizable engine design .

Copied to clipboard

Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng

Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation (2025.coling-main)

Challenge:	Large language models (LLMs) are difficult to interpret due to their black-box nature and randomness.
Approach:	They propose a new method which enhances influence functions by addressing fitting errors by eliminating knowledge bias present in the base model before fine-tuning.
Outcome:	The proposed method outperforms existing methods and achieves an average AUC of 91.64%.

Copied to clipboard

Jia Gu, Liang Pang, Huawei Shen, Xueqi Cheng

Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs (2025.emnlp-main)

Challenge:	LLMs are used to emulate sequential decision-making processes of humans . however, their ability to perform probabilistic sampling is limited .
Approach:	They propose to use large language models (LLMs) as agents to emulate the sequential decision-making processes of humans represented as Markov decision-makers (MDPs).
Outcome:	The proposed models can understand probabilities, but struggle with sampling precision . integrating coding tools can improve sampling precision, but this level of sampling precision still makes it difficult to simulate human behavior as agents.

Copied to clipboard

Hexiang Tan, Fei Sun, Sha Liu, Du Su, Qi Cao, Xin Chen, Jingang Wang, Xunliang Cai, Yuanzhuo Wang, Huawei Shen, Xueqi Cheng

Challenge:	Existing detection methods fail to account for self-consistent error . study identifies self-consistency errors and evaluates them .
Approach:	They propose a method that fuses hidden state evidence from an external verifier LLM to detect self-consistent errors.
Outcome:	The proposed method significantly enhances performance on self-consistent errors across three LLM families.

ImCoref-CeS: An Improved Lightweight Pipeline for Coreference Resolution with LLM-based Checker-Splitter Refinement (2026.acl-long)

Copied to clipboard

Kangyang Luo, Yuzhuo Bai, Shuzheng Si, Cheng Gao, Zhitong Wang, Yingli Shen, Wenhao Li, Zhu Liu, Yufeng Han, Jiayi Wu, Cunliang Kong, Maosong Sun

Simple or Complex? Complexity-controllable Question Generation with Soft Templates and Deep Mixture of Experts Model (2021.findings-emnlp)

Challenge:	Existing supervised neural methods for coreference resolution are underexplored . current methods rely on small language models, but their potential is underexploited .
Approach:	They propose a framework that integrates an enhanced supervised model with LLM-based reasoning.
Outcome:	The proposed method surpasses existing state-of-the-art methods in coreference resolution.

Copied to clipboard

Sheng Bi, Xiya Cheng, Yuan-Fang Li, Lizhen Qu, Shirong Shen, Guilin Qi, Lu Pan, Yinlin Jiang

UnitedQA: A Hybrid Approach for Open Domain Question Answering (2021.acl-long)

Challenge:	Existing work on complex questions does not consider controlling complexity of generated questions.
Approach:	They propose an end-to-end neural complexity-controllable question generation model that incorporates a mixture of experts as the selector of soft templates to capture question similarity while avoiding the expensive construction of actual templates.
Outcome:	The proposed model is superior to state-of-the-art methods in both automatic and manual evaluations on two benchmark QA datasets.

Copied to clipboard

Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao

Natural Language Interfaces

Challenge:	Recent work on open-domain question answering focuses on either extractive or generative readers exclusively.
Approach:	They propose a hybrid approach to extractive and generative readers that leverages both models.
Outcome:	The proposed approach outperforms state-of-the-art models on NaturalQuestions and TriviaQA respectively.

LLMDet: A Third Party Large Language Models Generated Text Detection Tool (2023.findings-emnlp)

Copied to clipboard

Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng, Tat-Seng Chua

Learning Global Controller in Latent Space for Parameter-Efficient Fine-Tuning (2024.acl-long)

Challenge:	Existing detection tools rely on access to LLMs and can only distinguish between machine-generated and human-authored text.
Approach:	They propose a model-specific, secure, efficient, and extendable detection tool that can source text from specific LLMs.
Outcome:	The proposed tool can source text from specific LLMs, such as GPT-2, OPT, LLaMA, and others.

Copied to clipboard

Zeqi Tan, Yongliang Shen, Xiaoxia Cheng, Chang Zong, Wenqi Zhang, Jian Shao, Weiming Lu, Yueting Zhuang

Green & Sustainable NLP

Challenge:	Large language models (LLMs) have shown remarkable performance, but their training costs are exorbitant.
Approach:	They propose a parameter-efficient method for exploring optimal solutions within latent space by using latent units to extract input representations from LLMs.
Outcome:	The proposed method improves performance on a range of natural language processing tasks.

Query-based Instance Discrimination Network for Relational Triple Extraction (2022.emnlp-main)

Copied to clipboard

Zeqi Tan, Yongliang Shen, Xuming Hu, Wenqi Zhang, Xiaoxia Cheng, Weiming Lu, Yueting Zhuang

When Retriever-Reader Meets Scenario-Based Multiple-Choice Questions (2021.findings-emnlp)

Challenge:	Recent approaches to extract relational triples from open domain texts suffer from error propagation, relation redundancy and lack of high-level connections.
Approach:	They propose a query-based approach to construct instance-level representations for relational triples . they use query embeddings and token embeddables to extract all types of triples in one step .
Outcome:	The proposed method achieves state-of-the-art on five widely used benchmarks.

Copied to clipboard

ZiXian Huang, Ao Wu, Yulin Shen, Gong Cheng, Yuzhong Qu

Information Retrieval

Challenge:	Scenario-based question answering (SQA) requires retrieving and reading paragraphs from a large corpus to answer a question contextualized by a long scenario description.
Approach:	They propose a model where the retriever is implicitly supervised only using QA labels via a novel word weighting mechanism.
Outcome:	The proposed model outperforms strong baselines on multiple-choice questions in three datasets.

MacLaSa: Multi-Aspect Controllable Text Generation via Efficient Sampling from Compact Latent Space (2023.findings-emnlp)

Copied to clipboard

Hanxing Ding, Liang Pang, Zihao Wei, Huawei Shen, Xueqi Cheng, Tat-Seng Chua

Green & Sustainable NLP

Challenge:	Existing approaches to multi-aspect controllable text generation require expensive iteration / searching within the discrete text space during the decoding stage, resulting in a degradation of text quality due to the domain discrepancies between different aspects.
Approach:	They propose a framework that estimates compact latent space for multiple aspects and performs efficient Sampling with a fast sampler to eliminate domain discrepancies.
Outcome:	The proposed framework outperforms baselines on attribute relevance and textual quality while maintaining a high inference speed.

SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation (2023.acl-long)

Copied to clipboard

Junkai Zhou, Liang Pang, Huawei Shen, Xueqi Cheng

Dialogue Systems & Conversational Agents

Challenge:	Existing work on large-scale corpora-based language models is limited and hard to generalize to all types of pre-trained language models.
Approach:	They propose a two-stage SimOAP strategy that over-samples and post-evaluates large-scale responses from existing models and selects a good response based on multiple evaluation metrics.
Outcome:	The proposed strategy outperforms baseline and automatic evaluation strategies in both automatic and human evaluations.

AnchorMem: Anchored Facts with Associative Contexts for Building Memory in Large Language Models (2026.findings-acl)

Copied to clipboard

Zhanyu Shen, Sijie Cheng, Zhicheng Guo, Weiqin Wang, Yile Wang, Hui Huang

Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification (2026.acl-long)

Challenge:	Existing memory systems rely on summarization to preserve contextual nuances and obscuring key retrieval features.
Approach:	They propose a method that decouples the retrieval unit from the generation context.
Outcome:	The proposed method outperforms baseline models on the LoCoMo benchmark.

Copied to clipboard

Zenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng

Challenge:	Large language models (LLMs) exhibit exceptional performance but pose inherent risks of generating toxic content.
Approach:	They propose a method that removes toxic subspaces from FFN parameters . they propose to use a lightweight method to eliminate toxic subespaces .
Outcome:	The proposed method achieves SOTA detoxification while preserving general capabilities without large-scale retraining.

Multimodal Instruction Tuning with Conditional Mixture of LoRA (2024.acl-long)

Copied to clipboard

Ying Shen, Zhiyang Xu, Qifan Wang, Yu Cheng, Wenpeng Yin, Lifu Huang

Zenghao Duan, Wenbin Duan, Zhiyi Yin, Yinghan Shen, Shaoling Jing, Jie Zhang, Huawei Shen, Xueqi Cheng

Challenge:	Multimodal Large Language Models (MLLMs) have demonstrated proficiency in diverse tasks across different domains.
Approach:	They propose a method that integrates multimodal instruction tuning with Conditional Mixture-of-LoRA.
Outcome:	Experimental results show that MixLoRA outperforms LoRA with the same or higher ranks . MLLMs have demonstrated remarkable proficiency in diverse tasks across domains .

Copied to clipboard

Low-Entropy Watermark Detection via Bayes’ Rule Derived Detector (2025.findings-acl)

Challenge:	Existing knowledge editing methods struggle when tasked with editing multiple related knowledge pieces for the same subject.
Approach:	They propose a benchmark to assess the effectiveness of knowledge editing methods . they use same-subject edits to ensure comprehensive updates to entity-centric knowledge .
Outcome:	The proposed method over-relys on subject information, neglecting other critical factors, resulting in reduced editing effectiveness.

Copied to clipboard

Beining Huang, Du Su, Fei Sun, Qi Cao, Huawei Shen, Xueqi Cheng

Numerical Reasoning

Challenge:	Existing methods for text watermarking ignore strong evidences embedded in low-entropy tokens, causing statistical measures to falsely indicate the absence of a watermark.
Approach:	They propose a Bayes' Rule derived watermark Detector which exploits watermark information from every token by leveraging the posterior probability of watermark’s presence.
Outcome:	The proposed method achieves 50% and 70% relative improvements over baselines in code generation and math problem-solving tasks.

TARGA: Targeted Synthetic Data Generation for Practical Reasoning over Structured Data (2025.acl-long)

Copied to clipboard

Xiang Huang, Jiayu Shen, Shanshan Huang, Sitao Cheng, Xiaxia Wang, Yuzhong Qu

GeoSQA: A Benchmark for Scenario-based Question Answering in the Geography Domain at High School Level (D19-1)

Challenge:	Existing methods for semantic parsing rely on extensive manually annotated datasets and limited generalization capability to unseen examples.
Approach:	They propose a framework that generates high-relevance synthetic data without manual annotation . they generate queries for the queries and use them as demonstrations for in-context learning .
Outcome:	The proposed framework outperforms non-fine-tuned methods on KBQA datasets and shows superior sample efficiency, robustness, and generalization capabilities under non-I.I.D. settings.

Copied to clipboard

Zixian Huang, Yulin Shen, Xiao Li, Yu’ang Wei, Gong Cheng, Lin Zhou, Xinyu Dai, Yuzhong Qu

Question Answering

Challenge:	SQA is an emerging application of NLP in the medical, geography, and legal domains.
Approach:	They propose a dataset of 1,981 scenarios and 4,110 multiple-choice questions in geography domain at high school level.
Outcome:	The proposed dataset consists of 1,981 scenarios and 4,110 multiple-choice questions in the geography domain at high school level.

Improving Disentangled Text Representation Learning with Information-Theoretic Guidance (2020.acl-main)

Copied to clipboard

Pengyu Cheng, Martin Renqiang Min, Dinghan Shen, Christopher Malon, Yizhe Zhang, Yitong Li, Lawrence Carin

Representation Learning

Challenge:	Disentangled representation learning (DRL) maps different aspects of data into distinct and independent low-dimensional latent vector spaces.
Approach:	They propose a method that manifests disentangled representations of text without supervision on semantics by minimizing the upper bound between style and content.
Outcome:	The proposed method improves on conditional text generation and text-style transfer tasks and improves style preservation.

SeaLLMs - Large Language Models for Southeast Asia (2024.acl-demos)

Copied to clipboard

Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Zhiqiang Hu, Chenhui Shen, Yew Ken Chia, Xingxuan Li, Jianyu Wang, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, Lidong Bing

Vision-Flan: Scaling Human-Labeled Tasks in Visual Instruction Tuning (2024.findings-acl)

Challenge:	Existing large language models favor high-resource languages, such as English, at the expense of low-resourced and regional languages.
Approach:	They propose a series of language models that specifically focuses on Southeast Asian languages.
Outcome:	SeaLLM models outperform ChatGPT-3.5 in non-Latin languages by large margins . linguistic disparity impedes access to state-of-the-art AI technologies for non-English-speaking populations .

Copied to clipboard

Zhiyang Xu, Chao Feng, Rulin Shao, Trevor Ashby, Ying Shen, Di Jin, Yu Cheng, Qifan Wang, Lifu Huang

Does DetectGPT Fully Utilize Perturbation? Bridging Selective Perturbation to Fine-tuned Contrastive Learning Detector would be Better (2024.acl-long)

Challenge:	Recent vision-language models (VLMs) have shown impressive capabilities as general visual assistants, but there are two challenges to their performance: (1) lacking task diversity in pretraining and visual instruction tuning; (2) annotation error and bias in GPT-4 synthesized instruction tuning data.
Approach:	They propose a two-stage instruction tuning framework that fine tunes VLMs firstly and further tuned on GPT-4 synthesized data.
Outcome:	The proposed framework outperforms the traditional single-stage visual instruction tuning framework and achieves state-of-the-art performance across a wide range of multi-modal evaluation benchmarks.

Copied to clipboard

Shengchao Liu, Xiaoming Liu, Yichen Wang, Zehua Cheng, Chengzhengxu Li, Zhaohan Zhang, Yu Lan, Chao Shen

The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis (2026.acl-long)

Challenge:	Existing methods to detect MGT from human-written texts are inadequate . existing methods are fine-tuned and zero-shot metric-based, but they can be more accurate.
Approach:	They propose a novel fine-tuned detector that can detect MGT from human-written texts by contrastive learning on selective perturbation.
Outcome:	The proposed method outperforms the state-of-the-art by 1.20% on four public datasets.

Copied to clipboard

Zihao Wei, Liang Pang, Jiahao Liu, Wenjie Shi, Jingcheng Deng, Shicheng Xu, Zenghao Duan, Jingang Wang, Fei Sun, Huawei Shen, Xueqi Cheng

QueryAgent: A Reliable and Efficient Reasoning Framework with Environmental Feedback based Self-Correction (2024.acl-long)

Challenge:	Explicit reasoning trajectories increase performance but often trigger overthinking . despite its importance, this study examines how each step of reasoning affects the final outcome .
Approach:	They propose a Reasoning Completion Point Detector that detects the RCP by monitoring rank dynamics of termination tokens.
Outcome:	The proposed method reduces token usage by up to 44% while preserving accuracy.

Copied to clipboard

Xiang Huang, Sitao Cheng, Shanshan Huang, Jiayu Shen, Yong Xu, Chaoyun Zhang, Yuzhong Qu

Adaptive Information Seeking for Open-Domain Question Answering (2021.emnlp-main)

Challenge:	Existing methods for semantic parsing fail when hallucinations are encountered . QueryAgent solves a question step-by-step and performs stepwise self-correction .
Approach:	They propose a framework that solves a query step-by-step and performs stepwise self-correction.
Outcome:	The proposed framework outperforms existing methods on GrailQA and GraphQ by 5.7 and 15.0 points.

Copied to clipboard

Yunchang Zhu, Liang Pang, Yanyan Lan, Huawei Shen, Xueqi Cheng

Natural Language Interfaces

Challenge:	Existing iterative approaches to open-domain question answering use predefined strategies . e.g., BM25, DPR, and hyperlink are defined as actions .
Approach:	They propose a novel adaptive information-seeking strategy for open-domain question answering . they propose to use a partially observed Markov decision process to select a proper retrieval action .
Outcome:	Experiments on SQuAD Open and HotpotQA fullwiki show that AISO outperforms baseline methods with predefined strategies in retrieval and answer evaluations.

Following the Autoregressive Nature of LLM Embeddings via Compression and Alignment (2025.emnlp-main)

Copied to clipboard

Jingcheng Deng, Zhongtao Jiang, Liang Pang, Zihao Wei, Liwei Chen, Kun Xu, Yang Song, Huawei Shen, Xueqi Cheng

ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees (2024.findings-emnlp)

Challenge:	Experimental results demonstrate that our method significantly outperforms traditional contrastive learning approaches when using the same amount of data.
Approach:	They propose a new contrastive learning method built on embedding conditional probability distributions that integrates two tasks: information compression and conditional distribution alignment.
Outcome:	The proposed method outperforms traditional contrastive learning approaches and achieves comparable performance to state-of-the-art models when using the same amount of data.

Copied to clipboard

Zhiyuan Wang, Jinhao Duan, Lu Cheng, Yue Zhang, Qingni Wang, Xiaoshuang Shi, Kaidi Xu, Heng Tao Shen, Xiaofeng Zhu

A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization (2023.findings-emnlp)

Challenge:	Uncertainty quantification (UQ) in natural language generation tasks remains an open challenge . however, black-box uncertainty measures require investigating with the proliferation of LLMs served via APIs.
Approach:	They propose a conformal uncertainty measure and a method to transform heuristic uncertainty notions into rigorous prediction sets.
Outcome:	Empirical results show that the proposed method outperforms state-of-the-art methods and can provide reliable guarantees for open-ended NLG tasks.

Copied to clipboard

Chenhui Shen, Liying Cheng, Xuan-Phi Nguyen, Yang You, Lidong Bing

Plot Retrieval as an Assessment of Abstract Semantic Association (2024.acl-srw)

Challenge:	Pre-trained language models have been used for abstractive single-document summarization (SDS) but they may not be suitable for multi-document summary (MDS)
Approach:	They propose to enforce hierarchy on both encoder and decoder to facilitate multi-document interactions for MDS.
Outcome:	Xiao et al. (2019) outperforms or is competitive with the previous best models.

Copied to clipboard

Shicheng Xu, Liang Pang, Jiangnan Li, Mo Yu, Fandong Meng, Huawei Shen, Xueqi Cheng, Jie Zhou

Information Retrieval

Challenge:	Existing information retrieval datasets cannot capture abstract semantic associations well.
Approach:	They propose a task that retrieves relevant plots from the book for a query using a labeled dataset.
Outcome:	The proposed task can be used to evaluate the performance of IR models on the novel task Plot Retrieval.

Learning Compressed Sentence Representations for On-Device Text Processing (P19-1)

Copied to clipboard

Dinghan Shen, Pengyu Cheng, Dhanasekar Sundararaman, Xinyuan Zhang, Qian Yang, Meng Tang, Asli Celikyilmaz, Lawrence Carin

Representation Learning

Challenge:	Existing methods for learning sentence embeddings assume they are continuous and real-valued.
Approach:	They propose four different strategies to transform continuous and generic sentence embeddings into a binarized form while preserving their rich semantic information.
Outcome:	The proposed methods reduce storage requirements by over 98% and improve performance on downstream tasks.

Shallow Focus, Deep Fixes: Enhancing Shallow Layers Vision Attention Sinks to Alleviate Hallucination in LVLMs (2025.emnlp-main)

Copied to clipboard

Xiaofeng Zhang, Yihao Quan, Chen Shen, Chaochen Gu, Xiaosong Yuan, Shaotian Yan, Jiawei Cao, Hao Cheng, Kaijie Wu, Jieping Ye

Improving Textual Network Embedding with Global Attention via Optimal Transport (P19-1)

Challenge:	Multimodal large language models (MLLMs) demonstrate excellent abilities for understanding visual information, but the hallucination remains a challenging problem.
Approach:	They propose a training-free approach to enhance vision attention sinks to facilitate convergence of the image token attention sink within shallow layers.
Outcome:	The proposed approach improves the convergence of the image token attention sink within shallow layers and strengthens the layer’s focus on the image itself.

Copied to clipboard

Liqun Chen, Guoyin Wang, Chenyang Tao, Dinghan Shen, Pengyu Cheng, Xinyuan Zhang, Wenlin Wang, Yizhe Zhang, Lawrence Carin

Representation Learning

Challenge:	Existing methods for learning textual network embeddings are noisy and sparse.
Approach:	They propose to use text-based attention parsing to learn context-aware network embeddings.
Outcome:	The proposed model outperforms state-of-the-art methods in a number of domains.

Rescorla-Wagner Steering of LLMs for Undesired Behaviors over Disproportionate Inappropriate Context (2025.emnlp-main)

Copied to clipboard

Rushi Wang, Jiateng Liu, Cheng Qian, Yifan Shen, Yanzhou Pan, Zhaozhuo Xu, Ahmed Abbasi, Heng Ji, Denghui Zhang

Robustness in NLP

Challenge:	Incorporating external context can enhance the response quality of Large Language Models (LLMs). however, real-world contexts often mix relevant information with disproportionate inappropriate content.
Approach:	They propose a Poisoned Context Testbed to pair queries with real-world contexts . they propose 'rw-Steering' to internalize inappropriate signals .
Outcome:	The proposed model improves response quality by 39.8% and reverses undesirable behavior curve.

Logical Structure as Knowledge: Enhancing LLM Reasoning via Structured Logical Knowledge Density Estimation (2026.findings-acl)

Copied to clipboard

Zhen Bi, Zhenlin Hu, Xueshu Chen, Mingyang Chen, Cheng Deng, Yida Xue, Zhen Wang, Qing Shen, Ningyu Zhang, Jungang Lou

Profiler: Black-box AI-generated Text Origin Detection via Context-aware Inference Pattern Analysis (2025.emnlp-main)

Challenge:	Existing data-centric paradigms equate quality with factuality or diversity and ignore the internal logical complexity of training samples.
Approach:	They propose a density-aware re-cognizing optimization strategy that prioritizes high-density logical samples to align training with the model's reasoning boundary.
Outcome:	The proposed metric outperforms existing methods and improves reasoning performance without increasing total data volume.

Copied to clipboard

Hanxi Guo, Siyuan Cheng, Xiaolong Jin, Zhuo Zhang, Guangyu Shen, Kaiyuan Zhang, Shengwei An, Guanhong Tao, Xiangyu Zhang

Challenge:	Existing methods to identify the origin of AI-generated texts fail to identify origin due to the high similarity of different LLMs.
Approach:	They propose a black-box AI-generated text origin detection method which accurately predicts the origin of an input text by extracting distinct context inference patterns.
Outcome:	The proposed method outperforms 10 state-of-the-art baselines and achieves a 25% increase in AUC score on average across natural language and code datasets.

RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling (2023.findings-emnlp)

Copied to clipboard

Jingcheng Deng, Liang Pang, Huawei Shen, Xueqi Cheng

CondenseLM: LLMs-driven Text Dataset Condensation via Reward Matching (2025.emnlp-main)

Challenge:	Existing research on retrieval-augmented language models has two main problems: determining what information to retrieve and effectively combining retrieved information during generation.
Approach:	They propose a retrieval-augmented language model that captures current and future information from source and target text into a latent space.
Outcome:	The proposed model is more efficient than explicit raw text, but limited by context length and noise.

Copied to clipboard

Cheng Shen, Yew-Soon Ong, Joey Tianyi Zhou

Green & Sustainable NLP

Challenge:	Existing methods for dataset condensation struggle to compress more information into samples . current methods struggle to extract enough training data for effective learning .
Approach:	They propose a paradigm for dataset condensation that uses an LLMs-driven approach to generate more informative and less biased samples.
Outcome:	The proposed method outperforms coreset selection and existing condensation methods by large margins while significantly reducing the computational cost.

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs (2024.findings-emnlp)

Copied to clipboard

Wenhua Cheng, Weiwei Zhang, Haihao Shen, Yiyang Cai, Xin He, Lv Kaokao, Yi Liu

Wav2SQL: Direct Generalizable Speech-To-SQL Parsing (2024.findings-acl)

Challenge:	Large Language Models (LLMs) have demonstrated exceptional proficiency in language-related tasks, but their deployment poses significant memory and storage requirements.
Approach:	They propose a method that optimizes rounding values and weight clipping within 200 steps.
Outcome:	The proposed method achieves exceptional results across 2 to 4 bits while maintaining low tuning costs and avoiding additional inference overhead.

Copied to clipboard

Huadai Liu, Rongjie Huang, Jinzheng He, Gang Sun, Ran Shen, Xize Cheng, Zhou Zhao

An End-to-End Generative Architecture for Paraphrase Generation (D19-1)

Challenge:	Existing models for speech-driven SQL parsing are based on a cascaded approach, resulting in data scarcity and inconsistent performance.
Approach:	They propose a direct generalizable speech-to-SQL parsing model which avoids error compounding across cascaded systems.
Outcome:	The proposed model avoids error compounding and achieves state-of-the-art results by 4.7% improvement over baseline.

Copied to clipboard

Qian Yang, Zhouyuan Huo, Dinghan Shen, Yong Cheng, Wenlin Wang, Guoyin Wang, Lawrence Carin

Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model (2024.emnlp-main)

Challenge:	Existing methods for generating paraphrases with linguistic knowledge are often domain specific and hard to scale, or yield inferior results.
Approach:	They propose an end-to-end conditional generative architecture for generating paraphrases via adversarial training which does not depend on extra linguistic information.
Outcome:	The proposed method outperforms existing models on automatic metrics and human evaluations on four public datasets.

Copied to clipboard

Wenqi Zhang, Zhenglin Cheng, Yuanyu He, Mengna Wang, Yongliang Shen, Zeqi Tan, Guiyang Hou, Mingqian He, Yanna Ma, Weiming Lu, Yueting Zhuang