Papers by Shang Liu

Safe: Enhancing Mathematical Reasoning in Large Language Models via Retrospective Step-aware Formal Verification (2025.acl-long)

Challenge:	Experimental results show that pretrained language models generate inconsistent factual knowledge in many conversational tasks.
Approach:	They propose a method which explicitly introduces extended feedforward networks (FFNs) in Transformers to enhance factual knowledge expressions given the specific patterns of knowledge-grounded dialogue inputs.
Outcome:	The proposed methods improve the factual expression capability of feedforward networks (FFNs) in knowledge-grounded dialogue systems by knowledge enhancement and alignment respectively.

Copied to clipboard

Chengwu Liu, Ye Yuan, Yichun Yin, Yan Xu, Xin Xu, Zaoyu Chen, Yasheng Wang, Lifeng Shang, Qun Liu, Ming Zhang

Topology Imbalance and Relation Inauthenticity Aware Hierarchical Graph Attention Networks for Fake News Detection (2022.coling-1)

Challenge:	Chain-of-Thought prompting is a de facto method to elicit reasoning capabilities from large language models (LLMs).
Approach:	They propose a step-aware formal verification framework Safe to address hallucinations in CoT prompting . they propose 'formal step' as a benchmark for step correctness theorem proving with 30,809 formal statements.
Outcome:	The proposed framework shows significant performance improvement while offering interpretable and verifiable evidence.

Copied to clipboard

Li Gao, Lingyun Song, Jie Liu, Bolin Chen, Xuequn Shang

Decomposable Neural Paraphrase Generation (P19-1)

Challenge:	Existing methods to detect fake news focus on mining lexical and syntactic features.
Approach:	They propose a topology imbalance and Relation inauthenticity aware Hierarchical Graph Attention Networks to identify fake news on social media.
Outcome:	The proposed method outperforms state-of-the-art methods on real-world datasets.

Copied to clipboard

Zichao Li, Xin Jiang, Lifeng Shang, Qun Liu

Text Generation

Challenge:	Existing models learn to generate paraphrases by mapping a sequence to another, with each word processed and generated in a uniform way.
Approach:	They propose a Transformer-based model that can learn and generate paraphrases at different levels of granularity in a disentangled way.
Outcome:	The proposed model achieves competitive in-domain performance compared to state-of-the-art models and significantly better performance when adapting to a new domain.

Self-Taught Agentic Long Context Understanding (2025.acl-long)

Copied to clipboard

Yufan Zhuang, Xiaodong Yu, Jialian Wu, Ximeng Sun, Ze Wang, Jiang Liu, Yusheng Su, Jingbo Shang, Zicheng Liu, Emad Barsoum

Information Retrieval

Challenge:	Extensive experiments across seven long-context tasks demonstrate that AgenticLU significantly outperforms state-of-the-art prompting methods and specialized long-consumer LLMs.
Approach:	They propose a framework to enhance an LLM's understanding of long-context questions by integrating targeted self-clarification with contextual grounding within an agentic workflow.
Outcome:	The proposed framework outperforms state-of-the-art prompting methods and specialized long-context LLMs in seven long-constitut tasks.

From Outcome to Process: Optimizing MoE Load Balancing with MCTS (2026.findings-acl)

Copied to clipboard

Wenjun Ke, Hengyuan Xu, Ziyu Shang, Yao He, Jiahao Wang, Zijie Xu, Peng Wang, Yuhang Lou, Jiajun Liu

Green & Sustainable NLP

Challenge:	Existing balancing strategies focus on constraining the final distribution of expert usage, but overlook the routing decisions made at each layer.
Approach:	They propose a three-stage framework that leverages process-level rewards to guide balanced expert routing.
Outcome:	Extensive experiments show that LayerMoE improves the performance of state-of-the-art LoRA-MoA baselines, yielding an average accuracy gain of 1.39%.

Uncertainty-Aware Routing for Principled Alignment with MoE Dynamics (2026.acl-long)

Copied to clipboard

Yilong Chen, Junyuan Shang, Yuchen Feng, Zhenyu Zhang, Naibin Gu, Ziqi Wang, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang

Challenge:	Mixture-of-Experts (MoE) is a cornerstone for scaling LLMs, yet its training dynamics remain poorly understood, often leading to sub-optimal specialization.
Approach:	They propose to use Helmholtz Free Energy and Router Entropy to study the MoE lifecycle and identify a universal Three-Stage Phase Transition .
Outcome:	The proposed model reduces perplexity and improves expert distinctiveness, offering a principled path toward thermodynamically aligned computation.

How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis (2022.findings-acl)

Copied to clipboard

Shaobo Li, Xiaoguang Li, Lifeng Shang, Zhenhua Dong, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu

G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks (2022.emnlp-main)

Challenge:	Recent studies show that pre-trained language models can fill in the missing factual words in cloze-style prompts such as ”Dante was born in [MASK]” .
Approach:	They propose to quantitatively measure and evaluate the word-level patterns that PLMs depend on to generate the missing factual words.
Outcome:	The proposed model fills in the missing factual words in cloze-style prompts by relying on effective clues or shortcut patterns.

Copied to clipboard

Zhongwei Wan, Yichun Yin, Wei Zhang, Jiaxin Shi, Lifeng Shang, Guangyong Chen, Xin Jiang, Qun Liu

mCLIP: Multilingual CLIP via Cross-lingual Transfer (2023.acl-long)

Challenge:	Existing domain-adaptive pre-training (DAPT) models tend to forget the general knowledge acquired by general PLMs, leading to catastrophic forgetting and sub-optimal performance.
Approach:	They propose a framework which augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge.
Outcome:	The proposed framework augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge.

Copied to clipboard

Guanhua Chen, Lu Hou, Yun Chen, Wenliang Dai, Lifeng Shang, Xin Jiang, Qun Liu, Jia Pan, Wenping Wang

Multilinguality

Challenge:	Existing multilingual vision-language pretrained models are biased towards English due to the lack of sufficient non-English image-text pairs.
Approach:	They propose to train a retrieval-efficient dual-stream multilingual VLP model by aligning CLIP model and a multilingual text encoder through a novel Triangle Cross-modal Knowledge Distillation method.
Outcome:	Empirical results show that mCLIP achieves new state-of-the-art performance for both zero-shot and finetuned multilingual image-text retrieval tasks.

ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch (2026.acl-long)

Copied to clipboard

Zheng Liu, Honglin Lin, Xiaoyang Wang, Xin Gao, Yu Li, Mengzhang Cai, Yun Zhu, Zhanping Zhong, Qizhi Pei, Zhuoshi Pan, Xiaoran Shang, Conghui He, Bin Cui, Wentao Zhang, Lijun Wu

Can Pretrained Language Models Derive Correct Semantics from Corrupt Subwords under Noise? (2023.starsem-1)

Challenge:	Existing open-source vision language models lack high-quality training data for chart reasoning . current models are simplistic and repetitive, while associated QA pairs are prone to hallucinations .
Approach:	They propose a framework to synthesize complex charts and reliable reasoning data from scratch.
Outcome:	Experimental results show that ChartVerse-8B surpasses existing models in QA and difficulty . lack of high-quality training data hampers development of open-source models .

Copied to clipboard

Xinzhe Li, Ming Liu, Shang Gao

Subtle Errors in Reasoning: Preference Learning via Error-injected Self-editing (2025.acl-long)

Challenge:	Existing studies have shown that Pretrained Language Models (PLMs) perform poorly under noise due to subword segmentation.
Approach:	They propose a framework for subword segmentation that provides a systematic categorization of segmentation corruption under noise and evaluation protocols by generating contrastive datasets with canonical-noisy word pairs.
Outcome:	The proposed framework provides a systematic categorization of segmentation corruption under noise and evaluation protocols by generating contrastive datasets with canonical-noisy word pairs.

Copied to clipboard

Kaishuai Xu, Tiezheng Yu, Wenjun Hou, Yi Cheng, Chak Tou Leong, Liangyou Li, Xin Jiang, Lifeng Shang, Qun Liu, Wenjie Li

MINER: Multi-Interest Matching Network for News Recommendation (2022.findings-acl)

Challenge:	Existing studies to improve mathematical ability typically involve applying preference learning to step-wise solution pairs, but they overlook critical subtle errors.
Approach:	They propose a preference learning framework that injects predefined subtle errors into pivotal tokens to construct hard pairs for error mitigation.
Outcome:	Extensive experiments show that the proposed framework improves on Qwen2-7B-Instruct and MATH with 4.5K training samples.

Copied to clipboard

Jian Li, Jieming Zhu, Qiwei Bi, Guohao Cai, Lifeng Shang, Zhenhua Dong, Xin Jiang, Qun Liu

Representation Learning

Challenge:	Existing methods learn a single user embedding from user’s historical behaviors to represent the reading interest.
Approach:	They propose a poly attention scheme to learn multiple interest vectors for each user, which encodes the different aspects of user interest.
Outcome:	The proposed approach significantly outperforms existing state-of-the-art methods on the MIND news recommendation benchmark.

LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling (2022.emnlp-main)

Copied to clipboard

Dongsheng Chen, Chaofan Tao, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu

On the Consistency of Commonsense in Large Language Models (2025.findings-acl)

Challenge:	Recent large-scale video-language pre-trained models have shown appealing performance on downstream tasks.
Approach:	They propose a video-text model that adapts a pre-trained image-language model into a text-based model without heavy pre-training.
Outcome:	The proposed model outperforms existing models on video-text retrieval and video question answering tasks without heavy pre-training.

Copied to clipboard

Guozheng Li, Peng Wang, Wenjun Ke, Zijie Xu, Jiajun Liu, Ziyu Shang

Acquisition and Application of Novel Knowledge in Large Language Models (2025.acl-long)

Challenge:	Existing evaluations of commonsense for large language models focus on downstream knowledge tasks, failing to probe whether LLMs truly understand and utilize knowledge or merely memorize it.
Approach:	They propose to automatically construct a large benchmark named CoCo which measures LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks.
Outcome:	The proposed benchmark systematically assesses LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks.

Copied to clipboard

Ziyu Shang, Jianghan Liu, Zhizhao Luo, Peng Wang, Wenjun Ke, Jiajun Liu, Zijie Xu, Guozheng Li

A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering (2021.acl-long)

Challenge:	Existing methods for constructing new datasets rely on timestamps or simple templates that do not accurately reflect the real world.
Approach:	They propose a knowledge dataset construction approach that simulates biological evolution using knowledge graphs to generate synthetic entities with diverse attributes.
Outcome:	The proposed framework outperforms knowledge augmentation methods by 3.3%-38%.

Copied to clipboard

Zhihong Shao, Lifeng Shang, Qun Liu, Minlie Huang

Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)

Challenge:	Weakly supervised question answering usually has only final answers as supervision signals while correct solutions are not provided.
Approach:	They propose to explicitly exploit the semantic correlations between question-answer pairs and predicted answers by maximizing mutual information between question and answer pairs.
Outcome:	The proposed method significantly outperforms previous learning methods in terms of task performance and is more effective in training models to produce correct solutions.

Copied to clipboard

Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong

Explainability & Interpretability in NLP

Challenge:	Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored.
Approach:	They propose a survey structured around the pipeline to identify and improve MI models.
Outcome:	The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency.

Neural-Symbolic Inference for Robust Autoregressive Graph Parsing via Compositional Uncertainty Quantification (2022.emnlp-main)

Copied to clipboard

Zi Lin, Jeremiah Liu, Jingbo Shang

ALERT: An LLM-powered Benchmark for Automatic Evaluation of Recommendation Explanations (2025.naacl-long)

Challenge:	Pre-trained models excel at graph semantic parsing with rich annotated data, but generalize poorly to out-of-distribution and long-tail examples.
Approach:	They propose a compositionality-aware approach to neural-symbolic inference informed by model confidence to capture different aspects of the graph prediction.
Outcome:	The proposed method outperforms state-of-the-art models on an English resource grammar parsing problem on standard in-domain and seven OOD corpora.

Copied to clipboard

Yichuan Li, Xinyang Zhang, Chenwei Zhang, Mao Li, Tianyi Liu, Pei Chen, Yifan Gao, Kyumin Lee, Kaize Ding, Zhengyang Wang, Zhihan Zhang, Jingbo Shang, Xian Li, Trishul Chilimbi

If an LLM Were a Character, Would It Know Its Own Story? Evaluating Lifelong Learning in LLMs (2026.acl-long)

Challenge:	Existing benchmarks for recommendation explanation evaluation lack item diversity and user preferences data.
Approach:	They propose a model-agnostic recommendation explanation evaluation benchmark based on Amazon e-commerce categories with implicit preferences . they propose two novel automatic evaluators that enable scalable and human-preference aligned evaluation of explanations .
Outcome:	The proposed model-agnostic evaluation benchmark outperforms existing methods in a variety of domains.

Copied to clipboard

Siqi Fan, Xiusheng Huang, Yiqun Yao, Xuezhi Fang, Kang Liu, Peng Han, Shuo Shang, Aixin Sun, Yequan Wang

Challenge:	Existing benchmarks for large language models (LLMs) fail to capture these dynamics, focusing on static, open-ended evaluations.
Approach:	They propose a benchmark to assess lifelong learning in large language models . they use two episodic datasets rich in narrative structure and character interactions .
Outcome:	Experiments on LLMs show that non-parametric methods outperform parametric ones in managing stateful learning.

Retrieval Augmented Fact Verification by Synthesizing Contrastive Arguments (2024.acl-long)

Copied to clipboard

Zhenrui Yue, Huimin Zeng, Lanyu Shang, Yifan Liu, Yang Zhang, Dong Wang

CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Models (2024.findings-acl)

Challenge:	Existing methods to verify claim credibility rely on embedded knowledge or unreliable context.
Approach:	They propose retrieval augmented fact verification through the synthesis of contrasting arguments (RAFTS) they use an embedding model to identify informative demonstrations and in-context prompts to generate the prediction and explanation.
Outcome:	The proposed method outperforms existing methods with smaller LLMs or unreliable contexts.

Copied to clipboard

Linhao Yu, Yongqi Leng, Yufei Huang, Shang Wu, Haixin Liu, Xinmeng Ji, Jiahui Zhao, Jinwang Song, Tingting Cui, Xiaoqing Cheng, Liutao Liutao, Deyi Xiong

MTRec: Multi-Task Learning over BERT for News Recommendation (2022.findings-acl)

Challenge:	Recent years have witnessed remarkable progress achieved by large language models in both natural language understanding and generation.
Approach:	They propose a large benchmark CMoralEval for moral evaluation of Chinese LLMs . they use a Chinese TV program discussing Chinese moral norms and Chinese moral anomies based on various sources .
Outcome:	The proposed dataset is characterized by diversity and authenticity.

Copied to clipboard

Qiwei Bi, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, Hanfang Yang

PsyScam: A Benchmark for Psychological Techniques in Real-World Scams (2025.findings-emnlp)

Challenge:	Existing news recommendation methods learn news representations solely based on news titles. Existing methods only utilize title information and neglect other valuable news information such as categories and entities.
Approach:	They propose a multi-task method to incorporate multi-field information into BERT, which improves its news encoding capability.
Outcome:	Extensive experiments on the MIND news recommendation benchmark show the proposed method is effective.

Copied to clipboard

Shang Ma, Tianyi Ma, Jiahao Liu, Wei Song, Zhenkai Liang, Xusheng Xiao, Yanfang Ye

Challenge:	PTs are employed by scammers to manipulate victims and cause lasting psychological trauma.
Approach:	They propose a benchmark to capture the PTs employed in real-worldscam reports and investigate how LLMs can be utilized to generate variants of scams based on the pts and the contexts provided by thesescams.
Outcome:	The proposed model can generate variants of scams based on the PTs employed in real-world scam reports and the contexts provided by these scams.

PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinations (2026.acl-long)

Copied to clipboard

Yuhe Wu, Guangyu Wang, Yuran Chen, Jiatong Zhang, Yutong Zhang, Yujie Chen, Jiaming Shang, Guang Zhang, Zhuang Liu

Compression of Generative Pre-trained Language Models via Quantization (2022.acl-long)

Challenge:	Existing benchmarks for hallucination evaluation rely on mixed queries and posterior evaluation, which quantifies hallucinosity severity but offers limited insight into where and why they occur.
Approach:	They propose a controlled benchmark that disentangles hallucinations into four dimensions: knowledge missing, knowledge errors, reasoning errors, and instruction-following errors.
Outcome:	The proposed model disentangles hallucinations into four dimensions: knowledge missing, knowledge errors, reasoning errors, and instruction-following errors.

Copied to clipboard

Chaofan Tao, Lu Hou, Wei Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Ping Luo, Ngai Wong

Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling (D18-1)

Challenge:	Existing methods to compress generative pre-trained language models fail on generative tasks due to homogeneous word embeddings and limited memory.
Approach:	They propose a token-level contrastive distillation method to learn distinguishable word embeddings and a module-wise dynamic scaling method to make quantizers adaptive to different modules.
Outcome:	The proposed method outperforms the state-of-the-art compression methods on generative PLMs by a clear margin.

Copied to clipboard

Liyuan Liu, Xiang Ren, Jingbo Shang, Xiaotao Gu, Jian Peng, Jiawei Han

GhostBERT: Generate More Features with Cheap Operations for BERT (2021.acl-long)

Challenge:	Existing efforts to train pre-trained language models have brought significant improvements to various NLP applications.
Approach:	They propose to compress bulky LMs while preserving useful information for a specific task.
Outcome:	The proposed method can detach any layer without affecting others, and stretch shallow and wide LMs to be deep and narrow.

Copied to clipboard

Zhiqi Huang, Lu Hou, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu

Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents (2024.acl-long)

Challenge:	Existing studies show that some parameters in pre-trained language models can be pruned away without severe accuracy degradation.
Approach:	They propose a method which generates more features with very cheap operations from the remaining features and can be applied to unpruned BERT models to enhance their performance.
Outcome:	Empirical results on the GLUE benchmark on three backbone models (i.e., BERT, RoBERTa and ELECTRA) verify the efficacy of the proposed method.

Copied to clipboard

Shihan Deng, Weikai Xu, Hongda Sun, Wei Liu, Tao Tan, Liujianfeng Liujianfeng, Ang Li, Jian Luan, Bin Wang, Rui Yan, Shuo Shang

FLASH: Focused Layer Attention Sink Hijacking (2026.findings-acl)

Challenge:	Existing benchmarks for LLM-based mobile agents are insufficient to evaluate their capabilities.
Approach:	They propose a benchmark to evaluate LLM-based mobile agents' planning capabilities . they expand UI operations by incorporating 103 APIs to accelerate task completion .
Outcome:	The proposed benchmarks are based on 103 collected APIs and real user queries . the data is categorized into three distinct groups: SAST, SAMT, and MAMT .

Copied to clipboard

Siyuan Deng, Yerong Li, Fanhua Shang, Hongying Liu

Robustness in NLP

Challenge:	Large Language Models (LLMs) remain vulnerable to jailbreaking attacks despite advances in safety alignment .
Approach:	They propose a new diagnostic auditing framework that dismantles the model's internal safety anchor by precisely scaling attention scores in these vulnerable layers.
Outcome:	The proposed framework achieves a state-of-the-art Attack Success Rate of over 77% with an unprecedented efficiency of 1.53 queries on average.

AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models (2023.acl-short)

Copied to clipboard

Siheng Li, Cheng Yang, Yichun Yin, Xinyu Zhu, Zesen Cheng, Lifeng Shang, Xin Jiang, Qun Liu, Yujiu Yang

Dialogue Systems & Conversational Agents

Challenge:	Existing research on information-seeking conversations is stymied by the lack of training data.
Approach:	They propose to use autoconv for synthetic conversation generation to capture the characteristics of the information-seeking process and fine tune an LLM with a few human conversations to generate synthetic conversations with high quality.
Outcome:	The proposed model improves on two commonly-used datasets and alleviates the dependence on human annotation.

Can Language Models Follow Multiple Turns of Entangled Instructions? (2025.findings-emnlp)

Copied to clipboard

Chi Han, Xin Liu, Haodong Wang, Shiyang Li, Jingfeng Yang, Haoming Jiang, Zhengyang Wang, Qingyu Yin, Liang Qiu, Changlong Yu, Yifan Gao, Zheng Li, Bing Yin, Jingbo Shang, Heng Ji

Controlled Text Generation Using Dictionary Prior in Variational Autoencoders (2022.findings-acl)

Challenge:	Despite of significant achievements in improving instruction-following capabilities of large language models, the ability to process multiple potentially entangled or conflicting instructions remains a considerable challenge.
Approach:	They construct multi-turn instruction with 1.1K high-quality multi-turned conversations using the human-in-the-loop approach and examine their capabilities.
Outcome:	The proposed model shows that it is difficult to integrate multiple turns and balance competing objectives when instructions intersect or conflict.

Copied to clipboard

Xianghong Fang, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, Dit-Yan Yeung

TAG: Gradient Attack on Transformer-based Language Models (2021.findings-emnlp)

Challenge:	Variational autoencoders (VAEs) have been widely applied in text generation tasks, but they suffer from insufficient representation capacity and poor controllability.
Approach:	They propose a data-driven prior that has expressivity and controllability.
Outcome:	The proposed prior enjoys expressivity and controllability and can be used in language modeling and controlled text generation.

Copied to clipboard

Jieren Deng, Yijue Wang, Ji Li, Chenghong Wang, Chao Shang, Hang Liu, Sanguthevar Rajasekaran, Caiwen Ding

LEMON: Reviving Stronger and Smaller LMs from Larger LMs with Linear Parameter Fusion (2024.acl-long)

Challenge:	Recent studies show that publicly shared gradients in the training process can reveal the private training data to a third-party.
Approach:	They propose a gradient attack algorithm to reconstruct the local training data using GLUE benchmarks.
Outcome:	The proposed algorithm achieves 1.5x recover rate and 2.5x ROUGE-2 over previous methods without the need of ground truth label.

Copied to clipboard

Yilong Chen, Junyuan Shang, Zhenyu Zhang, Shiyao Cui, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu

More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression (2025.findings-emnlp)

Challenge:	Existing methods to train a stronger and smaller model with the help of large models are limited by the model size and performance.
Approach:	They propose to learn competent initial points for smaller models by fusing parameters from larger models and introduce controllable receptive fields to model prior parameter characteristics.
Outcome:	The proposed method outperforms baselines in terms of effectiveness and efficiency.

Copied to clipboard

Jiebin Zhang, Dawei Zhu, Yifan Song, Wenhao Wu, Chuqiao Kuang, Xiaoguang Li, Lifeng Shang, Qun Liu, Sujian Li

Challenge:	storing more tokens in the KV cache at lower precision can enhance the long-context performance of large language models.
Approach:	They propose a token-precision trade-off strategy to optimize KV cache compression . they also propose storing more tokens in the KV at lower precision .
Outcome:	The proposed method achieves an optimal point within the Information Bottleneck compared to standalone KV pruning or KV quantization.

SynC-LLM: Generation of Large-Scale Synthetic Circuit Code with Hierarchical Language Models (2025.emnlp-main)

Copied to clipboard

Shang Liu, Yao Lu, Wenji Fang, Jing Wang, Zhiyao Xie

DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labeling (2021.emnlp-main)

Challenge:	Recent years, AI-assisted integrated circuit design methods have shown great potential in boosting IC design efficiency. however, this emerging technique is limited by the serious scarcity of publicly accessible large-scale circuit design data, which are mostly private IPs owned by semiconductor companies.
Approach:	They propose a hierarchical framework that exploits LLM's ability to generate new large-scale synthetic digital circuits by learning sequential logic skeletons and annotating function descriptions.
Outcome:	The proposed framework generates large-scale synthetic circuits that are valid and fully functional, and can significantly improve AI models’ performance in multiple IC design tasks.

Copied to clipboard

Baojun Wang, Zhao Zhang, Kun Xu, Guang-Yuan Hao, Yuyang Zhang, Lifeng Shang, Linlin Li, Xiao Chen, Xin Jiang, Qun Liu

Read before Generate! Faithful Long Form Question Answering with Machine Reading (2022.findings-acl)

Challenge:	Existing approaches to integrate lexical knowledge into deep learning models are limited by large-scale dynamic lexicons.
Approach:	They propose a plug-in lexicon incorporation approach for BERT based sequence labeling tasks . they adopt word-agnostic tag embeddings to avoid re-training the representation .
Outcome:	The proposed framework achieves new SOTA even with large scale lexicons, the authors show . they adopt word-agnostic tag embeddings to avoid re-training the representation .

Copied to clipboard

Dan Su, Xiaoguang Li, Jindi Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Pascale Fung

Retrieval-Augmented Parsing for Complex Graphs by Exploiting Structure and Uncertainty (2023.findings-emnlp)

Challenge:	Long-form question answering (LFQA) generates a paragraph-length answer for a given question.
Approach:	They propose a framework that jointly models answer generation and machine reading.
Outcome:	The proposed model generates a more factually accurate answer from millions of documents retrieved from a large dataset.

Copied to clipboard

Zi Lin, Quan Yuan, Panupong Pasupat, Jeremiah Liu, Jingbo Shang

Structured Data in NLP

Challenge:	Retrieval augmentation is effective for large graph parsing tasks, but can fail to identify the most informative exemplars . structure-aware and uncertainty-guided adaptive retrieval (SUGAR) exploits two unique sources of information: structural similarity and model uncertainty.
Approach:	They propose a structure-aware and uncertainty-guided adaptive retrieval approach that exploits structural similarity and model uncertainty to improve retrieval-augmented parsing for complex graph problems.
Outcome:	The proposed method improves retrieval-augmented parsing for graph parsers with large output graphs and non-trivial structure.

LLMSurgeon: Diagnosing Data Mixture of Large Language Models (2026.acl-long)

Copied to clipboard

Yaxin Luo, Jiacheng Cui, Xiaohan Zhao, Xinyi Shang, Jiacheng Liu, Xinyue Bi, Zhaoyi Li, Zhiqiang Shen

Pair-Based Joint Encoding with Relational Graph Convolutional Networks for Emotion-Cause Pair Extraction (2022.emnlp-main)

Challenge:	a lack of transparency in large language models makes auditing their "digital DNA" difficult.
Approach:	They propose a framework that casts DMS as an inverse problem under label-shift assumption . they propose LLMScan, a recipe-verifiable evaluation suite built from open-source LLMs .
Outcome:	The proposed framework casts DMS as an inverse problem under label-shift assumption . compared with existing frameworks, it recovers domain mixtures with high fidelity .

Copied to clipboard

Junlong Liu, Xichen Shang, Qianli Ma

BinaryBERT: Pushing the Limit of BERT Quantization (2021.acl-long)

Challenge:	Emotion cause pair extraction (ECPE) aims to extract emotion clauses and corresponding cause clauses.
Approach:	They propose a novel task called emotion-cause pair extraction to extract emotion clauses and corresponding cause clauses.
Outcome:	The proposed task can extract emotion clauses and cause clauses, and achieve state-of-the-art performance on the Chinese benchmark corpus.

Copied to clipboard

Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jin Jin, Xin Jiang, Qun Liu, Michael Lyu, Irwin King

Stepwise Reasoning Checkpoint Analysis: A Test Time Scaling Method to Enhance LLMs’ Reasoning (2025.emnlp-main)

Challenge:	Recent pre-trained language models have achieved remarkable performance improvement in various tasks, but the improvement generally comes at the cost of increasing model size and computation.
Approach:	They propose a binary quantization technique which initializes binaryBERT by splitting from a ternary network.
Outcome:	The proposed model achieves state-of-the-art performance on the GLUE and SQUAD benchmarks while being 24x smaller.

Copied to clipboard

Zezhong Wang, Xingshan Zeng, Weiwen Liu, Yufei Wang, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking (2025.acl-long)

Challenge:	Existing methods that use Chain-of-Thought suffer from path homogenization and inefficient use of intermediate results.
Approach:	They propose a framework that introduces checkpoints between reasoning steps to reduce path homogenization and create fault-tolerant mechanisms.
Outcome:	The proposed framework reduces path homogenization and creates fault-tolerant mechanism by utilizing high-quality intermediate results.

Copied to clipboard

Yilong Chen, Junyuan Shang, Zhenyu Zhang, Yanxi Xie, Jiawei Sheng, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang

DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy (2024.acl-long)

Challenge:	Large language models face inherent performance bottlenecks under parameter constraints . challenging tokens induce abrupt gradient spikes across layers, exposing stress points .
Approach:	They propose an inner thinking transformer that reimagines layer computations as implicit thinking steps.
Outcome:	Empirical results show that ITT outperforms Transformer/Loop variants in 11 benchmarks.

Copied to clipboard

Hongda Sun, Weikai Xu, Wei Liu, Jian Luan, Bin Wang, Shuo Shang, Ji-Rong Wen, Rui Yan

NACL: A General and Effective KV Cache Eviction Framework for LLM at Inference Time (2024.acl-long)

Challenge:	Recent advances in large language models (LLMs) have revolutionized the landscape of reasoning tasks.
Approach:	They propose a new approach that rethinks the reasoning process as an evolution from indeterminacy to determinacy.
Outcome:	The proposed model surpasses all baselines on various logical reasoning benchmarks.

Copied to clipboard

Yilong Chen, Guoxia Wang, Junyuan Shang, Shiyao Cui, Zhenyu Zhang, Tingwen Liu, Shuohuan Wang, Yu Sun, Dianhai Yu, Hua Wu

Joint Constrained Learning with Boundary-adjusting for Emotion-Cause Pair Extraction (2023.acl-long)

Challenge:	Large Language Models (LLMs) with extended context windows are expensive and infeasible on fixed memory hardware due to the surprisingly large memory consumption of KV Cache.
Approach:	They propose a general framework for long-context KV cache eviction that achieves more optimal and efficient evict in a single operation during the encoding phase.
Outcome:	The proposed framework improves performance on short- and long-text tasks by 80% and 76% respectively, reducing KV Cache by up to 5 with over 95% performance maintenance.

Copied to clipboard

Huawen Feng, Junlong Liu, Junhao Zheng, Haibin Chen, Xichen Shang, Qianli Ma

Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models (2025.findings-acl)

Challenge:	Emotion-Cause Pair Extraction (ECPE) aims to identify the document’s emotion clauses and corresponding cause clauses.
Approach:	They propose a constrained learning framework with boundary-adjusting for Emotion-Cause Pair Extraction that summarizes prior rules and forces the model to take them into consideration in optimization.
Outcome:	The proposed framework achieves competitive results compared with state-of-the-art methods on unbalanced data and proves robustness on unbalancing data.

Copied to clipboard

Qin Liu, Chao Shang, Ling Liu, Nikolaos Pappas, Jie Ma, Neha Anna John, Srikanth Doss, Lluis Marquez, Miguel Ballesteros, Yassine Benajiba

MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding (2024.findings-emnlp)

Challenge:	LLaVA-7B demonstrated a decline in safety alignment ability on multi-modal inputs compared to its LLM backbone.
Approach:	They propose a method to recover alignment ability from LLM backbone while preserving functional capabilities of VLMs.
Outcome:	The proposed framework recovers alignment ability that is inherent in the LLM backbone with minimal impact on fluency and linguistic capabilities of pre-trained VLMs.

Copied to clipboard

Qinzhuo Wu, Weikai Xu, Wei Liu, Tao Tan, Liujian Liujianfeng, Ang Li, Jian Luan, Bin Wang, Shuo Shang

Learning to Edit: Aligning LLMs with Knowledge Editing (2024.acl-long)

Challenge:	Recent mobile AI agents based on VLMs lack basic mobile capabilities due to their pre-trained nature.
Approach:	They propose a mobile AI agent based on VLMs that includes additional pre-training stages to enhance both intra- and inter-UI understanding.
Outcome:	The proposed model outperforms existing VLMs on the Chinese mobile dataset Mobile3M .

Copied to clipboard

Yuxin Jiang, Yufei Wang, Chuhan Wu, Wanjun Zhong, Xingshan Zeng, Jiahui Gao, Liangyou Li, Xin Jiang, Lifeng Shang, Ruiming Tang, Qun Liu, Wei Wang

Challenge:	Existing knowledge editing techniques rely on memorizing updated knowledge, impeding LLMs from effectively combining the new knowledge with their inherent knowledge when answering questions.
Approach:	They propose a Learning to Edit framework that equips LLMs with the ability to apply updated knowledge to input questions through a two-phase process .
Outcome:	The proposed framework outperforms existing methods in knowledge editing tasks and compares it with four benchmarks and two LLM architectures.

Generate & Rank: A Multi-task Framework for Math Word Problems (2021.findings-emnlp)

Copied to clipboard

Jianhao Shen, Yichun Yin, Lin Li, Lifeng Shang, Xin Jiang, Ming Zhang, Qun Liu

Gradually Excavating External Knowledge for Implicit Complex Question Answering (2023.findings-emnlp)

Challenge:	Existing studies formalize MWP as a generation task but mathematical expressions are prone to minor mistakes.
Approach:	They propose a ranking task for math word problem (MWP) that learns from its own mistakes and distinguishes between correct and incorrect expressions.
Outcome:	The proposed model outperforms baselines on the classical Math23k dataset and is 7% higher than the state-of-the-art.

Copied to clipboard

Chang Liu, Xiaoguang Li, Lifeng Shang, Xin Jiang, Qun Liu, Edmund Lam, Ngai Wong

Prompt-Based Length Controlled Generation with Multiple Control Types (2024.findings-acl)

Challenge:	Large language models (LLMs) have gained attention for their human-comparable capabilities but they may not solve open-domain implicit questions due to out-of-date domain knowledge, one-shot generation and restricted comprehensiveness.
Approach:	They propose a gradual knowledge excavation framework for open-domain complex question answering using extrinsic knowledge and historical knowledge.
Outcome:	The proposed framework achieves 78.17% accuracy with less than 6% parameters of its competitors, setting new SOTA in the 10B LLM class.

Copied to clipboard

Renlong Jie, Xiaojun Meng, Lifeng Shang, Xin Jiang, Qun Liu

Low-Resource NLP

Challenge:	Existing length control methods focus on a simple control type of “equal to” a target length.
Approach:	They propose a prompt-based method to achieve length controlled generation under different control types with high accuracy by using reinforcement learning and sample filtering with the reward signal given by rule-based reward models.
Outcome:	The proposed method significantly improves the accuracy of prompt-based length control on popular summarization datasets like CNNDM and NYT under multiple control types.

Multi-Docker-Eval: A ‘Shovel of the Gold Rush’ Benchmark on Automatic Environment Building for Software Engineering (2026.findings-acl)

Copied to clipboard

Kelin Fu, Tianyu Liu, Zeyu Shang, Yingwei MA, Jiaheng Liu, Jian Yang, Kaigui Bian

Green & Sustainable NLP

Challenge:	Automated environment configuration is a critical bottleneck in scaling software engineering (SWE) automation.
Approach:	They propose a reliable evaluation standard for automated environment configuration for 40 real-world repositories spanning 9 programming languages.
Outcome:	The proposed benchmark includes 40 real-world repositories spanning 9 programming languages and measures success in achieving executable states and efficiency under realistic constraints.

Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA (2026.acl-industry)

Copied to clipboard

Wenwei Li, Ming Xu, Tianle Xia, Lingxiang Hu, Yiding Sun, Linfang Shang, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang

Structured Data in NLP

Challenge:	Existing methods for QA in industrial environments are inherently relational and often updated.
Approach:	They propose a framework that optimizes retrieval and generation through two components: Graph-aware Retrieval and evidence-constrained reinforcement learning.
Outcome:	Experiments on an internal advertising QA dataset show consistent gains across expert-judged dimensions including accuracy, completeness, safety, and URL validity.

Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference (2023.acl-long)

Copied to clipboard

Junhao Zheng, Qianli Ma, Shengjie Qiu, Yue Wu, Peitian Ma, Junlong Liu, Huawen Feng, Xichen Shang, Haibin Chen

LLM-Guided Semantic-Aware Clustering for Topic Modeling (2025.acl-long)

Challenge:	Existing studies attribute catastrophic forgetting to fine-tuning, and they retain pre-trained knowledge indiscriminately without identifying what knowledge is transferable.
Approach:	They propose a unified objective for fine-tuning to retrieve the causality back from pre-trained data and use it to mitigate negative transfer while preserving knowledge.
Outcome:	The proposed method outperforms state-of-the-art fine-tuning methods on commonsense QA datasets and can be implemented as a plug-in module to inflate the performance of existing QA models.

Copied to clipboard

Jianghan Liu, Ziyu Shang, Wenjun Ke, Peng Wang, Zhizhao Luo, Jiajun Liu, Guozheng Li, Yining Li

CodeFort: Robust Training for Code Generation Models (2024.findings-emnlp)

Challenge:	Experimental results show that topic modeling is competitive compared to closed-source methods.
Approach:	They propose a semi-supervised topic modeling method that combines LLMs with clustering to improve topic generation and distribution.
Outcome:	The proposed method outperforms state-of-the-art methods that utilize GPT-4 on topic alignment and exhibits competitive performance compared to Neural Topic Models on topic quality.

Copied to clipboard

Yuhao Zhang, Shiqi Wang, Haifeng Qian, Zijian Wang, Mingyue Shang, Linbo Liu, Sanjay Krishna Gouda, Baishakhi Ray, Murali Krishna Ramanathan, Xiaofei Ma, Anoop Deoras

Programming Languages in NLP

Challenge:	Existing research efforts to improve code generation models are inadequate . code generation model performance is degraded under small perturbations .
Approach:	They propose a framework to improve the robustness of code generation models by generalizing code perturbations to enrich training data and enabling various robust training strategies.
Outcome:	The proposed framework increases pass rates and robustness drop rate against code-syntax perturbations.

DynamicNER: A Dynamic, Multilingual, and Fine-Grained Dataset for LLM-based Named Entity Recognition (2025.emnlp-main)

Copied to clipboard

Hanjun Luo, Yingbin Jin, Yiran Wang, Xinfeng Li, Tong Shang, Xuecheng Liu, Ruizhe Chen, Kun Wang, Hanan Salam, Qingsong Wen, Zuozhu Liu

Pre-training Language Models with Deterministic Factual Knowledge (2022.emnlp-main)

Challenge:	Existing datasets designed for Named Entity Recognition methods are inadequate for LLMs.
Approach:	They propose a dataset that is multilingual and multi-granular and enables LLMs to be applied to Named Entity Recognition methods.
Outcome:	The proposed dataset is multilingual and multi-granular, covering 8 languages and 155 entity types, with corpora spanning a diverse range of domains.

Copied to clipboard

Shaobo Li, Xiaoguang Li, Lifeng Shang, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu

Boosting Textural NER with Synthetic Image and Instructive Alignment (2024.findings-acl)

Challenge:	Existing studies show that Pre-trained Language Models fail to capture factual knowledge robustly.
Approach:	They propose to let PLMs learn the deterministic relationship between context and masked content to improve their ability to capture factual knowledge.
Outcome:	The proposed methods improve accuracy and consistency of factual knowledge capturing and boost performance of other knowledge-intensive tasks.

Copied to clipboard

Jiahao Wang, Wenjun Ke, Peng Wang, Hang Zhang, Dong Nie, Jiajun Liu, Guozheng Li, Ziyu Shang

TernaryBERT: Distillation-aware Ultra-low Bit BERT (2020.emnlp-main)

Challenge:	Named entity recognition (NER) is a key task reliant on textual data.
Approach:	They propose a method to transform NER into a multimodal task by using images from the internet as auxiliaries.
Outcome:	The proposed method surpasses all text-only baselines and improves F1 score by 1.4% to 2.3% on prominent MNER datasets.

Copied to clipboard

Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, Qun Liu

CrossWeigh: Training Named Entity Tagger from Imperfect Annotations (D19-1)

Challenge:	Transformer-based pre-training models like BERT are computationally expensive and limited to resource-constrained devices.
Approach:	They propose a method which ternarizes the weights in a fine-tuned BERT model.
Outcome:	The proposed method outperforms the other methods on the GLUE and SQUAD benchmarks while being 14.9x smaller.

Copied to clipboard

Zihan Wang, Jingbo Shang, Liyuan Liu, Lihao Lu, Jiacheng Liu, Jiawei Han

ProxyQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models (2024.acl-long)

Challenge:	Named entity recognition (NER) models can identify labels in 5.38% of test sentences . a framework to handle label mistakes during NER model training is proposed .
Approach:	They propose a framework to manually correct label mistakes in named entity recognition (NER) they aim to improve the accuracy of models by re-evaluating popular models on corrected test sets .
Outcome:	The proposed framework can detect label mistakes in 5.38% of test sentences . the proposed framework improves on three datasets with a high-performance model .

Copied to clipboard

Haochen Tan, Zhijiang Guo, Zhan Shi, Lu Xu, Zhili Liu, Yunlong Feng, Xiaoguang Li, Yasheng Wang, Lifeng Shang, Qun Liu, Linqi Song

ERCThinker: Fast-Slow Thinking for Emotion Recognition in Conversation (2026.acl-long)

Challenge:	Existing evaluation methods for large language models are labor-intensive and lack efficiency.
Approach:	They propose a framework dedicated to assessing long-text generation that includes in-depth human-curated meta-questions spanning various domains . they use a set of proxy-quests with pre-annotated answers to assess the content's quality by incorporating the generated texts as contextual background.
Outcome:	The proposed framework assesses the quality of long-text content by matching it with references through human evaluation or automated metrics.

Copied to clipboard

Yumeng Fu, Weitao Huang, Junjie Wu, Hao Teng, Shouduo Shang, Meishan Zhang, Bingquan Liu

Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement (2025.findings-acl)

Challenge:	Existing methods for ERC lack interpretability and shallow semantics capture deep semantics.
Approach:	They propose a Fast-Slow thinking framework for Emotion Recognition in Conversation . they use fine-grained emotion reasoning chains to capture deep semantics .
Outcome:	The proposed framework achieves state-of-the-art in explanation and judgment on a benchmark dataset.

Copied to clipboard

Xiaoqing Zhang, Yuhan Liu, Flood Sung, Xiuying Chen, Shuo Shang, Rui Yan

Programming Languages in NLP

Challenge:	Recent research indicates that large language models (LLMs) have demonstrated remark-able capabilities in various programming-related domains, such as code generation and code refinement.
Approach:	They propose a framework that combines exploration with refinement to reduce test-time computation overhead.
Outcome:	The proposed framework outperforms SOTA and AgentCoder on humanEval and MBPP benchmarks while reducing test-time computation overhead and scalability.

Open-world Multi-label Text Classification with Extremely Weak Supervision (2024.emnlp-main)

Copied to clipboard

Xintong Li, Jinya Jiang, Ria Dharmani, Jayanth Srinivasa, Gaowen Liu, Jingbo Shang

ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesis (2025.naacl-long)

Challenge:	Similar single-label XWS settings cannot be easily adapted for multi-l label classification.
Approach:	They propose a novel method for open-world multi-label text classification under extremely weak supervision where the user provides a brief description without any labels or ground-truth label space.
Outcome:	The proposed method exhibits a remarkable increase in ground-truth label space coverage on various datasets.

Copied to clipboard

Zezhong Wang, Xingshan Zeng, Weiwen Liu, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

Dialogue Systems & Conversational Agents

Challenge:	Large Language Models (LLMs) can be enhanced by using supervised fine-tuning . however, access to fine-timing data can be limited.
Approach:	They propose a Graph-based Sampling strategy and a Planned-generation strategy to enhance the coherence between dialogues by using 8,000 synthetic dialogues.
Outcome:	The proposed model achieves tool-calling performance comparable to or surpassing GPT-4 while maintaining strong general capabilities.

FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models (2024.acl-long)

Copied to clipboard

Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, Wei Wang

On the Role of Discriminative Models in Generative Relation Extraction (2026.acl-long)

Challenge:	Existing benchmarks focus on evaluating pure response quality, rather than assessing whether the response follows constraints stated in the instruction.
Approach:	They propose a Multi-level Fine-grained Constraints Following Benchmark for Large Language Models that adds a single constraint to the initial instruction at each increased level.
Outcome:	The proposed model can follow instructions with more constraints, and is deemed to have better instruction-following ability.

Copied to clipboard

Guozheng Li, Peng Wang, Zijie Xu, Jing Zhou, Jiajun Liu, Ziyu Shang

bert2BERT: Towards Reusable Pretrained Language Models (2022.acl-long)

Challenge:	Existing methods for relation extraction (RE) are discriminative and generative . previous studies show that discriminative models can support generative RE .
Approach:	They propose a framework that leverages discriminative models to produce a top-k set of candidate relations and integrates this knowledge into generative models via in-context or prompt learning.
Outcome:	The proposed framework achieves state-of-the-art on five widely used RE benchmarks.

Copied to clipboard

Cheng Chen, Yichun Yin, Lifeng Shang, Xin Jiang, Yujia Qin, Fengyu Wang, Zhi Wang, Xiao Chen, Zhiyuan Liu, Qun Liu

Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation (2022.findings-acl)

Challenge:	Pre-training large language models can be expensive and wasteful.
Approach:	They propose a method which can transfer the knowledge of an existing smaller pre-trained model to a large model through parameter initialization and a two-stage learning method to further accelerate the pre-training.
Outcome:	The proposed method can transfer the knowledge of an existing smaller pre-trained model to a large model through parameter initialization and significantly improve the pre-training efficiency of the large model.

Copied to clipboard

Wenliang Dai, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, Pascale Fung

Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Step (2025.findings-naacl)

Challenge:	Recent large-scale vision-language pre-training models are powerful in multimodal classification and retrieval tasks.
Approach:	They propose to augment a vision-language pre-training model with a textual pre-trained language model . the model achieves 44.5% zero-shot accuracy on multimodal generation tasks .
Outcome:	The proposed model achieves 44.5% zero-shot accuracy on open-ended visual question answering and image captioning tasks.

Copied to clipboard

Zezhong Wang, Xingshan Zeng, Weiwen Liu, Yufei Wang, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

PersonaAgent: Bridging Memory and Action for Personalized LLM Agents (2026.findings-acl)

Challenge:	Current research found the issue of Early Answering in large language models where the models already have an answer before generating the Chain-of-Thought (CoT).
Approach:	They propose a method to probe changes in confidence during the model’s reasoning and prioritize answers with correct reasoning among multiple candidates.
Outcome:	The proposed method reveals that in a significant number of question-answer cases, CoT appears to be unnecessary and this necessity correlates with the simplicity of the task, defined by the reasoning steps required.

Copied to clipboard

Weizhi Zhang, Xinyang Zhang, Chenwei Zhang, Liangwei Yang, Jingbo Shang, Zhepei Wei, Henry Peng Zou, Zijie Huang, Zhengyang Wang, Yifan Gao, Xiaoman Pan, Lian Xiong, Jingguo Liu, Philip S. Yu, Xian Li

Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training (2026.acl-industry)

Challenge:	Existing Large Language Model (LLM) enabled agents lack flexibility to respond to users’ varying needs and preferences.
Approach:	They propose a test-time user-preference alignment strategy that optimizes the persona prompt, ensuring real-time preference alignment through textual loss feedback between simulated and ground-truth responses.
Outcome:	The proposed framework outperforms baseline methods in real-time and in real applications.

Copied to clipboard

Tianle Xia, Ming Xu, Lingxiang Hu, Yiding Sun, Wenwei Li, Linfang Shang, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang

Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework (2025.findings-acl)

Challenge:	Retrieval-Augmented Generation (RAG) enhances large language models by incorporating external knowledge, yet traditional singleround retrieval struggles with complex multistep reasoning.
Approach:	They propose a framework that introduces path-centric reward shaping for agentic RAG training.
Outcome:	The proposed framework improves on existing methods with an average accuracy gain of 7.7 points.

Copied to clipboard

Kaishuai Xu, Tiezheng Yu, Yi Cheng, Wenjun Hou, Liangyou Li, Xin Jiang, Lifeng Shang, Qun Liu, Wenjie Li

Robustness in NLP

Challenge:	Existing methods for fine-tuning open-source LLMs are limited to text-based analysis under predefined general criteria.
Approach:	They propose a framework that fine-tunes LLMs to replicate the evaluation explanations and judgments of proprietary models.
Outcome:	The proposed evaluation framework outperforms existing fine-tuned evaluation methods in effectiveness and robustness.

Discover and Prove: An Open-source Agentic Framework for Hard Mode Automated Theorem Proving in Lean 4 (2026.acl-long)

Copied to clipboard

Chengwu Liu, Yichun Yin, Ye Yuan, Jiaxuan Xie, Botao Li, Siqi Li, Jianhao Shen, Yan Xu, Lifeng Shang, Ming Zhang

Towards Collaborative Neural-Symbolic Graph Semantic Parsing via Uncertainty (2022.findings-acl)

Challenge:	Existing approaches to solving mathematical problems fall into two broad categories: informal methods and formal methods.
Approach:	They propose to use LLM natural-language reasoning to discover answers . they introduce Discover And Prove framework that rewrites Hard Mode statements into Easy Mode ones for existing ATP provers.
Outcome:	The proposed framework can be used to prove hard mode statements on ATP benchmarks.

Copied to clipboard

Zi Lin, Jeremiah Zhe Liu, Jingbo Shang

Entity Resolution in Open-domain Conversations (2021.naacl-industry)

Challenge:	Recent work in task-independent graph semantic parsing has shifted from symbolic approaches to neural models, showing strong performance on different types of meaning representations.
Approach:	They propose a framework that incorporates prior knowledge from a symbolic parser into a decision criterion for beam search to address these limitations.
Outcome:	The proposed framework improves on the in-distribution test set but degrades significantly on long-tail situations while the symbolic parser performs more robustly.

Copied to clipboard

Mingyue Shang, Tong Wang, Mihail Eric, Jiangning Chen, Jiyang Wang, Matthew Welch, Tiantong Deng, Akshay Grewal, Han Wang, Yue Liu, Yang Liu, Dilek Hakkani-Tur

TinyBERT: Distilling BERT for Natural Language Understanding (2020.findings-emnlp)

Challenge:	Recent work on incorporating external knowledge into the response generation models has attracted great interest.
Approach:	They propose a neural entity linking approach to incorporate external knowledge into the response generation models to improve the relevancy of retrieved knowledge.
Outcome:	The proposed approach outperforms the baseline model by 62.8% relative to the baseline.

Copied to clipboard

Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun Liu

Unlocking Instructive In-Context Learning with Tabular Prompting for Relational Triple Extraction (2024.lrec-main)

Challenge:	Pre-trained language models are computationally expensive and difficult to efficiently execute on resource-restricted devices.
Approach:	They propose a Transformer distillation method that performs Transformer distillations at pre-training and task-specific learning stages.
Outcome:	The proposed method accelerates inference and reduces model size while maintaining accuracy.

Copied to clipboard

Guozheng Li, Wenjun Ke, Peng Wang, Zijie Xu, Ke Ji, Jiajun Liu, Ziyu Shang, Qiqing Luo

Low-Resource NLP

Challenge:	Existing methods for relational triple extraction (RTE) are unnatural and recast RTE tasks to text-to-text prompting formats.
Approach:	They propose a tabular prompting for RTE which frames RTE task into a table generation task and propose an instructive in-context learning which only selects and annotates samples considering triple semantics in massive unlabeled samples.
Outcome:	The proposed prompting for RTE with TableIE achieves state-of-the-art performance compared to other methods . the proposed prompts are based on off-the shelf LLMs and are scalable to multiple scenarios .

Learning Named Entity Tagger using Domain-Specific Dictionary (D18-1)

Copied to clipboard

Jingbo Shang, Liyuan Liu, Xiaotao Gu, Xiang Ren, Teng Ren, Jiawei Han

Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection (2026.acl-long)

Challenge:	Existing methods to build reliable named entity recognition systems require large amounts of manually-labeled training data.
Approach:	They propose a revised fuzzy CRF layer to handle tokens with multiple possible labels to address noisy distant supervision.
Outcome:	The proposed model can handle tokens with multiple possible labels under the traditional framework and improves on the existing model with a new Tie or Break scheme.

Copied to clipboard

Yuwei Zhang, Wenhao Yu, Shangbin Feng, Yifan Zhu, Letian Peng, Jayanth Srinivasa, Gaowen Liu, Jingbo Shang

Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Looping (2026.findings-acl)

Challenge:	Existing knowledge injection benchmarks for large language models lack standardized testing grounds.
Approach:	They propose a knowledge injection benchmark that leverages recently-added and expert-curated facts from Wikipedia’s “Did You Know...” entries.
Outcome:	The proposed framework improves reliability accuracy by 29.1%.

Copied to clipboard

Yao Chen, Yilong Chen, Yinqi Yang, Junyuan Shang, Zhenyu Zhang, Zefeng Zhang, Shuaiyi Nie, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang, Tingwen Liu

Improving Unsupervised Question Answering via Summarization-Informed Question Generation (2021.emnlp-main)

Challenge:	Existing approaches to increasing effective depth of LLMs rely on parameter reuse, extending computation through recursive execution.
Approach:	They propose a training-time sparse depth allocation framework that progressively increases depth for a small subset of parameters as training evolves.
Outcome:	The proposed model outperforms existing approaches to increasing the effective depth of language models while reducing training FLOPs overhead from approximately 16–20% to only 1–3% relative to a standard Transformer backbone.

Copied to clipboard

Chenyang Lyu, Lifeng Shang, Yvette Graham, Jennifer Foster, Xin Jiang, Qun Liu

Text Generation

Challenge:	Question Generation (QG) is the production of meaningful questions given a set of input passages and corresponding answers.
Approach:	They propose a method which uses questions generated heuristically from news summaries as a source of training data for a QG system.
Outcome:	The proposed method outperforms previous unsupervised models on three in-domain datasets and three out-of-domain ones.

Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios (2024.findings-acl)

Copied to clipboard

Shijue Huang, Wanjun Zhong, Jianqiao Lu, Qi Zhu, Jiahui Gao, Weiwen Liu, Yutai Hou, Xingshan Zeng, Yasheng Wang, Lifeng Shang, Xin Jiang, Ruifeng Xu, Qun Liu

More is not always better? Enhancing Many-Shot In-Context Learning with Differentiated and Reweighting Objectives (2025.acl-long)

Challenge:	Existing benchmarks focus on simple synthesized queries that do not reflect real-world complexity, thereby offering limited perspectives in evaluating tool utilization.
Approach:	They propose a benchmark to evaluate LLMs’ ability in tool utilization within real-world scenarios.
Outcome:	The proposed benchmark improves LLMs’ ability in tool utilization within real-world scenarios and eliminates the restriction of pre-defined toolset.

Copied to clipboard

Xiaoqing Zhang, Ang Lv, Yuhan Liu, Flood Sung, Wei Liu, Jian Luan, Shuo Shang, Xiuying Chen, Rui Yan

NewsDialogues: Towards Proactive News Grounded Conversation (2023.findings-acl)

Challenge:	Large language models excel at few-shot in-context learning but performance plateaus as ICL demonstrations increase from a few to many.
Approach:	They propose a novel optimization method that optimizes the negative log-likelihood objective and reweights the model to achieve many-shot performance.
Outcome:	The proposed method achieves significant performance improvements across a large-scale dataset.

Copied to clipboard

Siheng Li, Yichun Yin, Cheng Yang, Wangjie Jiang, Yiwei Li, Zesen Cheng, Lifeng Shang, Xin Jiang, Qun Liu, Yujiu Yang

Answer is All You Need: Instruction-following Text Embedding via Answering the Question (2024.acl-long)

Challenge:	Hot news is one of the most popular topics in daily conversations.
Approach:	They propose a task where a dialogue system can lead the conversation based on key topics of the news.
Outcome:	The proposed method can lead conversations based on key topics of the news . it can also be used in information-seeking and chit-chat scenarios .

Copied to clipboard

Letian Peng, Yuwei Zhang, Zilong Wang, Jayanth Srinivasa, Gaowen Liu, Zihan Wang, Jingbo Shang

SafeScientist: Enhancing AI Scientist Safety for Risk-Aware Scientific Discovery (2025.emnlp-main)

Challenge:	Existing methods for encoding instruction information fail to be sensitive to clearer criteria like “evaluate similarity based on emotion” . instead, we propose a different approach, which treats the instruction as a “question” about the input text and encodes the expected answers to obtain the representation accordingly.
Approach:	They propose a text embedder that captures characteristics of texts specified by user instructions clarifying the similarity criterion.
Outcome:	The proposed model improves instruction-following capabilities when applied to large language models and encoder-based LMs.

Copied to clipboard

Kunlun Zhu, Jiaxun Zhang, Ziheng Qi, Nuoxing Shang, Zijia Liu, Peixuan Han, Yue Su, Haofei Yu, Jiaxuan You

Enhancing Large Language Model for Knowledge Graph Completion via Structure-Aware Alignment-Tuning (2025.emnlp-main)

Challenge:	Recent advances in large language model (LLM) agents have significantly accelerated scientific discovery automation, yet raised critical ethical and safety concerns.
Approach:	They propose a framework to enhance safety and ethical responsibility in AI-driven scientific exploration.
Outcome:	The proposed framework significantly improves safety performance by 35% compared to traditional frameworks.

Copied to clipboard

Yu Liu, Yanan Cao, Xixun Lin, Yanmin Shang, Shi Wang, Shirui Pan

MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models (2024.emnlp-main)

Challenge:	Existing knowledge graph completion methods ignore inconsistent representation spaces between natural language and graph structures, leading to duplicate works and time-consuming processes.
Approach:	They propose a framework that enhances LLMs for KGC via structure-aware alignment-tuning to align graph embeddings with the natural language space through multi-task contrastive learning.
Outcome:	The proposed framework outperforms state-of-the-art methods on two KGC tasks across four benchmark datasets.

Copied to clipboard

Wai-Chung Kwan, Xingshan Zeng, Yuxin Jiang, Yufei Wang, Liangyou Li, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models (2021.acl-long)

Challenge:	Existing evaluation frameworks focus on single-turn evaluations, overlooking the models’ capabilities in multi-turn interactions.
Approach:	They propose a benchmark to evaluate the multi-turn conversational abilities of large language models (LLMs) by analyzing human-LLM conversations and constructing multi-turned queries for each category using GPT-4.
Outcome:	The proposed model outperforms open-source models in multi-turn tasks while retaining and recalling historical information.

Copied to clipboard

Yichun Yin, Cheng Chen, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu

Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering (2022.acl-long)

Challenge:	Pre-trained language models (PLMs) have achieved great success in natural language processing.
Approach:	They propose a method that automatically searches architecture hyper-parameters in BERT . they use one-shot learning and the search space to provide an adaptive development way .
Outcome:	The proposed method outperforms both the baseline and distillation-based methods on GLUE and SQUAD benchmarks.

Copied to clipboard

Jiawei Zhou, Xiaoguang Li, Lifeng Shang, Lan Luo, Ke Zhan, Enrui Hu, Xinyu Zhang, Hao Jiang, Zhao Cao, Fan Yu, Xin Jiang, Qun Liu, Lei Chen

Enhancing Agentic Textual Graph Retrieval with Synthetic Stepwise Supervision (2026.acl-long)

Challenge:	Existing methods to train dense passage retrieval have a large data gap between upstream and downstream relevance.
Approach:	They propose a method to pre-train the dense retriever with the text relevance induced by hyperlinks within Web documents.
Outcome:	The proposed method outperforms existing methods under different scenarios and in the open-domain question answering domain.

Copied to clipboard

Ge Chang, Jinbo Su, Jiacheng Liu, Pengfei Yang, Yuhao Shang, Huiwen Zheng, Hongli Ma, Yan Liang, Yuanchun Li, Yunxin Liu