Papers by Jingang Wang

SMARTAVE: Structured Multimodal Transformer for Product Attribute Value Extraction (2022.findings-emnlp)

Challenge:	Existing methods for finding out-of-domain intents suffer from in-domain overfitting problem . previous methods fail to transfer prior knowledge to downstream clustering .
Approach:	They propose a unified K-nearest neighbor contrastive learning framework to discover OOD intents . they propose IND pre-training objective to learn discriminative features while maintaining intra-class diversity .
Outcome:	The proposed framework improves on three benchmark datasets.

Copied to clipboard

Qifan Wang, Li Yang, Jingang Wang, Jitin Krishnan, Bo Dai, Sinong Wang, Zenglin Xu, Madian Khabsa, Hao Ma

Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification (2022.acl-long)

Challenge:	Existing methods for product attribute value extraction are noisy and incomplete with missing values for most retailers.
Approach:	They propose a Structure Mltimodal trAnsformeR for producT Attribute Value Extraction which jointly encodes the structured product information from multiple modalities.
Outcome:	The proposed method outperforms state-of-the-art methods on two multimodal product datasets.

Copied to clipboard

Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Jingang Wang, Juanzi Li, Wei Wu, Maosong Sun

Graph-Structured Speculative Decoding (2024.findings-acl)

Challenge:	Recent studies suggest that pre-trained language models have gained rich knowledge during pre-training.
Approach:	They propose to tune pre-trained language models with task-specific prompts to improve and stabilize prompttuning.
Outcome:	Extensive experiments on zero and few-shot text classification tasks show that prompt-tuning improves and stabilizes prompttun-ing.

Copied to clipboard

Zhuocheng Gong, Jiahao Liu, Ziyue Wang, Pengfei Wu, Jingang Wang, Xunliang Cai, Dongyan Zhao, Rui Yan

Green & Sustainable NLP

Challenge:	Speculative decoding is a promising technique to accelerate the inference of Large Language Models.
Approach:	They propose a method that uses a token graph to record multiple sequence hypotheses within a single draft stage.
Outcome:	The proposed method significantly accelerates the inference of Large Language Models (LLMs) it allows the LLM to choose from and select the longest sequence that meets its standards.

Generalized Intent Discovery: Learning from Open World Dialogue System (2022.coling-1)

Copied to clipboard

Yutao Mou, Keqing He, Yanan Wu, Pei Wang, Jingang Wang, Wei Wu, Yi Huang, Junlan Feng, Weiran Xu

FutureTOD: Teaching Future Knowledge to Pre-trained Language Model for Task-Oriented Dialogue (2023.acl-long)

Challenge:	Existing intent classification models rely on a pre-defined intent set and supervised labels, which is limited in some practical scenarios.
Approach:	They propose to extend an IND intent classifier to an open-world intent set including IND and OOD intents.
Outcome:	The proposed task can classify IND and OOD intents while discovering new unlabeled OOD types incrementally.

Copied to clipboard

Weihao Zeng, Keqing He, Yejie Wang, Chen Zeng, Jingang Wang, Yunsen Xian, Weiran Xu

Improving Input-label Mapping with Demonstration Replay for In-context Learning (2023.findings-emnlp)

Challenge:	Existing pre-trained language models rely on a contrastive framework and are difficult to use in practice.
Approach:	They propose a dialogue pre-training model which distills future knowledge to the representation of the previous dialogue context using a self-training framework.
Outcome:	The proposed model can be applied to various downstream dialogue tasks.

Copied to clipboard

Zhuocheng Gong, Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai, Dongyan Zhao, Rui Yan

Seen to Unseen: Exploring Compositional Generalization of Multi-Attribute Controllable Dialogue Generation (2023.acl-long)

Challenge:	In-context learning (ICL) is an emerging capability of large autoregressive language models where a few demonstrations are appended to the input to enhance the model’s understanding of downstream NLP tasks without directly adjusting the model parameters.
Approach:	They propose a method where a few demonstrations are appended to the input to enhance the model's understanding of downstream NLP tasks without directly adjusting the model parameters.
Outcome:	The proposed method significantly improves the input-label mapping in ICL demonstrations.

Copied to clipboard

Weihao Zeng, Lulu Zhao, Keqing He, Ruotong Geng, Jingang Wang, Wei Wu, Weiran Xu

Large-Scale Diverse Synthesis for Mid-Training (2026.findings-acl)

Challenge:	Existing controllable dialogue generation models focus on single attribute and lack generalization capability to out-of-distribution multiple attribute combinations.
Approach:	They propose a compositional generalization model that learns from seen attributes and generalizes to unseen combinations.
Outcome:	The proposed model can learn from seen attribute values and generalize to unseen combinations.

Copied to clipboard

Xuemiao Zhang, Chengying Tu, Can Ren, Rongxiang Weng, Hongfei Yan, Jingang Wang, Xunliang Cai

Question Answering

Challenge:	Existing data synthesis methods generate simplistic and homogeneous QA pairs with limited scale and diversity.
Approach:	They propose a framework to synthesize large-scale, diverse, and high-quality QA data for mid-training.
Outcome:	The proposed framework improves on 500B-token BoostQA data over pre-training benchmarks.

UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning (2022.emnlp-main)

Copied to clipboard

Yutao Mou, Pei Wang, Keqing He, Yanan Wu, Jingang Wang, Wei Wu, Weiran Xu

Multi-Programming Language Sandbox for LLMs (2025.acl-demo)

Challenge:	Existing methods to detect out-of-domain (OOD) intents ignore alignment between representation learning and scoring function, limiting performance.
Approach:	They propose a unified neighborhood learning framework to detect OOD intents . they propose to align representation learning with scoring function .
Outcome:	The proposed method is able to detect out-of-domain (OOD) intents from user queries.

Copied to clipboard

Shihan Dou, Jiazheng Zhang, Jianxiang Zang, Yunbo Tao, Weikang Zhou, Haoxiang Jia, Shichun Liu, Yuming Yang, Shenxi Wu, Zhiheng Xi, Muling Wu, Rui Zheng, Changze Lv, Limao Xiong, Shaoqing Zhang, Lin Zhang, Wenyu Zhan, Rongxiang Weng, Jingang Wang, Xunliang Cai, Yueming Wu, Ming Wen, Yixin Cao, Tao Gui, Xipeng Qiu, Qi Zhang, Xuanjing Huang

LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points (2026.acl-long)

Challenge:	MPLSandbox is an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs).
Approach:	They propose a multi-programming language sandbox that provides unified feedback from compilers and analysis tools for Large Language Models.
Outcome:	The proposed multi-language sandbox can provide comprehensive feedback from compilers and analysis tools for large language models (LLMs).

Copied to clipboard

Xuemiao Zhang, Can Ren, Chengying Tu, Rongxiang Weng, Hongfei Yan, Jingang Wang, Xunliang Cai

FRAME: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy (2025.findings-acl)

Challenge:	Existing training data is limited in high-quality training data, limiting the ability to produce high-performance LLMs.
Approach:	They propose a KP-graph-based synthesis framework that extracts KPs from QA seed data and constructs a graph of KP data from multiple seeds strongly linked by KP.
Outcome:	The proposed framework enables flexible control over discipline and difficulty distributions while balancing KP coverage and popularity.

Copied to clipboard

Xuemiao Zhang, Feiyu Duan, Xu Liangyu, Yongwei Zhou, Sirui Wang, Rongxiang Weng, Jingang Wang, Xunliang Cai

Structural Bias for Aspect Sentiment Triplet Extraction (2022.coling-1)

Challenge:	Multi-stage pretraining methods lack quantitative criteria for data partitioning and instead rely on intuitive heuristics.
Approach:	They propose a Four-quadRAnt Multi-stage prEtraining strategy that partitions data into four quadrants to achieve significant loss reductions four times.
Outcome:	The proposed strategy achieves 16.8% improvement over random across MMLU and CMMLU for the 3B model.

Copied to clipboard

Chen Zhang, Lei Ren, Fang Ma, Jingang Wang, Wei Wu, Dawei Song

Sentiment Analysis

Challenge:	Existing structural bias adapters for aspect sentiment triplet extraction are under-confident . a large-scale dataset for ASTE shows the adapter is effective and efficient to a larger scale.
Approach:	They propose to use a structural adapter to integrate structural bias into pretrained language models . they propose to add a relative position structure in place of the syntactic dependency structure .
Outcome:	The proposed adapter achieves state-of-the-art performance over strong baselines, but with a light parameter demand and low latency.

C-ICL: Contrastive In-context Learning for Information Extraction (2024.findings-emnlp)

Copied to clipboard

Ying Mo, Jiahao Liu, Jian Yang, Qifan Wang, Shun Zhang, Jingang Wang, Zhoujun Li

Low-Resource NLP

Challenge:	Existing methods for in-context learning with large language models focus on using correct or negative examples, ignoring the potential value of incorrect or negative samples.
Approach:	They propose a few-shot technique that leverages both correct and incorrect sample constructions to create in-context learning demonstrations.
Outcome:	The proposed technique outperforms previous few-shot in-context learning methods on a broad spectrum of related tasks.

TCPO: Thought-Centric Preference Optimization for Effective Embodied Decision-making (2025.emnlp-main)

Copied to clipboard

Kechen Jiao, Zhirui Fang, Jiahao Liu, Bei Li, Qifan Wang, Xinyu Liu, Junhao Ruan, Zhongjian Qiao, Yifan Zhu, Yaxin Xu, Jingang Wang, Xiu Li

Leveraging Unpaired Feedback for Long-Term LLM-based Recommendation Tuning (2025.findings-emnlp)

Challenge:	Existing post-SFT methods for embodied AI are constrained by sparse rewards and action-only optimization, resulting in low sample efficiency, poor consistency, and model degradation.
Approach:	They propose to integrate Thought-Centric Preference Optimization (TCPO) into embodied decision-making by transforming sparse reward signals into richer step sample pairs.
Outcome:	The proposed approach achieves an average success rate of 26.67% in the ALFWorld environment, and a 6% improvement over RL4VLM.

Copied to clipboard

Jizhi Zhang, Chongming Gao, Wentao Shi, Xin Chen, Jingang Wang, Xunliang Cai, Fuli Feng

Challenge:	a recent study highlights unpaired feedback as a key challenge for long-term LLM-based recommenders . unpaired user feedback is crucial for improving LLMs in dynamic user environments, authors say .
Approach:	They propose a framework that incorporates unpaired feedback into LLMs to improve long-term recommendation performance.
Outcome:	The proposed framework improves long-term recommendation performance by incorporating unpaired feedback without requiring paired supervision.

MUSTIE: Multimodal Structural Transformer for Web Information Extraction (2023.acl-long)

Copied to clipboard

Qifan Wang, Jingang Wang, Xiaojun Quan, Fuli Feng, Zenglin Xu, Shaoliang Nie, Sinong Wang, Madian Khabsa, Hamed Firooz, Dongfang Liu

GNN-encoder: Learning a Dual-encoder Architecture via Graph Neural Networks for Dense Passage Retrieval (2022.findings-emnlp)

Challenge:	Recent sequential modeling approaches focus on extracting information from textual sources while ignoring rich information from other modalities such as image and web layout.
Approach:	They propose a novel MUltimodal Structural Transformer that integrates multiple modalities for web information extraction.
Outcome:	The proposed model outperforms existing methods on WebSRC and Common Crawl benchmarks.

Copied to clipboard

Jiduan Liu, Jiahao Liu, Yang Yang, Jingang Wang, Wei Wu, Dongyan Zhao, Rui Yan

Information Retrieval

Challenge:	Existing approaches to perform large-scale query-passage retrieval are term-based, but they lose interaction between query-pastage pairs.
Approach:	They propose to fuse query (passage) information into query representations via graph neural networks that are constructed by queries and their top retrieved passages.
Outcome:	The proposed model outperforms existing models on MSMARCO, Natural Questions and TriviaQA datasets and achieves the new state-of-the-art on these datasets.

ASAP: A Chinese Review Dataset Towards Aspect Category Sentiment Analysis and Rating Prediction (2021.naacl-main)

Copied to clipboard

Jiahao Bu, Lei Ren, Shuang Zheng, Yang Yang, Jingang Wang, Fuzheng Zhang, Wei Wu

Sentiment Analysis

Challenge:	Sentiment analysis is a key task in e-commerce to detect fine-to-coarse sentiment polarities.
Approach:	They propose to use a large-scale Chinese restaurant review dataset ASAP to investigate the sentiment polarities underlying user reviews.
Outcome:	The proposed model outperforms state-of-the-art models on both tasks.

How Do Your Code LLMs perform? Empowering Code Instruction Tuning with Really Good Data (2024.emnlp-main)

Copied to clipboard

Yejie Wang, Keqing He, Dayuan Fu, Zhuoma GongQue, Heyang Xu, Yanxu Chen, Zhexu Wang, Yujia Fu, Guanting Dong, Muxi Diao, Jingang Wang, Mengdi Zhang, Xunliang Cai, Weiran Xu

Programming Languages in NLP

Challenge:	Recent research has shown that code pre-trained models improve coding capabilities.
Approach:	They propose a code data pruning strategy to identify which datasets are high-quality code instruction data.
Outcome:	The proposed model achieves state-of-the-art performance using fewer training data.

CLOWER: A Pre-trained Language Model with Contrastive Learning over Word and Character Representations (2022.coling-1)

Copied to clipboard

Borun Chen, Hongyin Tang, Jiahao Bu, Kai Zhang, Jingang Wang, Qifan Wang, Hai-Tao Zheng, Wei Wu, Liqian Yu

ReMamba: Equip Mamba with Effective Long-Sequence Modeling (2025.findings-emnlp)

Challenge:	Pre-trained language models (PLMs) have achieved remarkable performance gains across numerous downstream tasks in natural language understanding.
Approach:	They propose a Chinese pre-trained language model that implicitly encodes words into characters . they propose 'contrastive learning over word' and 'character' representations to improve learning .
Outcome:	The proposed model can encode words into fine-grained representations without modification of production pipelines.

Copied to clipboard

Danlong Yuan, Jiahao Liu, Bei Li, Huishuai Zhang, Jingang Wang, Xunliang Cai, Dongyan Zhao

Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training (2025.findings-acl)

Challenge:	Mamba models demonstrate superior inference efficiency and competitive performance on short-context tasks, but their capacity to comprehend long contexts is limited compared to transformer-based models.
Approach:	They propose a model which incorporates selective compression and adaptation techniques within a two-stage re-forward process, incurring minimal additional inference costs overhead.
Outcome:	The proposed model improves on the LongBench and L-Eval benchmarks by 3.2 and 1.6 points and attains performance almost on par with same-size transformer models.

Copied to clipboard

Zhijun Wang, Jiahuan Li, Hao Zhou, Rongxiang Weng, Jingang Wang, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang

Multilinguality

Challenge:	Large language models (LLMs) exhibit remarkable multilingual capabilities despite the extreme language imbalance in the pre-training data.
Approach:	They investigate the existence of code-switching in the pre-training corpus and categorize it into four types within two quadrants.
Outcome:	The proposed approach improves performance across benchmarks and representation space.

Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval (2021.acl-long)

Copied to clipboard

Hongyin Tang, Xingwu Sun, Beihong Jin, Jingang Wang, Fuzheng Zhang, Wei Wu

Representation Learning

Challenge:	Existing retrieval models based on dense representations show better performance than sparse representations.
Approach:	They propose a method to mimic the queries to each of the documents by an iterative clustering process and represent the documents using multiple pseudo queries.
Outcome:	The proposed model achieves state-of-the-art results on a large dataset while remaining high efficiency.

DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning (2024.acl-long)

Copied to clipboard

Yejie Wang, Keqing He, Guanting Dong, Pei Wang, Weihao Zeng, Muxi Diao, Weiran Xu, Jingang Wang, Mengdi Zhang, Xunliang Cai

Revisiting Scaling Laws for Language Models: The Role of Data Quality and Training Strategies (2025.acl-long)

Challenge:	Numerous code large language models (LLMs) have been proposed to enhance code generation performance.
Approach:	They propose a diverse instruction model DolphCoder with self-evaluating for code generation that learns diverse instruction targets and combines a code evaluation objective to enhance its code generation ability.
Outcome:	The proposed model achieves superior performance on the HumanEval and MBPP benchmarks, demonstrating new insights for future code instruction tuning work.

Copied to clipboard

Zhengyu Chen, Siqi Wang, Teng Xiao, Yudong Wang, Shiqi Chen, Xunliang Cai, Junxian He, Jingang Wang

MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarks (2026.acl-long)

Challenge:	Existing scaling laws suggest augmenting model size and training data results in enhanced performance, but recent studies reveal deviations, particularly in large language models, where performance improvements decelerate—a phenomenon known as sub-scaling.
Approach:	They propose a sub-optimal scaling law that better predicts performance in sub-scaling regimes by examining data quality and training strategies.
Outcome:	The proposed scaling law better predicts performance in sub-scaling regimes, highlighting the importance of data quality and diversity.

Copied to clipboard

Junhao Ruan, Abudukeyumu Abudula, Bei Li, Yongjing Yin, Xinyu Liu, Kechen Jiao, Xin Chen, Jingang Wang, Xunliang Cai, Tong Xiao, JingBo Zhu

Natural Language Interfaces

Challenge:	Existing conversational retrieval benchmarks suffer from costly, sparse human annotation or rigid, unnatural automated heuristics.
Approach:	They propose a framework for auditing, synthesizing, and benchmarking conversational retrieval.
Outcome:	The proposed framework is based on three LLM-based auditors and a multi-agent system . it mimics production-style challenges (hard topic switching, verbosity) and offers superior discriminative power.

IIET: Efficient Numerical Transformer via Implicit Iterative Euler Method (2025.emnlp-main)

Copied to clipboard

Xinyu Liu, Bei Li, Jiahao Liu, Junhao Ruan, Kechen Jiao, Hongyin Tang, Jingang Wang, Tong Xiao, JingBo Zhu

Green & Sustainable NLP

Challenge:	High-order numerical methods enhance performance in tasks like NLP but introduce a performance-efficiency trade-off due to increased computational overhead.
Approach:	They propose an iterative implicit Euler Transformer which simplifies high-order numerical methods by iterating implicit Eule.
Outcome:	The proposed method improves accuracy and reduces inference overhead by 55% while retaining 99.4% of the original task accuracy.

Task-agnostic Distillation of Encoder-Decoder Language Models (2024.lrec-main)

Copied to clipboard

Chen Zhang, Yang Yang, Qiuchi Li, Jingang Wang, Dawei Song

MixPAVE: Mix-Prompt Tuning for Few-shot Product Attribute Value Extraction (2023.findings-acl)

Challenge:	Existing distillation methods that focus on encoder-only LMs fail to handle the distillation of encoder decoder LM.
Approach:	They propose a method that finetunes pretrained language models (LMs) they propose 'MiniEnD' that allows for task-agnostic distillation of LMs.
Outcome:	The proposed distillation method is generally effective and competitive compared to other alternatives.

Copied to clipboard

Li Yang, Qifan Wang, Jingang Wang, Xiaojun Quan, Fuli Feng, Yu Chen, Madian Khabsa, Sinong Wang, Zenglin Xu, Dongfang Liu

Sibyl: Empowering Empathetic Dialogue Generation in Large Language Models via Sensible and Visionary Commonsense Inference (2025.coling-main)

Challenge:	Existing methods for product attribute value extraction focus on extracting values for a set of known attributes with sufficient training data.
Approach:	They propose a prompt tuning approach to extract attributes from product information using mixed prompts.
Outcome:	The proposed approach improves on two product benchmarks and shows parameter-efficient training and avoids model overfitting.

Copied to clipboard

Lanrui Wang, Jiangnan Li, Chenxu Yang, Zheng Lin, Hongyin Tang, Huan Liu, Yanan Cao, Jingang Wang, Weiping Wang

Jailbreak LLMs through Internal Stance Manipulation (2025.emnlp-main)

Challenge:	Recent studies have focused on integrating commonsense knowledge into chatbots to enhance their ability to understand and generate dialogue responses.
Approach:	They propose a framework that integrates commonsense knowledge into chatbots to enable them to elicit more empathetic responses.
Outcome:	The proposed framework enables LLMs to uncover the implicit requirements of the conversation, aiming to elicit more empathetic responses.

Copied to clipboard

Shuangjie Fu, Du Su, Beining Huang, Fei Sun, Jingang Wang, Wei Chen, Huawei Shen, Xueqi Cheng

Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT (2023.emnlp-main)

Challenge:	Existing approaches to exploit LLMs' inherent safety mechanism, including GCG and AutoDAN, are ineffective for certain malicious requests.
Approach:	They propose a method that generates jailbreak prompts to suppress a refusal stance and induce affirmative responses by modifying adversarial prompts.
Outcome:	The proposed method outperforms the best baseline approach in Llama-2-7b-chat and achieves a 92.2% success rate across all models.

Copied to clipboard

Xiaoshuai Song, Keqing He, Pei Wang, Guanting Dong, Yutao Mou, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu

VIRT: Improving Representation-based Text Matching via Virtual Interaction (2022.emnlp-main)

Challenge:	Existing methods to fine-tune discriminative models address these challenges by focusing on in-domain intents.
Approach:	They evaluate ChatGPT on OOD intent discovery and generalized intent discovery tasks . they outline the strengths and weaknesses of ChatGPt and outline their results .
Outcome:	The proposed task aims to extend a closed intent classifier to open-world intent sets.

Copied to clipboard

Dan Li, Yang Yang, Hongyin Tang, Jiahao Liu, Qifan Wang, Jingang Wang, Tong Xu, Wei Wu, Enhong Chen

XPrompt: Exploring the Extreme of Prompt Tuning (2022.emnlp-main)

Challenge:	Experimental results show that representation-based text matching methods suffer from performance degradation due to the lack of interactions between the pair of texts.
Approach:	They propose a virtual interaction mechanism that enables deep interaction between texts . they propose 'inteRacTion mechanism' that can be integrated into existing methods as plugins .
Outcome:	The proposed method outperforms state-of-the-art models on six text matching benchmarks.

Copied to clipboard

Fang Ma, Chen Zhang, Lei Ren, Jingang Wang, Qifan Wang, Wei Wu, Xiaojun Quan, Dawei Song

BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search (2026.findings-acl)

Challenge:	Prompt tuning learns soft prompts to condition pre-trained Language Models for performing downstream tasks in a parameter-efficient manner.
Approach:	They propose a Prompt tuning model with an eXtremely small scale that learns soft prompts to condition the frozen Pre-trained Language Models for performing downstream tasks in a parameter-efficient manner.
Outcome:	The proposed model outperforms the vanilla Prompt-Tuning and can significantly improve across tasks and model scales.

Copied to clipboard

Shiyu Liu, Yongjing Yin, Jianhao Yan, Yunbo Tang, Qinggang Zhang, Bei Li, Xin Chen, Jingang Wang, Xunliang Cai, Jinsong Su

Reasoning

Challenge:	Existing RL-based agentic search models fail to recognize reasoning boundaries and rarely admit "I DON'T KNOW" lack of reliability leads to plausible but unreliable answers, introducing significant risks .
Approach:	They propose a framework to cultivate reliable boundary awareness without compromising accuracy.
Outcome:	Experiments show that the proposed framework improves the reliability of agentic search models.

Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs (2025.emnlp-main)

Copied to clipboard

Hexiang Tan, Fei Sun, Sha Liu, Du Su, Qi Cao, Xin Chen, Jingang Wang, Xunliang Cai, Yuanzhuo Wang, Huawei Shen, Xueqi Cheng

Challenge:	Existing detection methods fail to account for self-consistent error . study identifies self-consistency errors and evaluates them .
Approach:	They propose a method that fuses hidden state evidence from an external verifier LLM to detect self-consistent errors.
Outcome:	The proposed method significantly enhances performance on self-consistent errors across three LLM families.

Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-Context Models (2024.emnlp-main)

Copied to clipboard

Xinyu Liu, Runsong Zhao, Pengcheng Huang, Chunyang Xiao, Bei Li, Jingang Wang, Tong Xiao, JingBo Zhu

FIRE: Flexible Integration of Data Quality Ratings for Effective Pretraining (2025.emnlp-main)

Challenge:	Existing methods, tasks and benchmarks to measure model’s effective memory length are limited.
Approach:	They propose a method called forgetting curve to measure the memorization capability of long-context models.
Outcome:	The proposed method is robust to the tested corpus and experimental settings, and can be applied to any model size.

Copied to clipboard

Xu Liangyu, Xuemiao Zhang, Feiyu Duan, Sirui Wang, Rongxiang Weng, Jingang Wang, Xunliang Cai

Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection (2024.lrec-main)

Challenge:	Existing methods to evaluate data quality rely on heuristic techniques or single quality signals.
Approach:	They propose a framework for integrating multiple data quality raters that integrates multiple quality signals into a unified space and provides a comprehensive quality signal for each data point.
Outcome:	The proposed framework outperforms existing methods and boosts model performance across a wide range of downstream tasks while requiring less than 37.5% tokens to reach the target performance.

Copied to clipboard

Pei Wang, Keqing He, Yejie Wang, Xiaoshuai Song, Yutao Mou, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu

Sentiment Analysis

Challenge:	Out-of-domain (OOD) intent detection is crucial for task-oriented dialogue systems.
Approach:	They conduct a comprehensive evaluation of large language models (LLMs) under various experimental settings and outline their strengths and weaknesses.
Outcome:	The proposed models exhibit strong zero-shot and few-shot capabilities, but is still at a disadvantage compared to models fine-tuned with full resource.

Minimal Distillation Schedule for Extreme Language Model Compression (2024.findings-eacl)

Copied to clipboard

Chen Zhang, Yang Yang, Qifan Wang, Jiahao Liu, Jingang Wang, Wei Wu, Dawei Song

Lifting the Curse of Capacity Gap in Distilling Language Models (2023.acl-long)

Challenge:	Existing methods for teacher assistant-based distillation require multiple trials to find the optimal teacher assistant.
Approach:	They propose a method that allows scheduling of an optimal teacher assistant in just one trial . they show that student performance is positively correlated with the scale-performance tradeoff .
Outcome:	The proposed method can select the optimal teacher assistant in just one trial . it can be used to compare performance of student and teacher assistants on GLUE benchmarks.

Copied to clipboard

Chen Zhang, Yang Yang, Jiahao Liu, Jingang Wang, Yunsen Xian, Benyou Wang, Dawei Song

RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank (2023.acl-long)

Challenge:	Existing studies have shown that pretrained language models require a tremendous amount of inference compute to perform.
Approach:	They propose to compress pretrained language models to small ones with a teacher-student paradigm to fill the capacity gap.
Outcome:	The proposed model achieves state-of-the-art performance at small FLOPs compared with competitive baselines.

Copied to clipboard

Jiduan Liu, Jiahao Liu, Qifan Wang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Kai Chen, Rui Yan

APrompt: Attention Prompt Tuning for Efficient Adaptation of Pre-trained Language Models (2023.emnlp-main)

Challenge:	Unsupervised sentence representation learning is one of the fundamental problems in natural language processing . contrastive learning methods fail to capture fine-grained ranking information among the sentences .
Approach:	They propose a novel approach for unsupervised sentence representation learning that integrates ranking consistency and ranking distillation with contrastive learning into a unified framework.
Outcome:	The proposed approach performs better over state-of-the-art models on STS and TR tasks.

Copied to clipboard

Qifan Wang, Yuning Mao, Jingang Wang, Hanchao Yu, Shaoliang Nie, Sinong Wang, Fuli Feng, Lifu Huang, Xiaojun Quan, Zenglin Xu, Dongfang Liu

Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression (2023.findings-emnlp)

Challenge:	Existing prompt tuning methods only introduce prompts at the input layer, limiting performance and leaving large room for improvement.
Approach:	They propose a method that involves tuning a small set of soft prompts for pre-trained language models.
Outcome:	The proposed method outperforms state-of-the-art methods with pre-trained models on the SuperGLUE benchmark.

Copied to clipboard

Jiduan Liu, Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai, Dongyan Zhao, Ran Wang, Rui Yan

Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discovery (2023.acl-long)

Challenge:	Large-scale pre-trained language models (LLMs) have demonstrated exceptional performance in various natural language processing (NLP) tasks.
Approach:	They propose a new compression paradigm that extracts knowledge from pre-trained language models to construct a knowledge store from which the model can leverage it for effective inference.
Outcome:	The proposed model extracts knowledge from LLMs to construct a knowledge store, which the model can leverage for effective inference.

Copied to clipboard

Yutao Mou, Xiaoshuai Song, Keqing He, Chen Zeng, Pei Wang, Jingang Wang, Yunsen Xian, Weiran Xu

APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection (2023.findings-emnlp)

Challenge:	Existing methods for generalized intent discovery lack pseudo label disambiguation and representation learning.
Approach:	They propose a prototype learning framework to decouple pseudo label disambiguation and representation learning.
Outcome:	The proposed method can decouple pseudo label disambiguation and representation learning.

Copied to clipboard

Pei Wang, Keqing He, Yutao Mou, Xiaoshuai Song, Yanan Wu, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu

Low-Resource NLP

Challenge:	Existing methods for detecting out-of-domain (OOD) intents are hard to label . previous studies use labeled in-domain data to learn intent representations .
Approach:	They propose a prototypical pseudo-labeling method for few-shot OOD detection . they propose 'protoOOD' framework and adaptive pseudo-labeled method .
Outcome:	The proposed method is able to detect out-of-domain (OOD) intents from user queries.

Let’s Ask GNN: Empowering Large Language Model for Graph In-Context Learning (2024.findings-emnlp)

Copied to clipboard

Zhengyu Hu, Yichuan Li, Zhengyu Chen, Jingang Wang, Han Liu, Kyumin Lee, Kaize Ding

Dynamic Fisher-weighted Model Merging via Bayesian Optimization (2025.naacl-long)

Challenge:	Textual Attributed Graphs (TAGs) are crucial for modeling complex real-world systems, yet leveraging large language models (LLMs) for TAGs presents unique challenges due to the gap between sequential text processing and graph-structured data.
Approach:	They propose a novel approach that leverages In-Context Learning to integrate graph data and task-specific information into large language models (LLMs) they employ a Graph Neural Network-powered structure-enhanced retriever to select labeled nodes across graphs, incorporating complex graph structures and their supervision signals.
Outcome:	Experiments on three tasks and seven LLMs show that AskGNN performs better than existing methods.

Copied to clipboard

Sanwoo Lee, Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai, Yunfang Wu

Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text (2026.acl-long)

Challenge:	Existing merging approaches involve scaling the parameters model-wise or integrating parameter importance parameter-wise.
Approach:	They propose a method for merging model-based models at the parameter level without training data or joint training.
Outcome:	The proposed model merging framework outperforms baseline models on validation sets.

Copied to clipboard

Zhihao Xu, Rumei Li, Jiahuan Li, Rongxiang Weng, Jingang Wang, Xunliang Cai, Xiting Wang

Information Extraction & Text Mining

Challenge:	Large language models (LLMs) can be used to effectively utilize tools in multi-turn interactions, but acquiring diverse and realistic multi-step tool-use data remains a challenge.
Approach:	They propose a text-based data synthesis pipeline that generates multi-turn tool-use trajectories from text corpora using relevance filtering, workflow tool extraction, trajectory grounding, and complexity refinement.
Outcome:	The proposed model achieves 14.9% improvement on the BFCL V3 Multi-turn benchmark while significantly reducing inference latency and costs.

Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism (2024.findings-acl)

Copied to clipboard

Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai

Green & Sustainable NLP

Challenge:	Existing approaches to generate draft tokens in large language models are expensive and resource-intensive.
Approach:	They propose an approach to generate draft tokens using a segment of the LLM and a self-distillation method to enhance the quality of draft token.
Outcome:	The proposed approach generates draft tokens using a segment of the LLM and a self-distillation method to improve quality and speed up generation.

The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis (2026.acl-long)

Copied to clipboard

Zihao Wei, Liang Pang, Jiahao Liu, Wenjie Shi, Jingcheng Deng, Shicheng Xu, Zenghao Duan, Jingang Wang, Fei Sun, Huawei Shen, Xueqi Cheng

Reasoning

Challenge:	Explicit reasoning trajectories increase performance but often trigger overthinking . despite its importance, this study examines how each step of reasoning affects the final outcome .
Approach:	They propose a Reasoning Completion Point Detector that detects the RCP by monitoring rank dynamics of termination tokens.
Outcome:	The proposed method reduces token usage by up to 44% while preserving accuracy.

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data (2025.findings-acl)

Copied to clipboard

Xuemiao Zhang, Xu Liangyu, Feiyu Duan, Yongwei Zhou, Sirui Wang, Rongxiang Weng, Jingang Wang, Xunliang Cai

EAVE: Efficient Product Attribute Value Extraction via Lightweight Sparse-layer Interaction (2024.findings-emnlp)

Challenge:	Existing methods of uniformly sampling data throughout the pretraining process are suboptimal because they overlook the model's evolving data preferences.
Approach:	They propose a Perplexity Difference (PD) based Preference Curriculum learning framework which perceives and uses the data preferred by LLMs as their capabilities improve . they propose PDPC to complete the arrangement of the dataset offline and ensure continuous training without interruption.
Outcome:	The proposed framework surpasses baselines on 1.3B and 3B models and achieves an increased average accuracy of over 8.1% across MMLU and CMMLU.

Copied to clipboard

Li Yang, Qifan Wang, Jianfeng Chi, Jiahao Liu, Jingang Wang, Fuli Feng, Zenglin Xu, Yi Fang, Lifu Huang, Dongfang Liu

PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models (2023.findings-acl)

Challenge:	Existing methods to extract product attribute value require multiple extractions to obtain all corresponding values.
Approach:	They propose an Efficient product Attribute Value Extraction approach using lightweight sparse-layer interaction.
Outcome:	The proposed method achieves significant efficiency gains with neutral or marginal loss in performance when the context is long and number of attributes is large.

Copied to clipboard

Zhuocheng Gong, Jiahao Liu, Qifan Wang, Yang Yang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Rui Yan

Scaling Laws Across Model Architectures: A Comparative Analysis of Dense and MoE Models in Large Language Models (2024.emnlp-main)

Challenge:	Quantization is a viable solution for pre-trained language models, but most existing methods are task-specific and require customized training and quantization with a large number of trainable parameters.
Approach:	They propose a "quantize before fine-tuning" framework that allows for quantization with a large number of trainable parameters on each individual task.
Outcome:	The proposed framework is compatible with quantization-aware training and post-training quantization and corrects quantization errors.

Copied to clipboard

Siqi Wang, Zhengyu Chen, Bei Li, Keqing He, Min Zhang, Jingang Wang

SampleMix: A Sample-wise Pre-training Data Mixing Strategy by Coordinating Data Quality and Diversity (2025.findings-emnlp)

Challenge:	a study of large language models (LLMs) reveals the transferability and discrepancies of scaling laws between Dense and MoE models.
Approach:	They investigate the transferability and discrepancies of scaling laws between Dense Models and Mixture of Experts models.
Outcome:	The results show that the power-law scaling framework also applies to MoE Models .

Copied to clipboard

Xiangyu Xi, Deyang Kong, Jian Yang, Jiawei Yang, Zhengyu Chen, Wei Wang, Jingang Wang, Xunliang Cai, Shikun Zhang, Wei Ye

BaseCal: Unsupervised Confidence Calibration via Base Model Signals (2026.acl-long)

Challenge:	Existing methods for pretraining data mixing for large language models neglect significant inter-domain overlaps and commonalities, failing to control the global diversity of the constructed training dataset.
Approach:	They propose a sample-wise data mixture approach that performs global cross-domain sampling by systematically evaluating the quality and diversity of each sample.
Outcome:	The proposed method exceeds existing domain-based methods in multiple downstream tasks and perplexity assessments.

Copied to clipboard

Hexiang Tan, Wanli Yang, Junwei Zhang, Xin Chen, Rui Tang, Du Su, Jingang Wang, Yuanzhuo Wang, Fei Sun, Xueqi Cheng

Fusion or Defusion? Flexible Vision-and-Language Pre-Training (2023.findings-acl)

Challenge:	Post-trained LLMs typically compromise reliability with severe overconfidence, resulting in inaccurate responses.
Approach:	They propose a solution that feeds PoLLMs into the base LLM to get confidence.
Outcome:	The proposed solution reduces expected calibration error (ECE) by 42.90% compared to the best unsupervised baselines.

Copied to clipboard

Rongyi Sun, Ziran Li, Yifeng Ding, Qifan Wang, Jingang Wang, Haitao Zheng, Wei Wu, Yunsen Xian

Making Pretrained Language Models Good Long-tailed Learners (2022.emnlp-main)

Challenge:	Existing approaches to vision-and-language pretraining (VLP) lack effectiveness and efficiency in downstream multimodal tasks.
Approach:	They propose a flexible vision-and-language pre-training model by incorporating cross-modal fusions into a dual-encoder architecture and a cross-module knowledge transfer strategy to guide the training process.
Outcome:	The proposed model is well-equipped with effectiveness and efficiency compared with other strong VLP models.

Copied to clipboard

Chen Zhang, Lei Ren, Jingang Wang, Wei Wu, Dawei Song