Papers by Dawei Song

What Does Your Smile Mean? Jointly Detecting Multi-Modal Sarcasm and Sentiment Using Quantum Probability (2021.findings-emnlp)

Challenge:	Existing prompt tuning approaches for attribute-controllable text generation are difficult to implement due to the lack of interpretability of deep neural networks.
Approach:	They propose a new approach that incorporates attribute knowledge of discriminator to optimize prompt tuning by steering a frozen CLM to produce attribute-specific texts.
Outcome:	The proposed approach can achieve state-of-the-art control performance while maintaining high-quality text generation.

Copied to clipboard

Yaochen Liu, Yazhou Zhang, Qiuchi Li, Benyou Wang, Dawei Song

Hierarchical Memory Organization for Wikipedia Generation (2025.acl-long)

Challenge:	Existing methods to model multi-modal sarcasm and sentiment are based on quantum probability . sarcasm and feelings embody intrinsic uncertainty of human cognition .
Approach:	They propose a quantum probability-driven multi-task learning framework for sarcasm and sentiment recognition using quantum superpositions and quantum interference.
Outcome:	The proposed model achieves state-of-the-art in multi-modal sarcasm and sentiment recognition.

Copied to clipboard

Eugene J. Yu, Dawei Zhu, Yifan Song, Xiangyu Wong, Jiebin Zhang, Wenxuan Shi, Xiaoguang Li, Qun Liu, Sujian Li

Challenge:	Existing methods for generating Wikipedia articles do not utilize memory directly for outline generation.
Approach:	They propose a method to generate Wikipedia articles autonomously by leveraging a hierarchical memory architecture.
Outcome:	The proposed framework outperforms baseline methods in producing informative and reliable articles.

UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation (2024.acl-long)

Copied to clipboard

Xun Liang, Shichao Song, Simin Niu, Zhiyu Li, Feiyu Xiong, Bo Tang, Yezhaohui Wang, Dawei He, Cheng Peng, Zhonghao Wang, Haiying Deng

AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories (2024.findings-emnlp)

Challenge:	Large language models (LLMs) produce hallucinated text, compromising their practical utility in professional contexts.
Approach:	They have developed an unconstrained hallucination generation evaluation benchmark that contains hallucines generated by large language models with minimal restrictions.
Outcome:	The proposed benchmarks are based on a Chinese-language dataset that is lacking in the field.

Copied to clipboard

Yifan Song, Weimin Xiong, Xiutian Zhao, Dawei Zhu, Wenhao Wu, Ke Wang, Cheng Li, Wei Peng, Sujian Li

Beyond Polarity: Continuous Affect-Enhanced Multimodal Aspect-Based Sentiment Classification (2026.findings-acl)

Challenge:	Existing studies focus on specialized agents designed for particular tasks.
Approach:	They propose to scale annotated interaction trajectories and fine-tune LLMs on AgentBank to get a series of agent models, Samoyed.
Outcome:	The proposed model can scale to get generalized agent capabilities.

Copied to clipboard

Ling-Ang Meng, Tianyu Zhao, Dawei Song, Jingxu Cao, Youhui Zuo

Information Retrieval

Challenge:	Existing methods for multimodal aspect-based sentiment classification exploit discrete polarity patterns and generic visual embeddings.
Approach:	They propose a Valence–Arousal–Dominance(VAD)-Enhanced MABSC framework that integrates VAD signals into multimodal sentiment reasoning and learns emotion-sensitive image representations.
Outcome:	The proposed framework brings VAD signals into multimodal sentiment reasoning and learns emotion-sensitive image representations.

Towards the Law of Capacity Gap in Distilling Language Models (2025.acl-long)

Copied to clipboard

Chen Zhang, Qiuchi Li, Dawei Song, Zheyu Ye, Yan Gao, Yao Hu

Reward Alignment Optimization: A Direct Point-wise Alignment Approach (2026.acl-long)

Challenge:	Language model (LM) distillation aims at distilling knowledge in a large teacher LM to a small student one.
Approach:	They propose to use the law of capacity gap to distill knowledge from a large teacher to a small student model.
Outcome:	The proposed model outperforms other language models on a larger scale by using the law of capacity gap inducted from a preliminary study on small-scale (3B) LMs.

Copied to clipboard

Zelin Li, Jia Leng, Dawei Song, Yangen Hu

Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge (2025.emnlp-main)

Challenge:	Existing Direct Alignment Algorithms (DAAs) are limiting in generalizaiton to implicit rewards.
Approach:	They propose a point-wise direct alignment method that uses an explicit reward model to specify exact target generation probabilities and align the policy offline towards them.
Outcome:	The proposed method outperforms existing direct alignment algorithms while enabling controllable target probability distributions.

Copied to clipboard

Yi Sui, Chaozhuo Li, Chen Zhang, Dawei Song, Qiuchi Li

WIKIGENBENCH:Exploring Full-length Wikipedia Generation under Real-World Scenario (2025.coling-main)

Challenge:	Retrieval-augmented generation (RAG) aims to mitigate the hallucination of Large Language Models (LLMs) however, external knowledge may contain noise and conflict with parametric knowledge of LLMs, leading to degraded performance.
Approach:	They propose a Dual-Stream Knowledge-Augmented Framework for Shared-Private Semantic Synergy that refines the traditional self-attention into a mixed-attention that distinguishes shared and private semantics for a controlled knowledge integration.
Outcome:	Extensive experiments show that the proposed framework achieves a superior performance over baselines.

Copied to clipboard

Jiebin Zhang, Eugene J. Yu, Qinyu Chen, Chenhao Xiong, Dawei Zhu, Han Qian, Mingbo Song, Weimin Xiong, Xiaoguang Li, Qun Liu, Sujian Li

Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning (2026.findings-acl)

Challenge:	Existing efforts to generate Wikipedia articles for new events fall short of real-world application.
Approach:	They propose a benchmark to generate Wikipedia articles for new events under real-world scenarios . they use systematic metrics and LLM-based metrics to assess verifiability, organization, and other aspects aligned with real-life scenarios.
Outcome:	The proposed benchmarks show that hierarchical-based methods generate more comprehensive content while fine-tuned methods achieve better verifiability.

Copied to clipboard

Yi Sui, Chaozhuo Li, Dawei Song

How Speculative Can Speculative Decoding Be? (2024.lrec-main)

Challenge:	Existing CoT compression methods struggle to balance accuracy and efficiency . long CoT reasoning also introduces an overthinking phenomenon, authors say .
Approach:	They propose a framework that performs step-wise CoT compression by modeling stage-specific redundancy sources and integrating with a retrieval-augmented guidance.
Outcome:	The proposed framework reduces average response length by 59.9% while improving accuracy by 4.8 points over existing methods.

Copied to clipboard

Zhuorui Liu, Chen Zhang, Dawei Song

How does Attention Affect the Model? (2021.findings-acl)

Challenge:	Large language models (LLMs) have a largely increased latency due to their ability to autoregressively model . speculative decoding is a technique that trades generation quality for speed .
Approach:	They propose to use a draft model to draft tokens autoregressively and then verify them in parallel.
Outcome:	The proposed model could draft tokens autoregressively and then verify them in parallel . the proposed model trades quality for speed and could fail in verification stage .

Copied to clipboard

Cheng Zhang, Qiuchi Li, Lingyu Hua, Dawei Song

Explainability & Interpretability in NLP

Challenge:	Existing studies on the effectiveness of attention in NLP do not consider changes in semantic capability of different components.
Approach:	They propose a framework that exploits a convex hull representation of sequence semantics in an n-dimensional Semantic Euclidean Space and defines indicators to capture the impact of attention on sequence semantic.
Outcome:	The proposed framework exploits a convex hull representation of sequence semantics in an n-dimensional Semantic Euclidean Space and defines indicators to capture the impact of attention on sequence semantic.

Structural Bias for Aspect Sentiment Triplet Extraction (2022.coling-1)

Copied to clipboard

Chen Zhang, Lei Ren, Fang Ma, Jingang Wang, Wei Wu, Dawei Song

More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression (2025.findings-emnlp)

Challenge:	Existing structural bias adapters for aspect sentiment triplet extraction are under-confident . a large-scale dataset for ASTE shows the adapter is effective and efficient to a larger scale.
Approach:	They propose to use a structural adapter to integrate structural bias into pretrained language models . they propose to add a relative position structure in place of the syntactic dependency structure .
Outcome:	The proposed adapter achieves state-of-the-art performance over strong baselines, but with a light parameter demand and low latency.

Copied to clipboard

Jiebin Zhang, Dawei Zhu, Yifan Song, Wenhao Wu, Chuqiao Kuang, Xiaoguang Li, Lifeng Shang, Qun Liu, Sujian Li

Challenge:	storing more tokens in the KV cache at lower precision can enhance the long-context performance of large language models.
Approach:	They propose a token-precision trade-off strategy to optimize KV cache compression . they also propose storing more tokens in the KV at lower precision .
Outcome:	The proposed method achieves an optimal point within the Information Bottleneck compared to standalone KV pruning or KV quantization.

InfoCL: Alleviating Catastrophic Forgetting in Continual Text Classification from An Information Theoretic Perspective (2023.findings-emnlp)

Copied to clipboard

Yifan Song, Peiyi Wang, Weimin Xiong, Dawei Zhu, Tianyu Liu, Zhifang Sui, Sujian Li

Information Extraction & Text Mining

Challenge:	Recent studies have identified the severe performance decrease on analogous classes as a key factor for catastrophic forgetting.
Approach:	They propose a replay-based continual text classification method that uses fast-slow and current-past contrastive learning to perform mutual information maximization and better recover previously learned representations.
Outcome:	The proposed method achieves state-of-the-art on three text classification tasks.

Sparse Teachers Can Be Dense with Knowledge (2022.emnlp-main)

Copied to clipboard

Yi Yang, Chen Zhang, Dawei Song

Green & Sustainable NLP

Challenge:	Existing methods for transferring knowledge from a teacher of large scale to a student of smaller scale are limiting in overall knowledgeableness.
Approach:	They propose a sparse teacher trick to remove over-parameterized teachers that produce student-unfriendly knowledge and thus limit overall knowledgeableness.
Outcome:	The proposed trick removes the parameters that result in student-unfriendliness and leads to compelling performance in comparison with baselines.

Controllable Text Generation with Residual Memory Transformer (2024.findings-acl)

Copied to clipboard

Hanqing Zhang, Si Sun, Haiming Wu, Dawei Song

Adaptive Parameterization for Neural Dialogue Generation (D19-1)

Challenge:	Large-scale Causal Language Models (CLMs) have been successful in text generation, but there is still a challenge to control the generation process.
Approach:	They propose a non-intrusive, lightweight control plugin to control the generation process of a CLM at arbitrary time steps.
Outcome:	The proposed plugin can handle any type of control conditions and cooperate with the base CLM through a residual learning paradigm.

Copied to clipboard

Hengyi Cai, Hongshen Chen, Cheng Zhang, Yonghao Song, Xiaofang Zhao, Dawei Yin

Dialogue Systems & Conversational Agents

Challenge:	Existing models of open-domain dialogue generate responses based on sequence-to-sequence paradigms.
Approach:	They propose an Adaptive Neural Dialogue generation model which manages various conversations with conversation-specific parameterization.
Outcome:	The proposed model performs better on a large-scale conversational dataset.

Task-agnostic Distillation of Encoder-Decoder Language Models (2024.lrec-main)

Copied to clipboard

Chen Zhang, Yang Yang, Qiuchi Li, Jingang Wang, Dawei Song

MoDification: Mixture of Depths Made Easy (2025.naacl-long)

Challenge:	Existing distillation methods that focus on encoder-only LMs fail to handle the distillation of encoder decoder LM.
Approach:	They propose a method that finetunes pretrained language models (LMs) they propose 'MiniEnD' that allows for task-agnostic distillation of LMs.
Outcome:	The proposed distillation method is generally effective and competitive compared to other alternatives.

Copied to clipboard

Chen Zhang, Meizhi Zhong, Qimeng Wang, Xuantao Lu, Zheyu Ye, Chengqiang Lu, Yan Gao, Yao Hu, Kehai Chen, Min Zhang, Dawei Song

Green & Sustainable NLP

Challenge:	Long-context efficiency is a trending topic in large language model (LLM) serving.
Approach:	They propose a method to combine long-context efficiency and mixture of depths to bring down both latency and memory.
Outcome:	The proposed method achieves 1.2 speedup in latency and 1.8 reduction in memory compared to original LLMs especially in long-context applications.

Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluation (2026.acl-long)

Copied to clipboard

Yanzhi Tian, Cunxiang Wang, Zeming Liu, Heyan Huang, Wenbo Yu, Dawei Song, Jie Tang, Yuhang Guo

Machine Translation

Challenge:	Large Language Models (LLMs) have advanced machine translation (MT) a meta-evaluation dataset focused on non-literal translations is lacking . experimental results show the inaccuracies of traditional MT metrics and the limitations of LLM-as-a-Judge.
Approach:	They propose a meta-evaluation framework that leverages sub-agents to evaluate machine translation metrics.
Outcome:	The proposed framework improves on the knowledge cutoff and score inconsistency problem.

XPrompt: Exploring the Extreme of Prompt Tuning (2022.emnlp-main)

Copied to clipboard

Fang Ma, Chen Zhang, Lei Ren, Jingang Wang, Qifan Wang, Wei Wu, Xiaojun Quan, Dawei Song

ConFiguRe: Exploring Discourse-level Chinese Figures of Speech (2022.coling-1)

Challenge:	Prompt tuning learns soft prompts to condition pre-trained Language Models for performing downstream tasks in a parameter-efficient manner.
Approach:	They propose a Prompt tuning model with an eXtremely small scale that learns soft prompts to condition the frozen Pre-trained Language Models for performing downstream tasks in a parameter-efficient manner.
Outcome:	The proposed model outperforms the vanilla Prompt-Tuning and can significantly improve across tasks and model scales.

Copied to clipboard

Dawei Zhu, Qiusi Zhan, Zhejian Zhou, Yifan Song, Jiebin Zhang, Sujian Li

Speech & Audio in NLP

Challenge:	Figures of speech often deviate from their literal meanings to express deeper semantic implications.
Approach:	They propose a concept of figurative unit, which is the carrier of a figure, and build a Chinese corpus for Contextualized Figure Recognition.
Outcome:	The proposed model is based on 12 types of figures commonly used in Chinese . it shows that the proposed tasks are challenging for existing models .

A Multi-task Learning Framework for Opinion Triplet Extraction (2020.findings-emnlp)

Copied to clipboard

Chen Zhang, Qiuchi Li, Dawei Song, Benyou Wang

Minimal Distillation Schedule for Extreme Language Model Compression (2024.findings-eacl)

Challenge:	Existing approaches to Aspect-based sentiment analysis (ABSA) use aspect terms and their corresponding sentiment polarities as a reference, but they lack opinion terms as .
Approach:	They propose a multi-task learning framework to extract aspect terms and opinion terms and parse their sentiment dependencies with a biaffine scorer.
Outcome:	The proposed framework outperforms baseline and state-of-the-art approaches on four SemEval benchmarks.

Copied to clipboard

Chen Zhang, Yang Yang, Qifan Wang, Jiahao Liu, Jingang Wang, Wei Wu, Dawei Song

Lifting the Curse of Capacity Gap in Distilling Language Models (2023.acl-long)

Challenge:	Existing methods for teacher assistant-based distillation require multiple trials to find the optimal teacher assistant.
Approach:	They propose a method that allows scheduling of an optimal teacher assistant in just one trial . they show that student performance is positively correlated with the scale-performance tradeoff .
Outcome:	The proposed method can select the optimal teacher assistant in just one trial . it can be used to compare performance of student and teacher assistants on GLUE benchmarks.

Copied to clipboard

Chen Zhang, Yang Yang, Jiahao Liu, Jingang Wang, Yunsen Xian, Benyou Wang, Dawei Song

Beyond Emotion: A Multi-Modal Dataset for Human Desire Understanding (2022.naacl-main)

Challenge:	Existing studies have shown that pretrained language models require a tremendous amount of inference compute to perform.
Approach:	They propose to compress pretrained language models to small ones with a teacher-student paradigm to fill the capacity gap.
Outcome:	The proposed model achieves state-of-the-art performance at small FLOPs compared with competitive baselines.

Copied to clipboard

Ao Jia, Yu He, Yazhou Zhang, Sagar Uprety, Dawei Song, Christina Lioma

Multimodality

Challenge:	Desire is a primitive instinct and a need for strongly expressing human desires to get or possess something.
Approach:	They propose to use MSED to model and understand human desire . they propose to provide a benchmark for human desire analysis .
Outcome:	The proposed dataset contains 9,190 text-image pairs with English text.

Bi-DCSpell: A Bi-directional Detector-Corrector Interactive Framework for Chinese Spelling Check (2024.findings-emnlp)

Copied to clipboard

Haiming Wu, Hanqing Zhang, Richeng Xuan, Dawei Song

Text Error Correction

Challenge:	Chinese Spelling Check (CSC) aims to detect and correct potentially misspelled characters in Chinese sentences.
Approach:	They propose a bi-directional Detector-Corrector framework for Chinese Spelling Check which mutually enhances the feature representation for detection and correction subtasks.
Outcome:	The proposed framework reduces the risk of over-correction and under-corrections while preserving the knowledge learnt from correction.

LongEmbed: Extending Embedding Models for Long Context Retrieval (2024.emnlp-main)

Copied to clipboard

Dawei Zhu, Liang Wang, Nan Yang, Yifan Song, Wenhao Wu, Furu Wei, Sujian Li

Representation Learning

Challenge:	Existing embedding models support only 512 input tokens, hindering their application in scenarios requiring long inputs.
Approach:	They evaluate the performance of existing embedding models by using a new benchmark and a training-free context window extension strategy.
Outcome:	The proposed model extends the input window of existing models by several folds.

Large Language Models for Data Annotation and Synthesis: A Survey (2024.emnlp-main)

Copied to clipboard

Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, Huan Liu

CoUDA: Coherence Evaluation via Unified Data Augmentation (2024.naacl-long)

Challenge:	Existing surveys focus on LLMs' specific utility for data annotation and synthesis.
Approach:	They propose to use large language models to generate annotations from raw data . they also propose to review learning strategies for models utilizing LLM-generated annotations .
Outcome:	The proposed models can be used to improve the efficacy of machine learning models by generating and labeling raw data with relevant information.

Copied to clipboard

Dawei Zhu, Wenhao Wu, Yifan Song, Fangwei Zhu, Ziqiang Cao, Sujian Li

Making Pretrained Language Models Good Long-tailed Learners (2022.emnlp-main)

Challenge:	Existing data augmentations for coherence evaluation rely on heuristic rules and lack designing criteria.
Approach:	They propose a data augmentation framework that breaks down coherence into global and local aspects and designs augmentation strategies for both aspects.
Outcome:	The proposed framework surpasses recent models in scoring and ranking tasks with 233M parameters.

Copied to clipboard

Chen Zhang, Lei Ren, Jingang Wang, Wei Wu, Dawei Song

Aspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks (D19-1)

Challenge:	Prompt-tuning has shown appealing performance in few-shot classification . however, it is less promising in long-tailed classification due to long tail .
Approach:	They propose to use prompt-tuning to make pretrained language models at least good long-tailed learners by bridging the gap between prompt- and commonly used finetun.
Outcome:	The proposed method makes pretrained language models at least good long-tailed learners, bridging the gap between prompt-tuning and finetunation.

Copied to clipboard

Chen Zhang, Qiuchi Li, Dawei Song

Exploiting Position Bias for Robust Aspect Sentiment Classification (2021.findings-acl)

Challenge:	Existing aspects-based sentiment classification models lack a mechanism to account for relevant syntactical constraints and word dependencies.
Approach:	They propose to build a Graph Convolutional Network over the dependency tree of a sentence to exploit syntactical information and word dependencies.
Outcome:	The proposed model is comparable to state-of-the-art models on three benchmarking collections.

Copied to clipboard

Fang Ma, Chen Zhang, Dawei Song

Expectation Preference Optimization: Reliable Preference Estimation for Improving the Reasoning Capability of Large Language Models (2025.emnlp-main)

Challenge:	Aspect sentiment classification models suffer from the issue of robustness when domains of test and training data are different or test data is adversarially perturbed.
Approach:	They propose two mechanisms for capturing position bias to reduce the probability of mis-attending . they propose position-biased weight and position-based dropout to enhance existing models .
Outcome:	The proposed approaches improve the robustness and effectiveness of existing models.

Copied to clipboard

Zelin Li, Dawei Song