Papers by Minlie Huang
AGD: Adversarial Game Defense Against Jailbreak Attacks in Large Language Models (2025.acl-long)
Copied to clipboard
Shilong Pan, Zhiliang Tian, Zhen Huang, Wanlong Yu, Zhihua Wen, Xinwang Liu, Kai Lu, Minlie Huang, Dongsheng Li
| Challenge: | Existing defenses, including post-training alignment and prompt engineering, struggle with adaptability to out-of-distribution (OOD) attacks. |
| Approach: | They propose an adversarial game-based defense method that dynamically adjusts LLMs’ internal representations to achieve a balanced trade-off between helpfulness and harmlessness. |
| Outcome: | The proposed method improves LLMs’ safety over all baselines. |
Goal Awareness for Conversational AI: Proactivity, Non-collaborativity, and Beyond (2023.acl-tutorials)
Copied to clipboard
| Challenge: | Conventional conversation researches focus on the responseability of the system, such as dialogue context understanding and response generation, but overlook the design of an essential property in intelligent conversations, i.e., goal awareness. |
| Approach: | This tutorial introduces the latest advances on the design of agent’s awareness of goals in a wide range of conversational systems. |
| Outcome: | This tutorial introduces the latest advances on the design of agent’s awareness of goals in a wide range of conversational systems. |
Learning Task Decomposition to Assist Humans in Competitive Programming (2024.acl-long)
Copied to clipboard
| Challenge: | Using language models (LMs) to solve complex problems, humans might struggle to understand and repair flawed ones. |
| Approach: | They propose to automatically decompose complex problems into simpler pieces that correspond to specific subtasks and measure their assistive value. |
| Outcome: | The proposed method enables non-experts to solve 33.3% more problems and speeds them up by 3.3x . |
Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog (D19-1)
Copied to clipboard
| Challenge: | Existing methods to learn dialog policy require elaborate design and user goals. |
| Approach: | They propose an algorithm that estimates the reward signal and infers the user goal in dialog sessions. |
| Outcome: | The proposed algorithm achieves higher task success than state-of-the-art models on a multi-domain task-oriented dialog dataset. |
Thoughts to Target: Enhance Planning for Target-driven Conversation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results demonstrate that our method significantly improves the planning ability of LLMs, especially in target-driven conversations. |
| Approach: | They propose a two-stage framework to improve the LLMs’ capability in planning conversations towards designated targets by distilling natural language plans from a target-driven conversation corpus and generating new plans with demonstration-guided in-context learning. |
| Outcome: | The proposed framework improves the ability of conversational models to plan towards designated targets and can be used to build extensive conversational AI. |
LOT: A Story-Centric Benchmark for Evaluating Chinese Long Text Understanding and Generation (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing benchmarks for natural language processing focus on understanding or generating short texts . lack of standardized benchmarks makes it difficult to assess and compare models . |
| Approach: | They propose a story-centric benchmark for Chinese long text modeling that aggregates two understanding tasks and two generation tasks. |
| Outcome: | The proposed model outperforms similar-sized models on understanding and generation tasks. |
E-NER: Evidential Deep Learning for Trustworthy Named Entity Recognition (2023.findings-acl)
Copied to clipboard
Zhen Zhang, Mengting Hu, Shiwan Zhao, Minlie Huang, Haotian Wang, Lemao Liu, Zhirui Zhang, Zhe Liu, Bingzhe Wu
| Challenge: | Named entity recognition (NER) systems focus on improving model performance, ignoring the need to quantify model uncertainty. |
| Approach: | They propose to introduce two uncertainty-guided loss terms to the conventional EDL and a series of uncertainty-guiding training strategies to solve these challenges. |
| Outcome: | The proposed method achieves better OOV/OOD detection performance and generalization ability on OOV entities compared to state-of-the-art methods. |
Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that instruction tuning is effective in instruction learning for unseen tasks, but it relies on a large amount of human-annotated samples, which restricts its generalization. |
| Approach: | They propose an instruction tuning technique which fine-tunes a pre-trained language model on a massive collection of tasks described via human-craft instructions and then tests its generalization ability on unseen tasks. |
| Outcome: | The proposed method improves IT performance versus labeled data and training tasks by constructing pseudo-labeled data from unlabele . data is used to build a model that can learn from human instructions for zero-shot generalization on unseen tasks. |
Automatic Comment Generation for Chinese Student Narrative Essays (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Existing studies focus on giving discrete scores for holistic quality or distinct traits, but real-world teachers usually provide detailed comments in natural language, which are more informative than single scores. |
| Approach: | They propose a model which generates comments for specified segments from given student narrative essays using a human-written Chinese dataset. |
| Outcome: | The proposed model outperforms baselines and has 91% success rate. |
The Side Effects of Being Smart: Safety Risks in MLLMs’ Multi-Image Reasoning (2026.acl-long)
Copied to clipboard
Renmiao Chen, Yida Lu, Shiyao Cui, Xuan Ouyang, Victor Shea-Jay Huang, Shumin Zhang, Chengwei Pan, Han Qiu, Minlie Huang
| Challenge: | Recent advances in multimodal reasoning may pose new safety risks . evaluators neglect reasoningbased safety, where harm emerges only through MLLMs . |
| Approach: | They introduce a benchmark for multi-image reasoning safety that includes 2,676 instances . they find that models with more advanced multi- image reasoning are more vulnerable . |
| Outcome: | The proposed benchmark consists of 2,676 instances covering 9 multi-image relations . the results show that models with more advanced multi- image reasoning are more vulnerable . |
DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing safety mechanisms for Large Language Models (LLMs) are inadequate to protect against jailbreak attacks, resulting in performance degradation on general tasks. |
| Approach: | They propose a method that directly updates a minimal set of relevant parameters to neutralize harmful behaviors while preserving the model’s utility. |
| Outcome: | The proposed model outperforms baseline methods in mitigating jailbreak attacks while preserving the model’s utility. |
Youling: an AI-assisted Lyrics Creation System (2020.emnlp-demos)
Copied to clipboard
Rongsheng Zhang, Xiaoxi Mao, Le Li, Lin Jiang, Lin Chen, Zhiwei Hu, Yadong Xi, Changjie Fan, Minlie Huang
| Challenge: | Recent studies have focused on a single pass of lyrics generation with little human intervention. |
| Approach: | They propose an AI-assisted lyrics creation system that supports one pass full-text generation and interactive generation modes. |
| Outcome: | The proposed system supports full-text generation and interactive generation modes . it also provides a revision module which enables users to revise undesired lyrics repeatedly. |
Advancing Collaborative Debates with Role Differentiation through Multi-Agent Reinforcement Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Multi-agent collaborative tasks exhibit exceptional capabilities in natural language applications and generation. |
| Approach: | They propose a multi-LLM Cooperation framework with automatic role assignment capabilities that allows multiple agents to embed roles in turn-based speaking. |
| Outcome: | The proposed framework improves collaboration and expertise among agents and teams by enabling them to share roles and develop complementary strengths from the optimization level. |
A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Weakly supervised question answering usually has only final answers as supervision signals while correct solutions are not provided. |
| Approach: | They propose to explicitly exploit the semantic correlations between question-answer pairs and predicted answers by maximizing mutual information between question and answer pairs. |
| Outcome: | The proposed method significantly outperforms previous learning methods in terms of task performance and is more effective in training models to produce correct solutions. |
ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors (2024.findings-emnlp)
Copied to clipboard
Zhexin Zhang, Yida Lu, Jingyuan Ma, Di Zhang, Rui Li, Pei Ke, Hao Sun, Lei Sha, Zhifang Sui, Hongning Wang, Minlie Huang
| Challenge: | Existing tools for detecting safety issues in LLMs are expensive and inefficient. |
| Approach: | They propose an LLM-based safety detector which annotates the safety of queries and provides explanations for its decisions. |
| Outcome: | The proposed detector outperforms baselines on four sets of query-response pairs and is effective as a safety evaluator for advanced LLMs. |
HyKnow: End-to-End Task-Oriented Dialog Modeling with Hybrid Knowledge Management (2021.findings-acl)
Copied to clipboard
| Challenge: | Task-oriented dialog systems typically manage structured knowledge to guide goal-oriented conversations. |
| Approach: | They propose a TOD system with hybrid knowledge management, HyKnow, which extends the belief state to manage both structured and unstructured knowledge. |
| Outcome: | The proposed model outperforms existing TOD systems in the evaluation of a multiWOZ dataset on unstructured knowledge with strong end-to-end performance. |
Data Efficient RLVR via Off-Policy Influence Guidance (2026.acl-long)
Copied to clipboard
Erle Zhu, Dazhi Jiang, Yuan Wang, Xujun Li, Jiale Cheng, Yuxian Gu, Yilin Niu, Aohan Zeng, Jie Tang, Minlie Huang, Hongning Wang
| Challenge: | Existing data selection methods for RLVR are heuristic-based, lacking theoretical guarantees and generalizability. |
| Approach: | They propose an off-policy influence estimation method that approximates data influence using offline trajectories. |
| Outcome: | The proposed method reduces the computational cost of policy rollouts and improves storage and computation efficiency. |
InstructSafety: A Unified Framework for Building Multidimensional and Explainable Safety Detector through Instruction Tuning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing safety detection systems have limitations in terms of their versatility and interpretability. |
| Approach: | They introduce a safety detection framework that unifies 7 common sub-tasks into a uniform formulation and process 39 human-annotated datasets for instruction tuning. |
| Outcome: | The proposed framework unifies 7 common sub-tasks into a uniform formulation and then runs on 39 human-annotated datasets to fine-tune it. |
Chaining Simultaneous Thoughts for Numerical Reasoning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Numerical reasoning over text is an essential skill for AI systems . structure modeling is effective, but structures restrict how a model should grasp the reasoning process . |
| Approach: | They propose a numerical reasoner that models reasoning steps using a directed acyclic graph without pre-defined decoding dependencies. |
| Outcome: | The proposed model produces diverse reasoning steps without pre-defined dependencies and compares relevant ones to reach a solution. |
Generating Informative Responses with Controlled Sentence Function (P18-1)
Copied to clipboard
| Challenge: | Sentence function is a significant factor to achieve the purpose of the speaker, but has not been touched in large-scale conversation generation. |
| Approach: | They propose a model to generate informative responses with controlled sentence function using a latent variable and a type controller to deal with compatibility. |
| Outcome: | The proposed model outperforms state-of-the-art models and generates responses with controlled sentence function and informative content. |
Adversary-Aware DPO: Enhancing Safety Alignment in Vision Language Models via Adversarial Training (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Adversary-aware DPO (ADPO) is a training framework that explicitly considers adversary. |
| Approach: | a new framework integrates adversarial training into a pre-trained large language model to enhance safety alignment . adversary-aware DPO provides a framework that explicitly considers adversary . |
| Outcome: | a new training framework outperforms baselines in safety alignment and general utility of large language models. |
“I’ve Decided to Leak”: Probing Internals Behind Prompt Leakage Intents (2025.emnlp-main)
Copied to clipboard
Jianshuo Dong, Yutong Zhang, Liu Yan, Zhenyu Zhong, Tao Wei, Ke Xu, Minlie Huang, Chao Zhang, Han Qiu
| Challenge: | Large language models (LLMs) exhibit prompt leakage vulnerabilities, raising intellectual property and confidentiality concerns. |
| Approach: | They use probing techniques to capture LLMs’ intent-related internal representations and show that they internalize prompt leakage intents in their hidden states before generating tokens. |
| Outcome: | The proposed probes achieve 90%+ AUROC across all tested models, even when applied to new system prompts and attacks. |
AutoDetect: Towards a Unified Framework for Automated Weakness Detection in Large Language Models (2024.findings-emnlp)
Copied to clipboard
Jiale Cheng, Yida Lu, Xiaotao Gu, Pei Ke, Xiao Liu, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang
| Challenge: | Large Language Models (LLMs) exhibit significant but subtle weaknesses, such as mistakes in instruction-following or coding tasks. |
| Approach: | They propose a framework to automatically expose weaknesses in Large Language Models (LLMs) they use three LLM-powered agents to perform comprehensive weakness identification . |
| Outcome: | The proposed framework shows that it is more effective than untargeted data augmentation methods like Self-Instruct to identify weaknesses in LLMs. |
Intent-Aware and Hate-Mitigating Counterspeech Generation via Dual-Discriminator Guided LLMs (2024.lrec-main)
Copied to clipboard
| Challenge: | Hate speech is an aggressive expression that incites hatred towards specific groups based on their group identity. |
| Approach: | They propose an LLMs-based framework for counterspeech generation that uses intent-aware discriminators to decode intents of LLM models. |
| Outcome: | The proposed framework matches intents with hate mitigation intents and performs well. |
A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for machine reading comprehension lack evidence labels for training models. |
| Approach: | They propose a method which supervises the evidence extractor with auto-generated evidence labels in an iterative process. |
| Outcome: | The proposed method improves on three MRC tasks on seven datasets. |
Reframe Your Life Story: Interactive Narrative Therapist and Innovative Moment Assessment with Large Language Models (2025.emnlp-main)
Copied to clipboard
Yi Feng, Jiaqi Wang, Wenxuan Zhang, Zhuang Chen, Shen Yutong, Xiyao Xiao, Minlie Huang, Liping Jing, Jian Yu
| Challenge: | Existing approaches to mental health support lack realism and capture therapeutic progression over time. |
| Approach: | They propose a framework that simulates expert narrative therapists by planning therapeutic stages, guiding reflection levels, and generating contextually appropriate responses through retrieval-augmentation. |
| Outcome: | The proposed framework outperforms standard methods in quality and depth on 260 simulated clients and 230 human participants. |
Mixture-of-Modules: Reinventing Transformers as Dynamic Assemblies of Modules (2024.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results show that MoMs consistently outperform vanilla transformers . |
| Approach: | They propose an architecture that allows for a mixture-of-modules computation that uses a finite set of modules defined by multi-head attention and feed-forward networks. |
| Outcome: | The proposed architecture outperforms vanilla Transformers and their variants in multiple ways. |
CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing reference-free metrics have obvious limitations for evaluating controlled text generation models. |
| Approach: | They propose an unsupervised reference-free metric which evaluates controlled text generation from different aspects by formulating each aspect into multiple text infilling tasks. |
| Outcome: | The proposed metric has higher correlations with human judgments while obtaining better generalization of evaluating generated texts from different models and with different qualities. |
EARL: Informative Knowledge-Grounded Conversation Generation with Entity-Agnostic Representation Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge-grounded conversation models lack knowledge that occurs in training data, resulting in incomplete knowledge generation. |
| Approach: | They propose an Entity-Agnostic Representation Learning method to introduce knowledge graphs to informative conversation generation using context of conversations and relational structure of knowledge graph. |
| Outcome: | The proposed model generates more informative, coherent, and natural responses than baseline models. |
Uncertainty-Aware Unlikelihood Learning Improves Generative Aspect Sentiment Quad Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on what to generate but ignore what not to generate . a template-agnostic method boosts original learning and reduces mistakes simultaneously . |
| Approach: | They propose a template-agnostic method to control the token-level generation . they introduce Monte Carlo dropout to understand the built-in uncertainty of pre-trained language models . |
| Outcome: | The proposed method boosts original learning and reduces mistakes simultaneously on four public datasets. |
LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety (2026.acl-long)
Copied to clipboard
Junxiao Yang, Haoran Liu, Jinzhe Tu, Jiale Cheng, Zhexin Zhang, Shiyao Cui, Jiaqi Weng, Jialing Tao, Hui Xue, Hongning Wang, Han Qiu, Minlie Huang
| Challenge: | Large language models (LLMs) have demonstrated better safety performance in high-resource languages than in low-resourced languages. |
| Approach: | They propose language-agnostic semantic alignment (LASA) which anchors safety alignment directly in semantic bottlenecks. |
| Outcome: | The proposed approach significantly improves safety across all languages: average attack success rate drops from 24.7% to 2.8% on LLaMA-3.1-8B-Instruct and remains within 3–4% across Qwen2.5 and Qwend3 Instruct models (7B–32B). |
MAGI: Multi-Agent Guided Interview for Psychiatric Assessment (2025.findings-acl)
Copied to clipboard
Guanqun Bi, Zhuang Chen, Zhoufu Liu, Hongkai Wang, Xiyao Xiao, Yuqiang Xie, Wen Zhang, Yongkang Huang, Yuxuan Chen, Libiao Peng, Minlie Huang
| Challenge: | Existing large language models (LLMs) do not align with psychiatric diagnostic protocols. |
| Approach: | They propose a framework that transforms the Mini International Neuropsychiatric Interview into automatic computational workflows through coordinated multi-agent collaboration. |
| Outcome: | The proposed framework transforms the gold-standard Mini International Neuropsychiatric Interview (MINI) into automatic computational workflows through coordinated multi-agent collaboration. |
Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systems (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models have shown promising results for few-shot learning in task-oriented dialog (ToD) systems. |
| Approach: | They propose a self-training approach that iteratively labels the most confident unlabeled data to train a stronger Student model. |
| Outcome: | The proposed approach improves state-of-the-art pre-trained models in few-shot learning scenarios for task-oriented dialog (ToD) systems when only a small number of labeled data are available. |
How Should We Enhance the Safety of Large Reasoning Models: An Empirical Study (2026.acl-long)
Copied to clipboard
Zhexin Zhang, Xian Qi Loye, Victor Shea-Jay Huang, Junxiao Yang, Qi Zhu, Shiyao Cui, Fei Mi, Lifeng Shang, Yingkang Wang, Hongning Wang, Minlie Huang
| Challenge: | Large Reasoning Models have achieved remarkable success on reasoning-intensive tasks, but their enhanced reasoning capabilities do not translate to improved safety performance. |
| Approach: | They propose to use supervised fine tuning to enhance the safety of Large Reasoning Models. |
| Outcome: | The proposed method improves the safety of large reasoning models on reasoning-intensive tasks. |
An Operation Network for Abstractive Sentence Compression (C18-1)
Copied to clipboard
| Challenge: | Sentence compression is a natural language generation task that condenses a sentence . Delete-based models remove unimportant words from the source sentence and generate a shorter sentence if the source is not a word deletion problem. |
| Approach: | They propose a neural network approach for abstractive sentence compression . they model the sentence compression process as an editing procedure . |
| Outcome: | The proposed approach outperforms state-of-the-art models in the abstractive sentence compression field. |
AlignBench: Benchmarking Chinese Alignment of Large Language Models (2024.acl-long)
Copied to clipboard
Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Andrew Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Xiaotao Gu, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, Jie Tang
| Challenge: | Effective evaluation of alignment for emerging Chinese LLMs is still significantly lacking, calling for real-scenario grounded, open-ended, challenging and automatic evaluations tailored for alignment. |
| Approach: | They propose a multi-dimensional benchmark for evaluating LLMs’ alignment in Chinese with 8 main categories, 683 real-scenario rooted queries and corresponding human verified references. |
| Outcome: | The benchmark uses a human-in-the-loop data curation pipeline, 683 real-scenario rooted queries and human verified references. |
A Survey of Post-Training Scaling in Large Language Models (2025.acl-long)
Copied to clipboard
Hanyu Lai, Xiao Liu, Junjie Gao, Jiale Cheng, Zehan Qi, Yifan Xu, Shuntian Yao, Dan Zhang, Jinhua Du, Zhenyu Hou, Xin Lv, Minlie Huang, Yuxiao Dong, Jie Tang
| Challenge: | Large language models (LLMs) have demonstrated proficiency in understanding and generating human natural languages. |
| Approach: | They propose a framework for scaling large language models using supervised fine-tuning, RLxF and test-time compute methodologies. |
| Outcome: | The proposed model can be used to understand and generate human natural languages. |
Dialogue Distillation: Open-Domain Dialogue Augmentation Using Unpaired Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing research has focused on training open-domain dialogue models using unpaired data. |
| Approach: | They propose a data-level distillation method for training open-domain dialogue models by utilizing unpaired data. |
| Outcome: | The proposed method produces high-quality dialogue pairs with diverse contents, and can improve competitive baselines. |
Guiding not Forcing: Enhancing the Transferability of Jailbreaking Attacks on LLMs via Removing Superfluous Constraints (2025.acl-long)
Copied to clipboard
| Challenge: | Jailbreaking attacks can effectively induce unsafe behaviors in Large Language Models (LLMs). |
| Approach: | They propose a conceptual framework to elucidate transferability of gradient-based jailbreaking methods . they identify superfluous constraints as significant barriers to improved transferability . |
| Outcome: | The proposed method increases the overall transfer attack success rate (T-ASR) across target and source models with varying safety levels from 18.4% to 50.3% while improving stability and controllability of jailbreak behaviors. |
Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to detect toxic generation of pretrained language models rely on templates, data extraction, crowdsourcing workers or automatic generation. |
| Approach: | They propose a method to construct adversarial contexts conditioned on a given response . they augment existing dataset BAD+ and construct a new dataset B AD+ . |
| Outcome: | The proposed method can detect toxic or biased content in large pretrained language models. |
DecompEval: Evaluating Generated Texts as Unsupervised Decomposed Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation metrics for natural language generation (NLG) tasks face the challenges on generalization ability and interpretability. |
| Approach: | They propose a metric that evaluates natural language generation tasks as an instruction-style question answering task and utilizes instruction-tuned pre-trained language models without training on evaluation datasets. |
| Outcome: | The proposed metric achieves state-of-the-art performance in untrained metrics for evaluating text summarization and dialogue generation, which exhibits strong dimension-level / task-level generalization ability and interpretability. |
UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing metrics for open-ended text generation are poorly correlated with human judgments . despite the success of existing metrics, there are few plausible outputs for the same input . |
| Approach: | They propose a UNreferenced measure for evaluating open-ended story generation . it is built on top of BERT and is trained to distinguish human-written stories from negative samples . |
| Outcome: | The proposed measure is more generalizable than state-of-the-art metrics and correlates better with human judgments. |
Answering Open-Domain Multi-Answer Questions via a Recall-then-Verify Framework (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to open-domain question answering use a rerank-then-read framework . existing approaches use reranked evidence to predict multiple valid answers . |
| Approach: | They propose to use a recall-then-verify framework to solve open-domain questions . the framework separates the reasoning process of each answer to make better use of retrieved evidence . |
| Outcome: | The proposed framework predicts significantly more gold answers on open-domain questions than existing systems that use an oracle reranker. |
Building Multi-domain Dialog State Trackers from Single-domain Dialogs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing multi-domain dialog state tracking models require significant manual effort to define domain relations and collect data. |
| Approach: | They propose a divide-and-conquer (DAC) DST paradigm and a multi-domain dialog synthesis framework to build multi- domain DST models from single-domain dialogues. |
| Outcome: | The proposed paradigm makes building multi-domain DST models easier on unseen domain combinations. |
Task-Adaptive Tokenization: Enhancing Long-Form Text Generation Efficacy in Mental Health and Beyond (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate text in mental health are limiting, but they are effective for many tasks. |
| Approach: | They propose a task-adaptive tokenizer that allows for the integration of task-specific tokens into the pre-trained model's tokenization step. |
| Outcome: | The proposed tokenization approach improves generation performance on psychological question-answering tasks in Chinese and English while using 60% fewer tokens. |
Transferable Persona-Grounded Dialogues via Grounded Minimal Edits (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing grounded dialogue models are limited by the distribution of data and the type of grounded concepts. |
| Approach: | They propose a framework which edits existing responses to be grounded on a given concept by disentangling and recombining persona-related and persona agnostic parts of the response. |
| Outcome: | The proposed framework outperforms baselines on the personaMi-nEdit dataset and shows that it can improve persona consistency while preserving the use of knowledge and empathy. |
ExpanRL: Hierarchical Reinforcement Learning for Course Concept Expansion in MOOCs (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing methods for concept expansion in MOOCs are inefficient because of the diversity of MOOC courses and rapid updates. |
| Approach: | They propose an end-to-end hierarchical reinforcement learning (HRL) model for concept expansion in MOOCs that employs a two-level mechanism of seed selection and concept expansion. |
| Outcome: | The proposed model improves on nine real MOOC datasets and maintains competitive performance under different settings. |
DC-Instruct: An Effective Framework for Generative Multi-intent Spoken Language Understanding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing prompt learning frameworks lack explicit modeling of dual-task dependencies and oversight of task-specific semantic differences among utterances. |
| Approach: | They propose a generative framework based on Dual-task Inter-dependent Instructions (DII) and Supervised Contrastive Instructions that leverages utterance semantics differences by guiding LLMs to determine whether a pair of utterrances share the same or similar labels. |
| Outcome: | The proposed framework outperforms existing models and state-of-the-art methods on public benchmark datasets and shows that it improves SLU reasoning. |
SocialEval: Evaluating Social Intelligence of Large Language Models (2025.acl-long)
Copied to clipboard
Jinfeng Zhou, Yuxuan Chen, Yihan Shi, Xuanming Zhang, Leqi Lei, Yi Feng, Zexuan Xiong, Miao Yan, Xunzhi Wang, Yaru Cao, Jianing Yin, Shuai Wang, Quanyu Dai, Zhenhua Dong, Hongning Wang, Minlie Huang
| Challenge: | Existing work on LLMs does not address their social intelligence (SI) and their discrepancy with humans. |
| Approach: | They propose a script-based bilingual SI benchmark that integrates outcome-oriented goal achievement evaluation and process-oriented interpersonal ability evaluation by manually crafting narrative scripts. |
| Outcome: | The proposed model is based on a script-based bilingual evaluation paradigm that integrates outcome- and process-oriented evaluation by manually crafting narrative scripts. |
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural approaches for natural language generation are typically developed offline for specific domains. |
| Approach: | They propose a method to expand NLG knowledge incrementally to new domains . major challenge is catastrophic forgetting, meaning a model forgets the knowledge it has learned before . |
| Outcome: | The proposed method outperforms other methods by effectively mitigating catastrophic forgetting issue. |
S^4: Operationalizing Speech Act Theory for Strategic Semi-Structured Psychiatric Interview (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for psychiatric interviewing degenerate into rigid interrogation or aimless chitchat due to a lack of strategic planning. |
| Approach: | They propose a framework for psychiatric interviewing grounded in Speech Act Theory that integrates a large-scale dataset with fine-grained psychic speech act annotations. |
| Outcome: | The proposed framework outperforms baselines in psychiatric interviewing. |
Training Language Model to Critique for Better Refinement (2025.findings-acl)
Copied to clipboard
Tianshu Yu, Chao Xiang, Mingchuan Yang, Pei Ke, Bosi Wen, Cunxiang Wang, Jiale Cheng, Li Zhang, Xinyu Mu, Chuxiong Sun, Minlie Huang
| Challenge: | Large language models (LLMs) have remarkable evaluation and critique capabilities, providing insightful feedback and identifying flaws in various tasks. |
| Approach: | They propose a framework to train critic models using refinement signals to generate feedback loops where critiques guide the model in refining its responses. |
| Outcome: | The proposed framework outperforms traditional methods and open-source models in terms of critique quality and refinement outcomes. |
Speculating LLMs’ Chinese Training Data Pollution from Their Tokens (2025.emnlp-main)
Copied to clipboard
Qingjie Zhang, Di Wang, Haoting Qian, Liu Yan, Tianwei Zhang, Ke Xu, Qi Li, Minlie Huang, Hewu Li, Han Qiu
| Challenge: | Experiments on GPT and other 23 LLMs indicate that tokens widely exist while GPT’s vocabulary behaves the worst: more than 23% long Chinese tokens (i.e., a token with more than two Chinese characters) are either porn or online gambling. |
| Approach: | They propose to locate Polluted Chinese (PoC) tokens in LLMs and build a PoC token detector to label them in vocabularies by considering each token’s semantics and related contents from the search engines. |
| Outcome: | The proposed method predicts that the ratio of “*” related webpages in GPT-4o's training data is around 0.5%. |
Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findings (2025.findings-acl)
Copied to clipboard
Shujian Yang, Shiyao Cui, Chuanrui Hu, Haicheng Wang, Tianwei Zhang, Minlie Huang, Jialiang Lu, Han Qiu
| Challenge: | Recent studies show that character substitutions in toxic Chinese text can confuse state-of-the-art LLMs. |
| Approach: | They propose a taxonomy of 3 perturbation strategies and 8 specific approaches in Chinese text to assess if they can detect perturbed Chinese toxic contents. |
| Outcome: | The proposed model can detect perturbed Chinese text with 8 different approaches . the proposed model is compared with 9 other LLMs from the US and China . |
Glyph: Scaling Context Windows via Visual-Text Compression (2026.acl-long)
Copied to clipboard
Jiale Cheng, Yusen Liu, Xinyu Zhang, Yulin Fei, Wenyi Hong, Ruiliang Lyu, Weihan Wang, Zhe Su, Xiaotao Gu, Xiao Liu, Yushi Bai, Jie Tang, Hongning Wang, Minlie Huang
| Challenge: | Large language models (LLMs) traditionally represent text as sequences of discrete tokens . a long-context scaling problem requires processing more tokens more efficiently . |
| Approach: | They propose a framework that renders long texts into compact visual pages and processes them with a vision-language model. |
| Outcome: | The proposed framework renders long texts into compact visual pages and processes them with a vision-language model. |
LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models (2025.findings-acl)
Copied to clipboard
Jiayi Gui, Yiming Liu, Jiale Cheng, Xiaotao Gu, Xiao Liu, Hongning Wang, Yuxiao Dong, Jie Tang, Minlie Huang
| Challenge: | Large Language Models (LLMs) have demonstrated notable capabilities across various tasks, showcasing complex problem-solving abilities. |
| Approach: | They propose a benchmark to evaluate the rule-based logical reasoning capabilities of Large Language Models (LLMs) they create simulated scenarios in which models execute or plan operations to achieve specific outcomes. |
| Outcome: | The proposed benchmark evaluates the performance of large language models on a variety of scenarios with varying difficulty levels. |
KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing knowledge-driven dialog data is limited due to the lack of dialog data which consists of multi-turn conversations on multiple topics and with knowledge annotations. |
| Approach: | They propose a Chinese multi-domain knowledge-driven conversation dataset which grounds the topics in multi-turn conversations to knowledge graphs. |
| Outcome: | The proposed dataset can be enhanced by introducing background knowledge, but there is still a large space for leveraging knowledge to model multi-turn conversations for further research. |
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark (2022.findings-acl)
Copied to clipboard
Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, Minlie Huang
| Challenge: | Dialogue safety problems severely limit the real-world deployment of generative conversational models. |
| Approach: | They propose a taxonomy for dialogue safety specifically designed to capture unsafe behaviors in human-bot dialogue settings. |
| Outcome: | The proposed taxonomy captures unsafe behaviors in human-bot dialogue settings with rich context-sensitive unsafe examples. |
CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset (2020.tacl-1)
Copied to clipboard
| Challenge: | Despite the significant contributions to the community, there is still a gap between existing dialogue corpora and real-life human dialogue data. |
| Approach: | They propose to develop Chinese cross-domain wizard-of-oz task-oriented dataset CrossWOZ with rich annotations of dialogue states and dialogue acts on both user and system sides. |
| Outcome: | The proposed dataset contains 6K dialogue sessions and 102K utterances for 5 domains, including hotel, restaurant, attraction, metro, and taxi. |
PPT: Pre-trained Prompt Tuning for Few-shot Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Prompt tuning for pre-trained language models has shown remarkable performance . however, prompt tuning is still not fully explored . |
| Approach: | They propose to pre-train prompts by adding soft prompts into the pre-training stage to obtain a better initialization. |
| Outcome: | The proposed framework outperforms full-model tuning under full-data and few-shot learning settings. |
PAL: Persona-Augmented Emotional Support Conversation Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has demonstrated the effectiveness of dialogue models in providing emotional support due to the lack of human resources for mental health support. |
| Approach: | They propose a framework for dynamically inferring and modeling seekers’ persona from the conversation history and a model that leverages persona information to provide personalized emotional support. |
| Outcome: | The proposed model outperforms baseline models on the studied benchmark. |
Black-Box Prompt Optimization: Aligning Large Language Models without Model Training (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models are often not well aligned with human intents, which requires additional training. |
| Approach: | They propose to use Black-Box Prompt Optimization (BPO) to perform alignments on large language models that are not well aligned with human intents. |
| Outcome: | The proposed model outperforms existing models and is model-agnostic. |
Long and Diverse Text Generation with Planning-based Hierarchical Variational Model (D19-1)
Copied to clipboard
| Challenge: | Existing methods for data-to-text generation are insufficient to produce long and diverse texts. |
| Approach: | They propose a planning-based hierarchical variational model that plans a sequence of groups and then realizes each sentence conditioned on the planning result and the previously generated context. |
| Outcome: | The proposed model outperforms state-of-the-art models in long and diverse text generation. |
CASE: Aligning Coarse-to-Fine Cognition and Affection for Empathetic Response Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing empathetic dialogue models only consider the affective aspect of empathy, which limits the capability of emotional response generation. |
| Approach: | They propose a model that aligns the user's cognition and affection at both the coarse-grained and fine-grounded levels and then automatically and manually evaluates the model. |
| Outcome: | The proposed model outperforms state-of-the-art models and generates more empathetic and informative responses. |
LaMemo: Language Modeling with Look-Ahead Memory (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to model long-term dependencies are limited to long texts with thousands of words. |
| Approach: | They propose a look-ahead memory that augments the recurrence memory by attending to the right-side tokens and interpolating with the old memory states to maintain long-term information in the history. |
| Outcome: | Experiments on widely used language modeling benchmarks show that LaMemo outperforms baseline models with recurrence memory. |
Self-Supervised Sentence Polishing by Adding Engaging Modifiers (2023.acl-demo)
Copied to clipboard
| Challenge: | a typical way to polish sentences is to add engaging modifiers, which enhance the meaning of the sentence. |
| Approach: | They propose a task that requires polishing sentences while maintaining fluency . they remove engaging modifiers from public resources and fine-tune LongLM to reconstruct original sentences from corrupted ones. |
| Outcome: | The proposed model generates more engaging sentences with suitable modifiers than strong baselines while keeping fluency. |
CDConv: A Benchmark for Contradiction Detection in Chinese Conversations (2022.emnlp-main)
Copied to clipboard
Chujie Zheng, Jinfeng Zhou, Yinhe Zheng, Libiao Peng, Zhen Guo, Wenquan Wu, Zheng-Yu Niu, Hua Wu, Minlie Huang
| Challenge: | Existing methods for detecting dialogue contradictions are difficult due to contextualization nature of conversations. |
| Approach: | They propose a benchmark for Contradiction Detection in Chinese Conversations . they use automatic conversation generation to simulate common user behaviors . |
| Outcome: | The proposed benchmark simulated the user behaviors that trigger chatbots to make contradictions . the results show that the current state-of-the-art chatbot can be easily goaded into making contradictions. |
Depression Detection in Clinical Interviews with LLM-Empowered Structural Element Graph (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for assessing depression only capture part of relevant elements . scarcity of participant data constrains interview modeling due to privacy concerns . |
| Approach: | They propose a structural element graph (SEGA) that transforms clinical interviews into an expertise-inspired directed acyclic graph for comprehensive modeling. |
| Outcome: | The proposed model outperforms baseline methods and powerful LLMs on two real-world clinical datasets. |
Difference-aware Knowledge Selection for Knowledge-grounded Conversation Generation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge selection models are limited by the context, but the difference between selected knowledge at different turns is often overlooked. |
| Approach: | They propose a difference-aware knowledge selection method that computes the difference between the candidate knowledge sentences provided at the current turn and those chosen in the previous turns. |
| Outcome: | The proposed method outperforms the state-of-the-art methods in a knowledge-grounded dialog. |
Internal Value Alignment in Large Language Models through Controlled Value Vector Activation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing LLMs do not possess consistent values, but many have been developed to align them at the behavioral level, including supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF). |
| Approach: | They propose a Controlled Value Vector Activation method that directly aligns the internal values of Large Language Models by interpreting how a value is encoded in their latent representations. |
| Outcome: | The proposed method achieves highest success rate across 10 basic values without hurting model performance and fluency, and ensures target values even with opposite and potentially malicious input prompts. |
ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format (2023.emnlp-demo)
Copied to clipboard
Qi Zhu, Christian Geishauser, Hsien-chin Lin, Carel van Niekerk, Baolin Peng, Zheng Zhang, Shutong Feng, Michael Heck, Nurul Lubis, Dazhen Wan, Xiaochen Zhu, Jianfeng Gao, Milica Gasic, Minlie Huang
| Challenge: | Existing tools for building TOD systems often lack a user-friendly interface . a toolkit with advanced, easily integrable modules is needed to bridge this gap . |
| Approach: | They propose a multifaceted dialogue system toolkit that integrates diverse datasets and models with a streamlined training process and in-depth evaluation tools. |
| Outcome: | The proposed toolkit combines RL and transfer learning to support the rapid development and evaluation of robust dialogue policies. |
Language Models Hallucinate, but May Excel at Fact Verification (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have produced non-factual outputs . however, current LLMs suffer from the hallucination issue . |
| Approach: | They propose to use instruction-tuned LLMs to generate factual outputs . they find that FLAN-T5-11B performs best as a fact verifier . |
| Outcome: | The proposed method outperforms more capable LLMs like GPT3.5 and ChatGPT in the human evaluation. |
NAST: A Non-Autoregressive Generator with Word Alignment for Unsupervised Text Style Transfer (2021.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive text style transfer models often ignore part of the source sentence and generate some irrelevant words with strong styles. |
| Approach: | They propose a non-autoregressive generator for unsupervised text style transfer which explicitly models word alignments to suppress irrelevant words. |
| Outcome: | The proposed generator significantly improves performance and provides explainable word alignments. |
Re3Dial: Retrieve, Reorganize and Rescale Conversations for Long-Turn Open-Domain Dialogue Pre-training (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-training models lack long-turn dialogue sessions due to the scarcity of long-term sessions. |
| Approach: | They propose a framework that can automatically construct billion-scale long-turn dialogues by reorganizing existing short-turn ones. |
| Outcome: | The proposed framework can automatically construct billion-scale long-turn dialogues by reorganizing existing short-turn ones. |
Understanding the Dark Side of LLMs’ Intrinsic Self-Correction (2025.acl-long)
Copied to clipboard
Qingjie Zhang, Di Wang, Haoting Qian, Yiming Li, Tianwei Zhang, Minlie Huang, Ke Xu, Hewu Li, Liu Yan, Han Qiu
| Challenge: | Recent studies show that LLMs’ intrinsic self-correction fails without oracle labels as feedback. |
| Approach: | They propose to use one simple task and three complex tasks with state-of-the-art LLMs like ChatGPT, Llama, and DeepSeek to interpret LLM's intrinsic self-correction. |
| Outcome: | The proposed methods reveal the dark side of LLMs’ intrinsic self-correction for different tasks, especially for those failure cases. |
Diversifying Dialog Generation via Adaptive Label Smoothing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to dialog generation suffer from the over-confidence issue, which leads to poor generation diversity. |
| Approach: | They propose an Adaptive Label Smoothing approach that can adaptively estimate a target label distribution at each time step for different dialog contexts. |
| Outcome: | The proposed approach outperforms competing models on two benchmark datasets in producing diverse responses. |
Robustness to Modification with Shared Words in Paraphrase Identification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Paraphrase identification models have been shown to be vulnerable and lack robustness in tasks such as text classification and natural language inference. |
| Approach: | They propose to modify an example such that a target model makes a wrong prediction by using beam search constrained by heuristic rules and a BERT-masked language model to generate substitution words compatible with the context. |
| Outcome: | The proposed model performance drops dramatically on modified examples, revealing the robustness issue. |
Learning to Ask Questions in Open-domain Conversational Systems with Typed Decoders (P18-1)
Copied to clipboard
| Challenge: | Extensive experiments show that typed decoders outperform state-of-the-art baselines and can generate more meaningful questions. |
| Approach: | They devised two typed decoders that generate questions with different types of interrogatives, topic words, and ordinary words. |
| Outcome: | Extensive experiments show that the typed decoders outperform state-of-the-art baselines and can generate more meaningful questions. |
ChID: A Large-scale Chinese IDiom Dataset for Cloze Test (P19-1)
Copied to clipboard
| Challenge: | cloze-style reading comprehension in Chinese is limited due to the lack of various corpora. |
| Approach: | They propose a large-scale Chinese cloze test dataset ChID which studies the comprehension of idiom in Chinese. |
| Outcome: | The proposed dataset compares the performance of the proposed model with human models. |
A Semantic-based Method for Unsupervised Commonsense Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to score candidates without labeled task data are difficult to use . e.g., pre-trained language models can be easily affected by irrelevant factors . |
| Approach: | They propose a method that generates plausible answers with generative models and uses them to select the correct answer. |
| Outcome: | The proposed method achieves the best results in unsupervised situations. |
DYNTEXT: Semantic-Aware Dynamic Text Sanitization for Privacy-Preserving LLM Inference (2025.findings-acl)
Copied to clipboard
Juhua Zhang, Zhiliang Tian, Minghang Zhu, Yiping Song, Taishu Sheng, Siyi Yang, Qiunan Du, Xinwang Liu, Minlie Huang, Dongsheng Li
| Challenge: | Existing methods to protect privacy of sensitive data are differential privacy (DP) and DP is used to protect users from privacy leakage. |
| Approach: | They propose an LDP-based Dynamic Text sanitization for privacy-preserving LLM inference that dynamically constructs semantic-aware adjacency lists of sensitive tokens to sample non-sensitive tokens for perturbation. |
| Outcome: | The proposed model excels on three datasets. |
Learning Goal-oriented Dialogue Policy with opposite Agent Awareness (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing approaches for goal-oriented dialogue policy learning focus on the target agent policy and treat the opposite agent policy as part of the environment. |
| Approach: | They propose a framework for policy learning in goal-oriented dialogues that uses the opposite agent's policy estimation to improve the target agent by regarding it as part of the target policy. |
| Outcome: | The proposed framework shows superior performance over state-of-the-art models on cooperative and competitive dialogue tasks. |
Towards Emotional Support Dialog Systems (2021.acl-long)
Copied to clipboard
| Challenge: | Emotional support is a crucial ability for many conversation scenarios, including social interactions, mental health support, and customer service chats. |
| Approach: | They propose an Emotional Support Conversation task and an ESC Framework to train emotional support into dialog systems. |
| Outcome: | The proposed framework provides an example of an Emotional Support Conversation task and shows that it is more effective than existing models. |
Rethinking and Refining the Distinct Metric (2022.acl-short)
Copied to clipboard
| Challenge: | Existing methods for calculating distinct scores have evident biases that assign higher penalties to longer sequences. |
| Approach: | They propose to scale the number of distinct tokens based on their expectations. |
| Outcome: | The proposed metric removes evident biases in the original distinct score . the proposed meter correlates better with human judgment in evaluating response diversity . |
IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation (2026.acl-long)
Copied to clipboard
Bosi Wen, Yilin Niu, Cunxiang Wang, Pei Ke, Xiaoying Ling, Ying Zhang, Aohan Zeng, Hongning Wang, Minlie Huang
| Challenge: | Existing evaluation models for instruction-following have many shortcomings, such as substantial costs and unreliable assessments. |
| Approach: | They propose an LLM critic for fine-grained instruction-following evaluation using a checklist generator and a constraint-level preference optimization method. |
| Outcome: | The proposed model beats strong LLM-as-a-Judge baselines in evaluations under lower computational overhead compared to baselines. |
EmoBench: Evaluating the Emotional Intelligence of Large Language Models (2024.acl-long)
Copied to clipboard
Sahand Sabour, Siyang Liu, Zheyuan Zhang, June Liu, Jinfeng Zhou, Alvionna Sunaryo, Tatia Lee, Rada Mihalcea, Minlie Huang
| Challenge: | Existing benchmarks for Emotional Intelligence (EI) focus on emotion recognition, neglecting essential EI capabilities. |
| Approach: | They propose a benchmark that proposes a comprehensive definition for machine EI . they propose 400 hand-crafted questions in English and Chinese to evaluate EI. |
| Outcome: | The proposed benchmarks focus on emotion recognition, neglecting EI capabilities . they are constructed from existing datasets, which include frequent patterns and errors . the proposed benchmark includes questions in English and Chinese that require thorough reasoning and understanding . |
IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for instruction-following lack data coverage and oversimplified pairwise evaluation paradigms that misalign with model optimization scenarios. |
| Approach: | They propose a meta-evaluation benchmark for instruction-following that covers diverse instruction and constraint types and a preference graph for each instruction. |
| Outcome: | Extensive experiments on IF-RewardBench show that the proposed benchmark achieves a stronger positive correlation with downstream task performance compared to existing benchmarks. |
Pre-Training to Learn in Context (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models are not explicitly trained to learn in context. |
| Approach: | They propose a framework to enhance in-context learning by pre-training language models on a large collection of "intrinsic tasks" they evaluate the in-constitution learning performance of the model trained with PICL on seven widely-used text classification datasets and the Super-NaturalInstrctions benchmark . |
| Outcome: | The proposed framework outperforms larger language models with nearly 4x parameters on seven widely-used datasets and the Super-NaturalInstrctions benchmark. |
WSpeller: Robust Word Segmentation for Enhancing Chinese Spelling Check (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Chinese spelling check (CSC) detects and corrects spelling errors in Chinese texts. |
| Approach: | They propose a Chinese spelling check model that takes into account word segmentation and a module that can assist the correction module by predicting correct word segmentations from sentences containing spelling errors. |
| Outcome: | The proposed model outperforms baselines on SIGHAN13, SIGHEN14, and SIGHAN15 and maintains equal performance on SSGHAN14. |
DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer (2021.emnlp-main)
Copied to clipboard
| Challenge: | Generating long passages that maintain long-range coherence is a long-standing problem in natural language generation (NLG). |
| Approach: | They propose a discourse-aware discrete variational Transformer that learns a latent variable sequence that summarizes the global structure of the text and then applies it to guide the generation process at each decoding step. |
| Outcome: | The proposed model can generate long texts with better long-range coherence by learning a latent variable sequence with each latent code. |
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has proposed to improve relevance modeling by having large language models actively involved in retrieval, i.e., to guide retrieval with generation. |
| Approach: | They propose to have large language models actively involved in retrieval to guide retrieval with generation. |
| Outcome: | The proposed method synergizes retrieval and generation in an iterative manner, and can generate better results in subsequent iterations. |
Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches that integrate commonsense knowledge into pre-trained language models simply transfer relational knowledge while ignoring rich connections within the knowledge graph. |
| Approach: | They propose a method that leverages structural and semantic information of the knowledge graph to generate commonsense-aware text. |
| Outcome: | The proposed method outperforms baseline models on three text generation tasks that require reasoning over commonsense knowledge. |
LegalAgentBench: Evaluating LLM Agents in Legal Domain (2025.acl-long)
Copied to clipboard
Haitao Li, Junjie Chen, Jingli Yang, Qingyao Ai, Wei Jia, Youfeng Liu, Kai Lin, Yueyue Wu, Guozhi Yuan, Yiran Hu, Wuyue Wang, Yiqun Liu, Minlie Huang
| Challenge: | Existing general-domain benchmarks do not capture complexity of real-world judicial cognition and decision-making. |
| Approach: | They propose a benchmark specifically designed to evaluate LLM Agents in the legal domain. |
| Outcome: | The proposed benchmark includes 17 corpora from real-world legal scenarios and provides 37 tools for interacting with external knowledge. |
JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained models for knowledgegraph-to-text generation ignore graph structure during encoding and lack elaborate pre-training tasks to explicitly model graph-text alignments. |
| Approach: | They propose a graph-text joint representation learning model called JointGT which incorporates a structure-aware semantic aggregation module into each Transformer layer to preserve the graph structure. |
| Outcome: | The proposed model achieves state-of-the-art performance on various KG-to-text datasets. |
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing models for story generation suffer from repetition, logic conflicts, and lack of long-range coherence . |
| Approach: | They propose to utilize commonsense knowledge from external knowledge bases to generate reasonable stories by multi-task learning. |
| Outcome: | The proposed model can generate more reasonable stories than state-of-the-art models, compared with existing models, showing that it can capture useful semantic and syntactic features. |
Multi-Source Probing for Open-Domain Conversational Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models of open-domain dialogue comprehension have limited conversational understanding and response generation. |
| Approach: | They propose a multi-source probing method to probe dialogue comprehension abilities of open-domain dialogue models. |
| Outcome: | The proposed method aggregates features from multiple sources to accomplish diverse task goals and conducts downstream tasks in a generative manner consistent with dialogue model pre-training to leverage model capabilities. |
A Corpus for Understanding and Generating Moral Stories (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing tasks for evaluating story understanding and generation focus on reasoning plots from context, but they focus on bridging plots with implied morals. |
| Approach: | They propose two understanding tasks and two generation tasks to assess machines' ability to bridge story plots and implied morals. |
| Outcome: | The proposed tasks are based on a dataset of Chinese and English moral stories . they show that the proposed models can perform better than existing models . |
Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing tasks that use commonsense reasoning as multi-choice reading comprehension lack direct assessment to machine commonsence and impede its practicability to realistic scenarios. |
| Approach: | They propose a method that first extracts the underlying concepts which are served as bridges in the reasoning chain and then integrates these concepts to generate the final explanation. |
| Outcome: | The proposed model outperforms the state-of-the-art models in automatic and human evaluation. |
ETHICIST: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimation (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models memorize a considerable fraction of training data, leading to privacy risk of information leakage. |
| Approach: | They propose a method for targeted training data extraction using a smoothed soft prompting and calibrated confidence estimation. |
| Outcome: | The proposed method significantly improves the extraction performance on a recently proposed public benchmark. |
Stylized Story Generation with Style-Guided Planning (2021.findings-acl)
Copied to clipboard
| Challenge: | Current storytelling systems focus more on generating stories with coherent plots regardless of the narration style. |
| Approach: | They propose a novel task, stylized story generation, that first plans stylized keywords and then generates the whole story with the guidance of the keywords. |
| Outcome: | The proposed model can generate emotion-driven or event-driven stories based on the ROCStories dataset . |
CoMAE: A Multi-factor Hierarchical Framework for Empathetic Response Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for empathetic response generation ignore hierarchical relationships between different factors, leading to a weak ability of empathy modeling. |
| Approach: | They propose a multi-factor hierarchical framework for empathetic response generation which models the above three key factors in a hierarchically structured way. |
| Outcome: | The proposed model generates more empathetic responses than previous methods. |
MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions (2023.acl-long)
Copied to clipboard
| Challenge: | A moral dialogue system aligned with users’ values could enhance conversation engagement and user connections. |
| Approach: | They propose a framework to train and evaluate moral dialogue systems based on communication mechanisms of morality and a method to construct moral discussions between simulated users and the dialogue system. |
| Outcome: | The proposed framework can train and evaluate moral dialogue systems based on simulated users and their values . |
CR-Walker: Tree-Structured Graph Reasoning and Dialog Acts for Conversational Recommendation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems that explore user preference through conversational interactions do not exploit the context and knowledge to make accurate recommendations. |
| Approach: | They propose a model that performs tree-structured reasoning on a knowledge graph and generates informative dialog acts to guide language generation. |
| Outcome: | The proposed model can arrive at more accurate recommendation and generate more informative and engaging responses. |
LongSafety: Evaluating Long-Context Safety of Large Language Models (2025.acl-long)
Copied to clipboard
Yida Lu, Jiale Cheng, Zhexin Zhang, Shiyao Cui, Cunxiang Wang, Xiaotao Gu, Yuxiao Dong, Jie Tang, Hongning Wang, Minlie Huang
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities in understanding and generating long sequences. |
| Approach: | They propose a benchmark to evaluate LLM safety in open-ended long-context tasks . they find that relevant context and extended input sequences can exacerbate safety risks . |
| Outcome: | The proposed benchmark identifies significant safety vulnerabilities in 16 LLMs . strong safety performance in short-context scenarios does not correlate with safety in long-contact tasks . |
DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are susceptible to generating harmful content when prompted with carefully crafted inputs, a vulnerability known as LLM jailbreaking. |
| Approach: | They propose an end-to-end generative approach for jailbreak rewriting inspired by diffusion models that uses a sequence-tosequence (seq2sequ) diffusion model as a generator, conditioning on the original prompt and guiding the denoising process with a novel attack loss. |
| Outcome: | Experiments on Advbench and Harmbench show that the proposed method outperforms autoregressive jailbreak models across evaluation metrics including ASR, fluency, diversity and diversity. |
Robustness Testing of Language Understanding in Task-Oriented Dialog (2021.acl-long)
Copied to clipboard
Jiexi Liu, Ryuichi Takanobu, Jiaxin Wen, Dazhen Wan, Hongguang Li, Weiran Nie, Cheng Li, Wei Peng, Minlie Huang
| Challenge: | a lack of systematic studies on the robustness of language understanding models in task-oriented dialog systems is limiting . authors propose a model-agnostic toolkit LAUG to approximate natural language perturbations . |
| Approach: | They propose a model-agnostic toolkit LAUG to approximate natural language perturbations for testing the robustness of language understanding models in task-oriented dialog systems. |
| Outcome: | The proposed toolkit reveals critical robustness issues in state-of-the-art models. |
Battling against Tough Resister: Strategy Planning with Adversarial Game for Non-collaborative Dialogues (2025.acl-long)
Copied to clipboard
| Challenge: | Non-collaborative dialogue involves two participants with conflicting interests engaging in multiround dialogue to achieve their own goals. |
| Approach: | They propose a Game-based Adversarial self-play InterActive training paradigm which constructs an adversarial two-player (a persuader and a resister) zero-sum game and guides the game to approximate Nash Equilibrium (NE) via reinforcement learning. |
| Outcome: | The proposed model achieves state-of-the-art performance on three datasets. |
DCMKC: A Dual Consistency Matching Approach for Multi-hop Question Answering in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing reasoning based on chains of thought (CoTs) fails to find logical connections between reasoning steps . |
| Approach: | They propose a method to match KG reasoning chains with CoTs based on semantic similarity . they use a knowledge graph to find relevant information "within" each reasoning step . |
| Outcome: | The proposed method outperforms baselines on multi-answer questions with 5.1% improvement over baselines. |
Instruction Pre-Training: Language Models are Supervised Multitask Learners (2024.emnlp-main)
Copied to clipboard
| Challenge: | Unsupervised multitask pre-training has been the key to the success of language models (LMs) however, scaling it in the post-training stage trends towards better generalization. |
| Approach: | They propose a framework that augments massive raw corpora with instruction-response pairs to pre-train LMs. |
| Outcome: | The proposed framework augments massive raw corpora with instruction-response pairs to pre-train LMs. |
An Interpretable Reasoning Network for Multi-Relation Question Answering (C18-1)
Copied to clipboard
| Challenge: | Existing models for multi-relation question answering require elaborated analysis and reasoning over multiple fact triples in knowledge base. |
| Approach: | They propose a model that employs an interpretable hop-by-hop reasoning process for question answering . it decides which part of an input question should be analyzed at each hop and then drives next-hop thinking . |
| Outcome: | The proposed model yields state-of-the-art results on two datasets. |
Mitigating the Learning Bias towards Repetition by Self-Contrastive Training for Open-Ended Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing language models generate repetitive texts with greedy decoding or beam search. |
| Approach: | They propose a self-contrastive training technique to penalize the output of a premature checkpoint of the same model when it incorrectly predicts repetition. |
| Outcome: | The proposed training mitigates repetition while maintaining fluency while minimizing the overestimation of token-level repetition probabilities. |
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning (2021.acl-long)
Copied to clipboard
Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, Jie Zhou
| Challenge: | Existing pre-training objectives do not explicitly model relational facts in text . Experimental results show that ERICA can improve typical PLMs on several language understanding tasks, including relation extraction, entity typing and question answering. |
| Approach: | They propose a contrastive learning framework ERICA to obtain a deep understanding of entities and relations in text. |
| Outcome: | The proposed framework can improve PLMs on several language understanding tasks, especially under low-resource settings. |
Unveiling the Implicit Toxicity in Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies focus on probing toxic outputs that can be easily detected with existing toxicity classifiers, but LLMs can generate diverse implicit toxic output that are difficult to detect via simply zero-shot prompting. |
| Approach: | They propose a reinforcement learning based attacking method to induce the implicit toxic outputs in large language models by fine-tuning toxicity classifiers. |
| Outcome: | The proposed method generates implicit toxic outputs that are difficult to detect via zero-shot prompting on five widely-adopted toxicity classifiers. |
AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversation (2023.findings-acl)
Copied to clipboard
| Challenge: | Crowdsourced dialogue corpora are limited in scale and topic coverage due to the expensive cost of data curation. |
| Approach: | They construct an augmented dataset for the emotional support conversation task using large language models for dialogue augmentation. |
| Outcome: | The proposed approach outperforms baselines of dialogue augmentation and improves the model's generalization ability to open-domain topics. |
Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark (2022.findings-emnlp)
Copied to clipboard
Jingyan Zhou, Jiawen Deng, Fei Mi, Yitong Li, Yasheng Wang, Minlie Huang, Xin Jiang, Qun Liu, Helen Meng
| Challenge: | a number of safety concerns hinder the deployment of open-domain dialog systems, such as offensive languages and toxic behaviors, such social bias is difficult to detect. |
| Approach: | They propose a Dial-Bias Framework for analyzing social bias in conversations . they introduce a Chinese social bias dialog dataset and conduct in-depth ablation studies . |
| Outcome: | The proposed framework is the first annotated Chinese social bias dialog dataset . the proposed framework also provides a fine-grained dialog bias measurement benchmark . |
DPGA-TextSyn: Differentially Private Genetic Algorithm for Synthetic Text Generation (2025.findings-acl)
Copied to clipboard
Zhonghao Sun, Zhiliang Tian, Yiping Song, Yuyi Si, Juhua Zhang, Minlie Huang, Kai Lu, Zeyu Xiong, Xinwang Liu, Dongsheng Li
| Challenge: | Existing methods to fine-tune large language models pose privacy risks . researchers have synthesized data with strong generation capabilities closed-source LLMs to alleviate this problem . |
| Approach: | They propose to combine general LLMs with genetic algorithm to produce relevant and diverse synthetic text under differential privacy constraints. |
| Outcome: | The proposed method significantly improves the performance of the model in downstream tasks while ensuring privacy. |
Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization (2024.acl-long)
Copied to clipboard
| Challenge: | Despite the success of jailbreaking attacks, there is a lack of effort in defending against them. |
| Approach: | They propose to integrate goal prioritization at both training and inference stages to counteract this conflict between the goals of being helpful and ensuring safety. |
| Outcome: | The proposed approach reduces the Attack Success Rate (ASR) of jailbreaking from 66.4% to 3.6% for ChatGPT and reduces it from 71.0% to 6.6% for Llama2-13B. |
Bridging the Gap between Synthetic and Natural Questions via Sentence Decomposition for Semantic Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing methods to train a parser to perform zero-shot learning are limited by the lack of training data. |
| Approach: | They propose a decomposition-based method to unify the sentence structures of questions . their method can generalize to natural questions with novel text expressions . |
| Outcome: | The proposed method improves on synthetic data and on complex web questions with novel expressions. |
HPSS: Heuristic Prompting Strategy Search for LLM Evaluators (2025.findings-acl)
Copied to clipboard
Bosi Wen, Pei Ke, Yufei Sun, Cunxiang Wang, Xiaotao Gu, Jinfeng Zhou, Jie Tang, Hongning Wang, Minlie Huang
| Challenge: | Existing efforts to optimize text evaluation prompts neglect the combinatorial impact of multiple factors, leading to insufficient optimization of the evaluation pipeline. |
| Approach: | They propose to integrate 8 key factors for evaluation prompts and integrate them into an algorithm that searches for well-behaved prompting strategies for LLM evaluators. |
| Outcome: | The proposed method outperforms existing methods and human-designed evaluation prompts on four evaluation tasks. |
Persona-Guided Planning for Controlling the Protagonist’s Persona in Story Generation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to control the protagonist's persona in story generation are implicitly and sparsely embodied in stories, so we propose a planning-based generation model called ConPer to explicitly model the relationship between personas and events. |
| Approach: | They propose a model to control the protagonist's persona in story generation by predicting one target sentence and planning the plot as a sequence of keywords with the guidance of the predicted persona-related events and commonsense knowledge. |
| Outcome: | The proposed model outperforms state-of-the-art models for generating more coherent and persona-controllable stories. |
ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems (2020.acl-demos)
Copied to clipboard
Qi Zhu, Zheng Zhang, Yan Fang, Xiang Li, Ryuichi Takanobu, Jinchao Li, Baolin Peng, Jianfeng Gao, Xiaoyan Zhu, Minlie Huang
| Challenge: | ConvLab-2 inherits Convlab's framework but integrates more powerful dialogue models and supports more datasets. |
| Approach: | They present ConvLab-2, an open-source toolkit that enables researchers to build task-oriented dialogue systems with state-of-the-art models and perform an end-to-end evaluation. |
| Outcome: | The new tool inherits ConvLab's framework and extends it by integrating many recently proposed state-of-the-art dialogue models. |
CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation (2024.acl-long)
Copied to clipboard
Pei Ke, Bosi Wen, Andrew Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang
| Challenge: | Existing models for NLP evaluations lack the ability to generate informative critiques in pointwise grading and pairwise comparison especially without references. |
| Approach: | They propose a method which can acquire pointwise grading critiques with pseudo references and revise these critiques via multi-path prompting to obtain informative evaluation data in different tasks and settings. |
| Outcome: | The proposed method outperforms all open-source models and even GPT-4 in system-level correlations of pointwise grading. |
Continual Prompt Tuning for Dialog State Tracking (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to train a model on a sequence of tasks are not efficient enough to mitigate catastrophic forgetting. |
| Approach: | They propose a parameter-efficient framework that prevents forgetting and enables knowledge transfer between tasks by learning and freezing a pre-trained model. |
| Outcome: | The proposed framework avoids forgetting and enables knowledge transfer between tasks. |
ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings (2024.emnlp-main)
Copied to clipboard
| Challenge: | Attaching suffixes to harmful instructions can hack the defense of Large language models (LLMs) However, due to the unreadable of adversarial suffix, it can be relatively easily penetrated by common defense methods such as perplexity filters. |
| Approach: | They propose an algorithm to embed adversarial suffixes into coherent and understandable text to attack Large language models (LLMs) using a Advbench dataset. |
| Outcome: | The proposed approach reduces the computation time of adversarial suffixes and achieves a much better attack success rate than existing techniques. |
New Terms, New Toxicity: Consensus-based Chinese Neologism Toxicity Detection via Search-Augmented LLMs (2026.acl-long)
Copied to clipboard
Shiyao Cui, QingLin Zhang, Di Wang, Yida Lu, Zhexin Zhang, Jinhua Gao, Jinglin Yang, Min He, Han Qiu, Minlie Huang
| Challenge: | Neologisms can foster new linguistic consensus by stabilizing shared meanings and usage in common communicative norms. |
| Approach: | They propose a taxonomy that captures the origins and consensus-verification criteria of toxic neologisms . they propose 'SeTox' framework that integrates real-time web context for naeologim detection . |
| Outcome: | The proposed framework outperforms large-scale models in detecting neologism toxicity. |
CharacterGLM: Customizing Social Characters with Large Language Models (2024.emnlp-industry)
Copied to clipboard
Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang
| Challenge: | Character-based dialogue systems (CharacterDial) allow users to customize social characters for social interactions. |
| Approach: | They will collect a large-scale Chinese corpus of characters with diverse categories and behaviors and develop CharacterGLM models to address these challenges. |
| Outcome: | Experiments show that CharacterGLM outperforms most popular open- and closed-source LLMs and performs comparable to GPT-4. |
Facilitating Multi-turn Emotional Support Conversation with Positive Emotion Elicitation: A Reinforcement Learning Approach (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to provide emotional support (ESC) ignore the effect on ES and lack explicit goals to guide emotional positive transition. |
| Approach: | They propose a new paradigm to formalize multi-turn ESC as a process of positive emotion elicitation. |
| Outcome: | The proposed model outperforms existing models in achieving positive emotion elicitation while maintaining conversational goals like coherence. |
HoWToBench: Holistic Evaluation for LLM’s Capability in Human-level Writing using Tree of Writing (2026.acl-long)
Copied to clipboard
Andrew Zhuoer Feng, Cunxiang Wang, Yu Luo, Lin Fan, Irene Zhou, Zikang Wang, Xiaotao Gu, Jie Tang, Hongning Wang, Minlie Huang
| Challenge: | Evaluating the writing capabilities of large language models remains a significant challenge due to the multidimensional nature of writing skills and the limitations of existing metrics. |
| Approach: | They propose to model the aggregation weights of sub-features in a tree-structured workflow and propose a Chinese writing benchmark that mitigates biases. |
| Outcome: | The proposed tree-of-writing (ToW) measures the writing capabilities of large language models (LLMs) in Chinese and shows that it mitigates biases and achieves a *0.93* Pearson correlation with human judgments. |
SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained models neglect to consider linguistic knowledge of texts . existing models neglect linguistic information, which is important for sentiment analysis . |
| Approach: | They propose a model that introduces word-level linguistic knowledge into pre-trained models to enhance sentiment analysis by querying SentiWordNet to acquire sentiment polarity. |
| Outcome: | The proposed model obtains state-of-the-art performance on a variety of sentiment analysis tasks. |
OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metrics (2021.acl-long)
Copied to clipboard
Jian Guan, Zhexin Zhang, Zhuoer Feng, Zitao Liu, Wenbiao Ding, Xiaoxi Mao, Changjie Fan, Minlie Huang
| Challenge: | Existing automatic metrics are observed to correlate poorly with human evaluation. |
| Approach: | They propose to use OpenMEVA to evaluate open-ended story generation metrics. |
| Outcome: | The proposed test suite assesses the capabilities of open-ended story generation metrics on annotated stories and auto-constructed test examples. |
PsychePass: Calibrating LLM Therapeutic Competence via Trajectory-Anchored Tournaments (2026.findings-acl)
Copied to clipboard
| Challenge: | evaluating therapeutic competence of large language models remains challenging due to unstructured and longitudinal nature of counseling. |
| Approach: | They propose a framework that calibrates the therapeutic competence of LLMs via trajectory-anchored tournaments. |
| Outcome: | The proposed framework calibrates the therapeutic competence of LLMs via trajectory-anchored tournaments. |
Crisp: Cognitive Restructuring of Negative Thoughts through Multi-turn Supportive Dialogues (2025.emnlp-main)
Copied to clipboard
Jinfeng Zhou, Yuxuan Chen, Jianing Yin, Yongkang Huang, Yihan Shi, Xikun Zhang, Libiao Peng, Rongsheng Zhang, Tangjie Lv, Zhipeng Hu, Hongning Wang, Minlie Huang
| Challenge: | Existing approaches to cognitive restructuring (CR) are limited by entrenched cognitive distortions, emotional resistance, and individual differences. |
| Approach: | They propose a framework that structures CR as theory-grounded multi-stage multi-turn dialogue and a multi-channel loop mechanism to account for diverse individual distortions. |
| Outcome: | The proposed framework integrates supportive strategies for emotional management and a multi-channel loop mechanism to account for diverse individual distortions. |
MHALO: Evaluating MLLMs as Fine-grained Hallucination Detectors (2025.findings-acl)
Copied to clipboard
| Challenge: | Hallucination remains a critical challenge for multimodal large language models, undermining their reliability in real-world applications. |
| Approach: | They propose a benchmark specifically designed for evaluating MLLMs’ capability in performing token-level hallucination detection (FHD) . they use curated training data to train a specialized model that significantly outperforms existing models. |
| Outcome: | The proposed model outperforms existing models in the evaluation of 9 MLLMs and reaches an average F1IoU of 40.59%. |
COLD: A Benchmark for Chinese Offensive Language Detection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Offensive language detection is crucial for maintaining a civilized social media platform and deploying pre-trained language models. |
| Approach: | They propose a benchmark benchmark for Chinese offensive language analysis including a Chinese Offensive Language Dataset and a baseline detector which is trained on the dataset. |
| Outcome: | The proposed benchmark contributes to Chinese offensive language detection which is challenging for existing resources. |
StoryTrans: Non-Parallel Story Author-Style Transfer with Discourse Representations and Content Enhancing (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies on text style transfer neglect long style transfer at the discourse level. |
| Approach: | They propose a model that transfers text style into target styles with learnable style embeddings . they use a mask-and-fill framework to explicitly fuse style-specific keywords into generation . |
| Outcome: | The proposed model outperforms baselines in style transfer and content preservation. |
PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Support (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing research on text-based mental health counseling is limited due to the lack of relevant corpora in Chinese language. |
| Approach: | They propose a Chinese dataset of psychological health support in the form of question and answer pair that is crawled from a mental health service platform and contains 22K questions and 56K long and wellstructured answers. |
| Outcome: | The proposed dataset contains 22K questions and 56K long and wellstructured answers. |
Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decomposition (2020.acl-main)
Copied to clipboard
| Challenge: | Many studies have applied reinforcement learning to train a dialog policy . but modeling a real-world user simulator is challenging and requires domain expertise . |
| Approach: | They propose to build dialog policies with two agents as dialog agents to avoid building a user simulator beforehand. |
| Outcome: | The proposed method can build a system policy and a user policy simultaneously . it can achieve high task success rate through conversational interaction . |
Aligning Recommendation and Conversation via Dual Imitation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing conversational recommendation systems ignore the advantage of user interest shift in connecting recommendation and conversation, leading to an ineffective loose coupling structure. |
| Approach: | They propose a dual imitation to explicitly align recommendation and conversation paths . they propose to generate high-quality responses with accurate recommendations and coherent explanations . |
| Outcome: | The proposed model outperforms the state-of-the-art models on recommendation and conversation performance with automatic, human, and novel explainability metrics. |
AutoCAD: Automatically Generate Counterfactuals for Mitigating Shortcut Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating counterfactuals rely on human efforts or task-specific designs. |
| Approach: | They propose to use a fully automatic and task-agnostic CAD generation framework to generate diverse counterfactuals. |
| Outcome: | The proposed framework outperforms human-in-the-loop and task-specific CAD methods on multiple out-of-domain and challenge benchmarks. |
Click: Controllable Text Generation with Sequence Likelihood Contrastive Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Current language models have shown impressive capability of generating fluent and grammatical text, but they often produce behaviors misaligned with human expectations. |
| Approach: | They propose a new language model called Leo for controllable text generation which employs a contrastive loss on sequence likelihood which fundamentally decreases the generation probability of negative samples. |
| Outcome: | The proposed model outperforms baselines on language detoxification, sentiment steering, and repetition reduction tasks. |
A Unified Dialogue User Simulator for Few-shot Data Augmentation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to augment large-scale task-oriented dialogues rely on annotated data. |
| Approach: | They propose to build a unified dialogue user simulation model by pre-training on publicly available datasets. |
| Outcome: | The proposed model can be tuned on a target domain with few-shot data. |
360∘REA: Towards A Reusable Experience Accumulation with 360∘ Assessment for Multi-Agent System (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies focus on optimizing the agent team or employing self-reflection to iteratively solve complex tasks. |
| Approach: | They propose a hierarchical multi-agent framework that uses 360 assessment to accumulate experience through fine-grained assessment. |
| Outcome: | The proposed framework is based on corporate organizational practices and employs a dual-level experience pool for agents to accumulate experience through fine-grained assessment. |
ConvLab: Multi-Domain End-to-End Dialog System Platform (P19-3)
Copied to clipboard
Sungjin Lee, Qi Zhu, Ryuichi Takanobu, Zheng Zhang, Yaoqin Zhang, Xiang Li, Jinchao Li, Baolin Peng, Xiujun Li, Minlie Huang, Jianfeng Gao
| Challenge: | ConvLab is an open-source multi-domain end-to-end dialog system platform . it allows researchers to quickly set up experiments with reusable components and compare a large set of different approaches in common environments. |
| Approach: | They propose to use an open-source multi-domain end-to-end dialog system platform to train and evaluate dialog bots in common environments. |
| Outcome: | The proposed system enables researchers to quickly set up experiments with reusable components and compare a large set of different approaches in common environments. |
Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence (2021.acl-long)
Copied to clipboard
| Challenge: | Existing generation models struggle to maintain a coherent event sequence throughout the generated text. |
| Approach: | They propose a long text generation model which can represent prefix sentences at sentence level and discourse level in the decoding process. |
| Outcome: | The proposed model can generate more coherent texts than state-of-the-art models. |
Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generation (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing non-AutoRegressive (NAR) text generation models lack proper pre-training, making them far behind pre-trained autoregressive models. |
| Approach: | They propose a novel pre-training task to promote prediction consistency in non-autoregressive (NAR) generation. |
| Outcome: | The proposed model outperforms existing pre-trained models and achieves 17 times speedup in throughput. |
KuiLeiXi: a Chinese Open-Ended Text Adventure Game (2021.acl-demo)
Copied to clipboard
Yadong Xi, Xiaoxi Mao, Le Li, Lei Lin, Yanjiang Chen, Shuhan Yang, Xuhan Chen, Kailun Tao, Zhi Li, Gongzheng Li, Lin Jiang, Siyan Liu, Zeng Zhao, Minlie Huang, Changjie Fan, Zhipeng Hu
| Challenge: | Recent advances in pre-trained language models have made it possible to generate human-like text. |
| Approach: | They propose to integrate an open-ended text adventure game in Chinese, named KuiLeiXi, where players interact with the AI until the plot goals are reached. |
| Outcome: | The proposed game lacks incentives and relies on players to explore on their own. |
ARAML: A Stable Adversarial Training Framework for Text Generation (D19-1)
Copied to clipboard
| Challenge: | Existing generative adversarial networks suffer from the instability of reinforcement learning training algorithms such as policy gradient, leading to unstable performance. |
| Approach: | They propose a framework where the discriminator assigns rewards to samples acquired from a stationary distribution near the data rather than the generator’s distribution. |
| Outcome: | The proposed framework outperforms state-of-the-art text GANs with a more stable training process. |
Model Extrapolation Expedites Alignment (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve LLM alignment training require expensive computational resources. |
| Approach: | They propose a model extrapolation method to expedite LLMs’ alignment with human preferences by amplifying parameter changes based on a first-order approximation without any additional training overhead. |
| Outcome: | The proposed method outperforms a fully-trained model on leading benchmarks and significantly outperformed open-source models. |