Papers by Dongyan Zhao

Decouple knowledge from paramters for plug-and-play language modeling (2023.findings-acl)

Challenge:	Existing methods for video-grounded dialogue generation do not allow information from different modalities to complement each other.
Approach:	They propose a video-grounded dialogue generation model that integrates video data into pre-trained language models to allow information from different modalities to complement each other.
Outcome:	The proposed model outperforms state-of-the-art models on automatic and human evaluations on two public datasets.

Copied to clipboard

Xin Cheng, Yankai Lin, Xiuying Chen, Dongyan Zhao, Rui Yan

Cross-Lingual Question Answering over Knowledge Base as Reading Comprehension (2023.findings-eacl)

Challenge:	Pre-trained language models (PLMs) have made impressive results in a wide range of NLP tasks.
Approach:	They propose a pre-training model with editable and scalable key-value memory and leverage knowledge in an explainable manner by knowledge retrieval in the pasted macro ‘MEMORY’.
Outcome:	The proposed model decouples the knowledge storage from model parameters with an editable and scalable key-value memory and leverages knowledge in an explainable manner by knowledge retrieval in the pasted macro ‘MEMORY’.

Copied to clipboard

Chen Zhang, Yuxuan Lai, Yansong Feng, Xingyu Shen, Haowei Du, Dongyan Zhao

Graph-Structured Speculative Decoding (2024.findings-acl)

Challenge:	Existing high-quality xMRC datasets can be further utilized to fine-tune our model.
Approach:	They propose a cross-lingual question answering over knowledge base approach that converts KB subgraphs into passages to narrow the gap between KB schemas and questions.
Outcome:	The proposed approach outperforms baselines and achieves strong few-shot and zero-shot performance on two xKBQA datasets in 12 languages.

Copied to clipboard

Zhuocheng Gong, Jiahao Liu, Ziyue Wang, Pengfei Wu, Jingang Wang, Xunliang Cai, Dongyan Zhao, Rui Yan

Green & Sustainable NLP

Challenge:	Speculative decoding is a promising technique to accelerate the inference of Large Language Models.
Approach:	They propose a method that uses a token graph to record multiple sequence hypotheses within a single draft stage.
Outcome:	The proposed method significantly accelerates the inference of Large Language Models (LLMs) it allows the LLM to choose from and select the longest sequence that meets its standards.

Chain-of-Discussion: A Multi-Model Framework for Complex Evidence-Based Question Answering (2025.coling-main)

Copied to clipboard

Mingxu Tao, Dongyan Zhao, Yansong Feng

Question Answering

Challenge:	Large Language Models (LLMs) have demonstrated remarkable language generation capabilities, propelling advancements in various understanding/generation tasks, including opendomain question answering (QA).
Approach:	They propose a chain-of- Discussion framework to leverage synergy among multiple open-source Large Language Models (LLMs) aiming to provide more correct and more comprehensive answers for open-ended QA, although they are not strong enough individually.
Outcome:	The proposed framework leverages the synergy among multiple open-source Large Language Models (LLMs) to provide more correct and comprehensive answers for open-ended QA, although they are not strong enough individually.

Capturing Relations between Scientific Papers: An Abstractive Model for Related Work Section Generation (2021.acl-long)

Copied to clipboard

Xiuying Chen, Hind Alamro, Mingzhe Li, Shen Gao, Xiangliang Zhang, Dongyan Zhao, Rui Yan

Relation-Aware Question Answering for Heterogeneous Knowledge Graphs (2023.findings-emnlp)

Challenge:	Existing related work generation models are inflexible and extract sentences from multiple papers to form a related work discussion.
Approach:	They propose a Relation-aware Related work generator which generates an abstractive related work from the given multiple scientific papers in the same research area.
Outcome:	The proposed model improves over existing models and can be used to familiarize researchers with the state of the art in the field.

Copied to clipboard

Haowei Du, Quzhe Huang, Chen Li, Chen Zhang, Yang Li, Dongyan Zhao

Large Language Models Can Learn Representation in Natural Language (2024.findings-acl)

Challenge:	Existing retrieval-based approaches to solve multihop Knowledge Base Question Answering (KBQA) fail to utilize information from head-tail entities and the semantic connection between relations to enhance the information capturing of relations in KGs.
Approach:	They propose to use a dual relation graph to find the answer entity in a knowledge graph . they use primal entity graph reasoning, dual relation grafitment and interaction .
Outcome:	The proposed approach achieves significant performance gain over the prior state-of-the-art on two public datasets, WebQSP and CWQ.

Copied to clipboard

Yiduo Guo, Yaobo Liang, Dongyan Zhao, Nan Duan

Towards Context-Aware Code Comment Generation (2020.findings-emnlp)

Challenge:	Large Language Models (LLMs) are unable to complete complex tasks involving multiple entities, such as tool APIs.
Approach:	They propose a method which uses natural language representations to refine entity descriptions for improved retrieval and LLM utilization.
Outcome:	The proposed method improves GPT-4's performance on classification tasks and API call tasks.

Copied to clipboard

Xiaohan Yu, Quzhe Huang, Zheng Wang, Yansong Feng, Dongyan Zhao

Programming Languages in NLP

Challenge:	Existing methods for code comments generate comments manually, but they suffer from poor scalability and high maintenance cost due to the expensive overhead of writing comment templates.
Approach:	They propose a method to automatically generate code comments at a function level by targeting object-oriented programming languages.
Outcome:	The proposed approach outperforms the state-of-the-art methods and is comparable with existing methods.

Rethinking Task-Specific Knowledge Distillation: Contextualized Corpus as Better Textbook (2022.emnlp-main)

Copied to clipboard

Chang Liu, Chongyang Tao, Jianxin Liang, Tao Shen, Jiazhan Feng, Quzhe Huang, Dongyan Zhao

Probing Multimodal Large Language Models for Global and Local Semantic Representations (2024.lrec-main)

Challenge:	Existing methods for knowledge distillation use a two-stage paradigm: general distillation with a task-agnostic general corpus and task-specific distillation using augmented task- specific corpus.
Approach:	They propose a contextualized corpus that contextualizes task corpus with large-scale general corpus through relevance-based text retrieval to improve student learning.
Outcome:	The proposed model improves on the GLUE benchmark and shows that it is better than generalized corpus and augmented task-specific corpus.

Copied to clipboard

Mingxu Tao, Quzhe Huang, Kun Xu, Liwei Chen, Yansong Feng, Dongyan Zhao

English as Defense Proxy: Mitigating Multilingual Jailbreak via Eliciting English Safety Knowledge (2025.findings-emnlp)

Challenge:	Existing studies have focused on the ability of MLLMs to generate single tokens one by one, while lacking studies about how their representation vectors can encode global multimodal information.
Approach:	They propose to use image-caption corpus to train Multimodal Large Language Models (MLLMs) . they find that the topmost layers encode more global semantic information .
Outcome:	The proposed models can encode more global semantic information, rather than the topmost layers, and perform better on visual-language entailment tasks.

Copied to clipboard

Zekai Zhang, Yiduo Guo, Jiuheng Lin, Shanghaoran Quan, Huishuai Zhang, Dongyan Zhao

Understanding Procedural Text using Interactive Entity Networks (2020.emnlp-main)

Challenge:	Large language models excel in many tasks, but their safety guarantees vary by language.
Approach:	They propose a unified approach that leverages English as a universal safety anchor.
Outcome:	The proposed approach leverages English as defense proxy (E-Proxy) to transfer safety knowledge across languages.

Copied to clipboard

Jizhi Tang, Yansong Feng, Dongyan Zhao

Challenge:	Recent efforts to track multiple entities in a procedural text treat each entity separately . e.g., scientific articles, instruction books, recipes, often contain multiple entities involved .
Approach:	They propose a recurrent network with memory equipped cells for state tracking . they maintain different attention matrices through specific memories to model different types of entity interactions .
Outcome:	The proposed model outperforms state-of-the-art models on a benchmark dataset.

Iterative Document Representation Learning Towards Summarization with Polishing (D18-1)

Copied to clipboard

Xiuying Chen, Shen Gao, Chongyang Tao, Yan Song, Dongyan Zhao, Rui Yan

EpiCoDe: Boosting Model Performance Beyond Training with Extrapolation and Contrastive Decoding (2025.findings-acl)

Challenge:	Existing summarization methods read through document only once to generate a document representation, resulting in a sub-optimal representation.
Approach:	They propose an iterative model for supervised extractive text summarization which polishes the document representation on many passes through the document.
Outcome:	The proposed model outperforms state-of-the-art extractive systems on CNN/DailyMail and DUC2002 datasets.

Copied to clipboard

Mingxu Tao, Jie Hu, Mingchuan Yang, Yunhuai Liu, Dongyan Zhao, Yansong Feng

Align-then-Enhance: Multilingual Entailment Graph Enhancement with Soft Predicate Alignment (2023.findings-acl)

Challenge:	Existing methods to enhance performance of Large language models are limited due to the cost of training data and privacy concerns.
Approach:	They propose a method that enhances a finetuned model with its inferior version and adopts contrastive decoding to reduce predicted errors.
Outcome:	The proposed method outperforms existing methods in data-scarcity scenarios across three domains and shows that it is more robust and robust.

Copied to clipboard

Yuting Wu, Yutong Hu, Yansong Feng, Tianyi Li, Mark Steedman, Dongyan Zhao

Unlocking the Potential of Model Merging for Low-Resource Languages (2024.findings-emnlp)

Challenge:	Existing approaches to learn typed entailment graphs with predicates as nodes and enttailment relations as edges are incomplete.
Approach:	They propose a task to utilize entailment information from one EG to enhance another in a different language.
Outcome:	The proposed framework outperforms existing graphs in multilingual entailment graph enhancement tasks.

Copied to clipboard

Mingxu Tao, Chen Zhang, Quzhe Huang, Tianyao Ma, Songfang Huang, Dongyan Zhao, Yansong Feng

More than Classification: A Unified Framework for Event Temporal Relation Extraction (2023.acl-long)

Challenge:	Adapting large language models (LLMs) to new languages requires continual pre-training followed by supervised fine-tuning.
Approach:	They propose a model merging solution that integrates LLMs with distinct capabilities into a single model without additional training.
Outcome:	The proposed model merging outperforms CT-then-SFT in low-resource languages with scarce data.

Copied to clipboard

Quzhe Huang, Yutong Hu, Shengqi Zhu, Yansong Feng, Chang Liu, Dongyan Zhao

Language Models can be Deductive Solvers (2024.findings-naacl)

Challenge:	Existing methods for event temporal relation extraction ignore meaning of relations and wipe out their intrinsic dependency.
Approach:	They propose a unified event temporal relation extraction framework that transforms temporal relations into logical expressions of time points and completes the ETRE by predicting the relations between certain time points.
Outcome:	The proposed framework outperforms the state-of-the-art model on TB-Dense and MATRES by 0.3% on both datasets.

Copied to clipboard

Jiazhan Feng, Ruochen Xu, Junheng Hao, Hiteshi Sharma, Yelong Shen, Dongyan Zhao, Weizhu Chen

One Time of Interaction May Not Be Enough: Go Deep with an Interaction-over-Interaction Network for Response Selection in Dialogues (P19-1)

Challenge:	Recent advances have enabled Large Language Models (LLMs) to potentially exhibit reasoning capabilities, but complex logical reasoning remains a challenge.
Approach:	They propose a novel language model that internalizes and emulates the reasoning processes of logical solvers and avoids parsing errors by learning strict adherence to solver syntax and grammar.
Outcome:	The proposed model outperforms state-of-the-art solver-augmented language models and few-shot prompting methods on public deductive reasoning benchmarks.

Copied to clipboard

Chongyang Tao, Wei Wu, Can Xu, Wenpeng Hu, Dongyan Zhao, Rui Yan

Who Is Speaking to Whom? Learning to Identify Utterance Addressee in Multi-Party Conversations (D19-1)

Challenge:	Currently, retrieval-based dialogues are performed in shallow ways . a recent study investigated the problem of context-response matching in open-domain .
Approach:	They propose a model that lets utterance-response interaction go deep by stacking interaction blocks.
Outcome:	The proposed model outperforms state-of-the-art methods on three benchmark data sets.

Copied to clipboard

Ran Le, Wenpeng Hu, Mingyue Shang, Zhenjun You, Lidong Bing, Dongyan Zhao, Rui Yan

Improving Input-label Mapping with Demonstration Replay for In-context Learning (2023.findings-emnlp)

Challenge:	In multi-party conversations, addressee information is not always explicit . researchers have spent great efforts to understand conversations between two participants, which is known as multi-part conversation.
Approach:	They propose a who-to-whom model which models users and utterances in a conversation session jointly in an interactive way.
Outcome:	The proposed model outperforms baseline models on the Ubuntu Multi-Party Conversation Corpus and shows consistent improvements.

Copied to clipboard

Zhuocheng Gong, Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai, Dongyan Zhao, Rui Yan

How Many Answers Should I Give? An Empirical Study of Multi-Answer Reading Comprehension (2023.findings-acl)

Challenge:	In-context learning (ICL) is an emerging capability of large autoregressive language models where a few demonstrations are appended to the input to enhance the model’s understanding of downstream NLP tasks without directly adjusting the model parameters.
Approach:	They propose a method where a few demonstrations are appended to the input to enhance the model's understanding of downstream NLP tasks without directly adjusting the model parameters.
Outcome:	The proposed method significantly improves the input-label mapping in ICL demonstrations.

Copied to clipboard

Chen Zhang, Jiuheng Lin, Xiao Liu, Yuxuan Lai, Yansong Feng, Dongyan Zhao

Reasoning

Challenge:	Despite recent progress in multi-answer MRC, there is no systematic analysis of how this phenomenon arises and how to better address it.
Approach:	They develop a taxonomy to categorize commonly-seen multi-answer MRC instances and examine how well different paradigms deal with different types of multi-announced questions.
Outcome:	The proposed taxonomy categorizes commonly-seen multi-answer instances and analyzes how well different paradigms deal with different types of multi-announced instances.

Selection and Generation: Learning towards Multi-Product Advertisement Post Generation (2020.emnlp-main)

Copied to clipboard

Zhangming Chan, Yuchi Zhang, Xiuying Chen, Shen Gao, Zhiqiang Zhang, Dongyan Zhao, Rui Yan

Mixture-of-Modules: Reinventing Transformers as Dynamic Assemblies of Modules (2024.emnlp-main)

Challenge:	E-commerce websites have billions of products, so it is impossible to write all copywriting manually.
Approach:	They propose a model to generate an AD post using a select network and a MGenNet network to generate a post including selected products.
Outcome:	The proposed model achieves impressive performance on a large-scale real-world AD post dataset.

Copied to clipboard

Zhuocheng Gong, Ang Lv, Jian Guan, Wei Wu, Huishuai Zhang, Minlie Huang, Dongyan Zhao, Rui Yan

Learning to Plan by Updating Natural Language (2024.findings-emnlp)

Challenge:	Empirical results show that MoMs consistently outperform vanilla transformers .
Approach:	They propose an architecture that allows for a mixture-of-modules computation that uses a finite set of modules defined by multi-head attention and feed-forward networks.
Outcome:	The proposed architecture outperforms vanilla Transformers and their variants in multiple ways.

Copied to clipboard

Yiduo Guo, Yaobo Liang, Chenfei Wu, Wenshan Wu, Dongyan Zhao, Nan Duan

Analyzing and Reducing the Performance Gap in Cross-Lingual Transfer with Fine-tuning Slow and Fast (2023.acl-long)

Challenge:	Large Language Models (LLMs) have shown remarkable performance in basic natural language tasks.
Approach:	They propose a method that iteratively updates the task plan with new steps and behavioral instructions to guide LLMs to generate the correct solutions step by step.
Outcome:	The proposed method improves performance on five reasoning type tasks and can be directly applied to other LLMs.

Copied to clipboard

Yiduo Guo, Yaobo Liang, Dongyan Zhao, Bing Liu, Nan Duan

BioGen: Generating Biography Summary under Table Guidance on Wikipedia (2021.findings-acl)

Challenge:	Existing research shows that a multilingual pre-trained language model fine-tuned with one (source) language performs well on downstream tasks for non-source languages . However, there is a clear performance gap between the source and non-sourced languages - this gap can be reduced by reducing forgetting.
Approach:	They propose a method to fine-tune a multilingual pre-trained language model fine- tuned with one (source) language and four training policies to address the performance gap.
Outcome:	The proposed method outperforms baselines on the XNLI dataset by a clear margin.

Copied to clipboard

Shen Gao, Xiuying Chen, Chang Liu, Dongyan Zhao, Rui Yan

PPTC-R benchmark: Towards Evaluating the Robustness of Large Language Models for PowerPoint Task Completion (2024.findings-emnlp)

Challenge:	Existing methods for summarizing text have not captured the salient information from an article.
Approach:	They propose a table-guided abstractive biography summarization that utilizes factual tables to capture important information and generate a summary of a biography.
Outcome:	The proposed method is the first large-scale biography summarization dataset with tables.

Copied to clipboard

Zekai Zhang, Yiduo Guo, Yaobo Liang, Dongyan Zhao, Nan Duan

Robustness in NLP

Challenge:	Large Language Models (LLMs) are increasingly used for task completion in real-world situations.
Approach:	They propose a PowerPoint Task Completion-Robustness (PPTC-R) benchmark to measure LLMs’ robustness to the user PPT task instruction and software version (Powerpoint).
Outcome:	The proposed benchmark compares 3 closed-source and 4 open-source LLMs to the PowerPoint task instruction and software version (Powerpoint) .

Learning a Matching Model with Co-teaching for Multi-turn Response Selection in Retrieval-based Dialogue Systems (P19-1)

Copied to clipboard

Jiazhan Feng, Chongyang Tao, Wei Wu, Yansong Feng, Dongyan Zhao, Rui Yan

MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation (2023.acl-long)

Challenge:	Existing methods for learning a robust matching model from noisy training data are retrieval-based or generation-based.
Approach:	They propose a general co-teaching framework that learns matching models from noisy training data.
Outcome:	The proposed learning framework can improve existing models on two public data sets.

Copied to clipboard

Jiazhan Feng, Qingfeng Sun, Can Xu, Pu Zhao, Yaming Yang, Chongyang Tao, Dongyan Zhao, Qingwei Lin

Multi-Granularity Information Interaction Framework for Incomplete Utterance Rewriting (2023.findings-emnlp)

Challenge:	MMDialog is a dataset of 1.08 million real-world dialogues with 1.53 million unique images across 4,184 topics.
Approach:	They propose to use a curated set of 1.08 million dialogues with 1.53 million unique images to generalize the open domain.
Outcome:	The proposed system can predict responses to multi-modal content with state-of-the-art techniques and measure their performance.

Copied to clipboard

Haowei Du, Dinghao Zhang, Chen Li, Yang Li, Dongyan Zhao

Paraphrasing

Challenge:	Recent approaches in Incomplete Utterance Rewriting (IUR) fail to capture the source of important words, introducing words from irrelevant utterances.
Approach:	They propose a framework to capture the multi-granularity of semantic information and fetch the relevant utterance.
Outcome:	The proposed framework outperforms state-of-the-art models on two benchmark datasets . it can capture the source of important words and fetch the relevant utterance .

Learning to Update Knowledge Graphs by Reading News (D19-1)

Copied to clipboard

Jizhi Tang, Yansong Feng, Dongyan Zhao

Three Sentences Are All You Need: Local Path Enhanced Document Relation Extraction (2021.acl-short)

Challenge:	Existing methods to update knowledge graphs rely on elaborately designed IE systems and domain-specific rules.
Approach:	They propose a novel neural network method to update knowledge graphs (KGs) they use a text-based attention mechanism to guide updating messages through KGs .
Outcome:	The proposed method can effectively broadcast news information to KG structures and perform necessary link-adding or link-deleting operations to ensure the KG up-to-date according to news snippets.

Copied to clipboard

Quzhe Huang, Shengqi Zhu, Yansong Feng, Yuan Ye, Yuxuan Lai, Dongyan Zhao

Reciprocal Learning of Knowledge Retriever and Response Ranker for Knowledge-Grounded Conversations (2022.coling-1)

Challenge:	Document-level relation extraction (RE) is more challenging than sentence RE as it often requires reasoning over multiple sentences.
Approach:	They propose a method to heuristically select evidence sentences for document-level relation extraction.
Outcome:	The proposed method can be easily combined with BiLSTM to achieve good performance on benchmark datasets even better than fancy graph neural network based methods.

Copied to clipboard

Jiazhan Feng, Chongyang Tao, Zhen Li, Chang Liu, Tao Shen, Dongyan Zhao

Information Retrieval

Challenge:	Recent work on grounding dialogue agents with knowledge documents has sparked increased attention . hand-labeling data to that end is time-consuming and many datasets lack knowledge annotations .
Approach:	They propose a reciprocal learning approach to optimize a knowledge retriever and a response ranker for knowledge-grounded response retrieval without ground-truth knowledge labels.
Outcome:	The proposed model outperforms previous state-of-the-art methods on two public benchmarks.

Synergistic Interplay between Search and Large Language Models for Information Retrieval (2024.acl-long)

Copied to clipboard

Jiazhan Feng, Chongyang Tao, Xiubo Geng, Tao Shen, Can Xu, Guodong Long, Dongyan Zhao, Daxin Jiang

Multilingual Generation in Abstractive Summarization: A Comparative Study (2024.lrec-main)

Challenge:	Information retrieval (IR) is an indispensable technique for locating relevant resources from vast amounts of data.
Approach:	They propose a framework that facilitates information refinement through synergy between RMs and LLMs.
Outcome:	The proposed framework improves the performance of large-scale retrieval benchmarks on web searches and low-resource retrieval tasks.

Copied to clipboard

Jinpeng Li, Jiaze Chen, Huadong Chen, Dongyan Zhao, Rui Yan

Summarizing Procedural Text: Data and Approach (2022.findings-emnlp)

Challenge:	Existing models for multilingual generation lack thorough analysis due to extensive linguistic diversity.
Approach:	They propose to classify multilingual generation methodologies into three categories based on their underlying modeling principles . they introduce an automatic metric to mitigate spurious correlations associated with language mixing .
Outcome:	The proposed model improves in high-resource, low-resourced, and zero-shot scenarios.

Copied to clipboard

Shen Gao, Haotong Zhang, Xiuying Chen, Rui Yan, Dongyan Zhao

GNN-encoder: Learning a Dual-encoder Architecture via Graph Neural Networks for Dense Passage Retrieval (2022.findings-emnlp)

Challenge:	Procedural text summarization task is a popular task in the NLP field because of its long length and complexity.
Approach:	They propose a procedural text summarization task with two granularity . they propose an Entity-State Graph-based Summarizer (ESGS) which aggregates contextual information for each procedure.
Outcome:	The proposed model can summarize the entire procedural text or give an overview for each step or both . Experiments on two datasets confirm the proposed model's effectiveness.

Copied to clipboard

Jiduan Liu, Jiahao Liu, Yang Yang, Jingang Wang, Wei Wu, Dongyan Zhao, Rui Yan

Information Retrieval

Challenge:	Existing approaches to perform large-scale query-passage retrieval are term-based, but they lose interaction between query-pastage pairs.
Approach:	They propose to fuse query (passage) information into query representations via graph neural networks that are constructed by queries and their top retrieved passages.
Outcome:	The proposed model outperforms existing models on MSMARCO, Natural Questions and TriviaQA datasets and achieves the new state-of-the-art on these datasets.

Multi-grained Attention Network for Aspect-Level Sentiment Classification (D18-1)

Copied to clipboard

Feifan Fan, Yansong Feng, Dongyan Zhao

Combining Curriculum Learning and Knowledge Distillation for Dialogue Generation (2021.findings-emnlp)

Challenge:	Existing approaches to aspect sentiment classification use coarse-grained attention mechanisms . a novel approach captures word-level interaction between aspect and context .
Approach:	They propose a novel multi-grained attention network model for aspect level sentiment classification . they use a fine-grounded attention mechanism to capture word-level interaction between aspect and context .
Outcome:	The proposed model outperforms the state-of-the-art methods on three datasets . it shows that aspect-level interactions can bring extra useful information and improve performance .

Copied to clipboard

Qingqing Zhu, Xiuying Chen, Pengfei Wu, JunFei Liu, Dongyan Zhao

ReMamba: Equip Mamba with Effective Long-Sequence Modeling (2025.findings-emnlp)

Challenge:	Existing studies have shown that curriculum learning facilitates dialogue generation tasks while knowledge distillation can yield significant performance boosts for student models.
Approach:	They propose a combination of curriculum learning and knowledge distillation for dialogue generation models . they cluster training cases according to their complexity and employ an adversarial training strategy .
Outcome:	The proposed model improves compared with baselines.

Copied to clipboard

Danlong Yuan, Jiahao Liu, Bei Li, Huishuai Zhang, Jingang Wang, Xunliang Cai, Dongyan Zhao

Learning to Organize a Bag of Words into Sentences with Neural Networks: An Empirical Study (2021.naacl-main)

Challenge:	Mamba models demonstrate superior inference efficiency and competitive performance on short-context tasks, but their capacity to comprehend long contexts is limited compared to transformer-based models.
Approach:	They propose a model which incorporates selective compression and adaptation techniques within a two-stage re-forward process, incurring minimal additional inference costs overhead.
Outcome:	The proposed model improves on the LongBench and L-Eval benchmarks by 3.2 and 1.6 points and attains performance almost on par with same-size transformer models.

Copied to clipboard

Chongyang Tao, Shen Gao, Juntao Li, Yansong Feng, Dongyan Zhao, Rui Yan

Challenge:	Existing approaches to encode natural languages without orders are lacking.
Approach:	They conduct a comprehensive analysis of the ability of neural models to organize sentences from a bag of words under three typical scenarios.
Outcome:	The proposed models can reorder or reconstruct sentences from a bag of words under three typical scenarios.

Marrying Up Regular Expressions with Neural Networks: A Case Study for Spoken Language Understanding (P18-1)

Copied to clipboard

Bingfeng Luo, Yansong Feng, Zheng Wang, Songfang Huang, Rui Yan, Dongyan Zhao

Challenge:	Experimental results show that the combination of regular expressions and NNs improves learning effectiveness when a small number of training examples are available.
Approach:	They propose to combine a neural network (NN) with regular expressions (RE) to improve supervised learning for NLP by exploiting the rich expressiveness of REs at different levels within a NN.
Outcome:	The proposed approach significantly improves learning effectiveness when a small number of training examples are available.

Sampling Matters! An Empirical Study of Negative Sampling Strategies for Learning of Matching Models in Retrieval-based Dialogue Systems (D19-1)

Copied to clipboard

Jia Li, Chongyang Tao, Wei Wu, Yansong Feng, Dongyan Zhao, Rui Yan

Finding the Dominant Winning Ticket in Pre-Trained Language Models (2022.findings-acl)

Challenge:	Existing studies focus on constructing a matching model with sophisticated neural architectures, but do little to how to effectively learn such architectures from data.
Approach:	They propose to sample negative examples to automatically construct a training set for effective model learning in retrieval-based dialogue systems by using four sampling strategies.
Outcome:	The proposed learning method improves the performance of matching models on two benchmarks with three matching models.

Copied to clipboard

Zhuocheng Gong, Di He, Yelong Shen, Tie-Yan Liu, Weizhu Chen, Dongyan Zhao, Ji-Rong Wen, Rui Yan

ReasVQA: Advancing VideoQA with Imperfect Reasoning Process (2025.naacl-long)

Challenge:	Existing studies on pre-trained language models show that they can fine-tune parameters but achieve good downstream performance.
Approach:	They find that a dominant winning ticket takes up 0.05% of the parameters and is transferable across different tasks.
Outcome:	The proposed model can achieve comparable performance with the full-parameter model, the authors show . the dominant winning ticket takes up 0.05% of the parameters, and the model is transferable across tasks, they show - the authors conclude .

Copied to clipboard

Jianxin Liang, Xiaojun Meng, Huishuai Zhang, Yueqian Wang, Jiansheng Wei, Dongyan Zhao

Visual Data in NLP

Challenge:	Existing approaches to VideoQA often fail when complex reasoning or temporal relationships are involved.
Approach:	They propose a method that leverages reasoning processes generated by Multimodal Large Language Models to improve VideoQA models.
Outcome:	The proposed method improves VideoQA models on three benchmarks.

Structure-Discourse Hierarchical Graph for Conditional Question Answering on Long Documents (2023.findings-acl)

Copied to clipboard

Haowei Du, Yansong Feng, Chen Li, Yang Li, Yunshi Lan, Dongyan Zhao

Multi-Granularity Structural Knowledge Distillation for Language Model Compression (2022.acl-long)

Challenge:	Existing approaches to conditional question answering on long documents ignore document structure and discourse relations between sentences in document sections.
Approach:	They construct a Structure-Discourse Hierarchical Graph and conduct bottom-up information propagation to address this issue.
Outcome:	The proposed approach outperforms the existing methods on the conditional question answering on long documents by 3.0 EM score and 2.4 F1 score on answer measuring, and 2.2 EM and 1.9 F1 scores on jointly answer and condition measuring.

Copied to clipboard

Chang Liu, Chongyang Tao, Jiazhan Feng, Dongyan Zhao

Leveraging Denoised Abstract Meaning Representation for Grammatical Error Correction (2023.findings-acl)

Challenge:	Existing methods to transfer knowledge to a small model are not enough to represent the rich semantics of a text.
Approach:	They propose to distill the knowledge to a student hierarchically across layers using a large teacher-student framework.
Outcome:	Experimental results show that the proposed method outperforms distillation methods on GLUE benchmark.

Copied to clipboard

Hejing Cao, Dongyan Zhao

Towards Efficient Dialogue Pre-training with Transferable and Interpretable Latent Structure (2022.emnlp-main)

Challenge:	Popular GEC models use large-scale synthetic corpora or use a large number of human-designed rules.
Approach:	They propose a model that incorporates denoised AMR as additional knowledge to get AMRs more reliable.
Outcome:	The proposed model reduces training time by 32% while inference time is comparable.

Copied to clipboard

Xueliang Zhao, Lemao Liu, Tingchen Fu, Shuming Shi, Dongyan Zhao, Rui Yan

Knowledge-Grounded Dialogue Generation with Pre-trained Language Models (2020.emnlp-main)

Challenge:	Existing models that use millions of parameters on massive data are inefficient and lack interpretability.
Approach:	They propose a model with a latent structure that is easily transferable from the general domain to downstream tasks in a lightweight and transparent way.
Outcome:	The proposed model performs better than four strong baseline models in terms of automatic and human evaluations and is 5x faster than the strongest baseline model.

Copied to clipboard

Xueliang Zhao, Wei Wu, Can Xu, Chongyang Tao, Dongyan Zhao, Rui Yan

Dialogue Summarization with Static-Dynamic Structure Fusion Graph (2023.acl-long)

Challenge:	Empirical results indicate that pre-trained language models can significantly outperform state-of-the-art methods in both automatic evaluation and human judgment.
Approach:	They propose to equip a pre-trained language model with a knowledge selection module to generate knowledge-grounded dialogues.
Outcome:	The proposed model outperforms state-of-the-art methods in evaluation and human judgment.

Copied to clipboard

Shen Gao, Xin Cheng, Mingzhe Li, Xiuying Chen, Jinpeng Li, Dongyan Zhao, Rui Yan

Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis (2021.naacl-main)

Challenge:	Dialogue summarization is a challenging task since it has dynamic interaction nature and inconsistent information flow among various speakers.
Approach:	They propose a Static-Dynamic graph-based Dialogue Summarization model which fuses prior knowledge from human expertise and adaptively learns the graph structure in an end-to-end learning fashion.
Outcome:	The proposed model can help people capture the highlights of a semi-structured and multi-participant dialogue without reviewing the complex dialogue context.

Copied to clipboard

Xiao Liu, Da Yin, Yansong Feng, Yuting Wu, Dongyan Zhao

Counterfactual Recipe Generation: Exploring Compositional Generalization in a Realistic Scenario (2022.emnlp-main)

Challenge:	Existing studies focus on analyzing structured data, while mining causal relationship among factors from unstructured data is of great importance.
Approach:	They propose a graph-based causal inference framework which builds causal graphs from fact descriptions without much human involvement.
Outcome:	The proposed framework can capture nuance from fact descriptions among confusing charges and provide explainable discrimination in few-shot settings.

Copied to clipboard

Xiao Liu, Yansong Feng, Jizhi Tang, Chengang Hu, Dongyan Zhao

The Magic of IF: Investigating Causal Reasoning Abilities in Large Language Models of Code (2023.findings-acl)

Challenge:	Existing models fail to learn and use culinary knowledge in a compositional way, argues a new study.
Approach:	They propose a task that asks models to modify a base recipe according to the change of an ingredient.
Outcome:	The proposed model can perform compositional generalization in a realistic setting . existing models have difficulties in modifying ingredients while preserving original style .

Copied to clipboard

Xiao Liu, Da Yin, Chen Zhang, Yansong Feng, Dongyan Zhao

Stylized Dialogue Generation with Feature-Guided Knowledge Augmentation (2023.findings-emnlp)

Challenge:	entailment a)
Approach:	entailment : We want to explore whether Code-LLMs with code prompts are better . encoding a code prompt is better than text-only LLMs, they say .
Outcome:	entailment : Our results show that Code-LLMs with code prompts are better compared to text-only LLMs.

Copied to clipboard

Jinpeng Li, Zekai Zhang, Xiuying Chen, Dongyan Zhao, Rui Yan

Length-Adaptive Distillation: Customizing Small Language Model for Dynamic Token Pruning (2023.findings-emnlp)

Challenge:	Existing methods synthesize pseudo data through back translation but lack guidance on target style features.
Approach:	They propose a knowledge-augmented stylized dialogue generation model with a feature-guided style knowledge selection module that utilizes context and response features.
Outcome:	The proposed model produces a satisfactory performance on two public benchmarks on both semantic and stylized levels.

Copied to clipboard

Chang Liu, Chongyang Tao, Jianxin Liang, Jiazhan Feng, Tao Shen, Quzhe Huang, Dongyan Zhao

An Intra-Class Relation Guided Approach for Code Comment Generation (2023.findings-eacl)

Challenge:	Existing methods to accelerate inference speed are model compression and dynamic computation (e.g., dynamic token pruning).
Approach:	They propose a two-stage knowledge distillation framework that produces a customized small language model for dynamic token pruning.
Outcome:	The proposed framework can make the small language model more customized for dynamic token pruning and achieve better speed-performance trade-off.

Copied to clipboard

Zhenni Wang, Xiaohan Yu, Yansong Feng, Dongyan Zhao

Generating Classical Chinese Poems via Conditional Variational Autoencoder and Adversarial Training (D18-1)

Challenge:	Recent work in code comment generation assumes that all information required to generate comments is encoded in the target function itself, yet in most realistic situations, it is hard to understand a function in isolation from the surrounding context.
Approach:	They propose a graph-based learning framework to capture various relations among functions in a class file.
Outcome:	The proposed method outperforms baseline models on automatic and human evaluation metrics on a Java dataset collected from real-world projects.

Copied to clipboard

Juntao Li, Yan Song, Haisong Zhang, Dongmin Chen, Shuming Shi, Dongyan Zhao, Rui Yan

Robustness in NLP

Challenge:	Existing models for automatic poetry generation lack term novelty and thematic consistency.
Approach:	They propose a conditional variational autoencoder with adversarial training for classical Chinese poem generation.
Outcome:	The proposed model outperforms existing models on a large poetry corpus on 'classical Chinese' . it generates poems with novel terms and learns their thematic consistency with their titles.

Transformation of Dense and Sparse Text Representations (2020.coling-main)

Copied to clipboard

Wenpeng Hu, Mengyu Wang, Bing Liu, Feng Ji, Jinwen Ma, Dongyan Zhao

How to Represent Context Better? An Empirical Study on Context Modeling for Multi-turn Response Selection (2022.findings-emnlp)

Challenge:	Existing approaches to NLP to leverage sparsity have been limited due to the gap with dense representations.
Approach:	They propose a Semantic Transformation method to bridge dense and sparse spaces and propose supervised NLP tasks to use both spaces.
Outcome:	Experiments with classification tasks and natural language inference tasks show that the proposed method is effective.

Copied to clipboard

Jiazhan Feng, Chongyang Tao, Chang Liu, Rui Yan, Dongyan Zhao

VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Format (2025.findings-emnlp)

Challenge:	Existing work on building a conversational system for open domain human-machine conversation is attracting more attention . early models concatenate all utterances or independently encode each dialogue turn, which may lead to an inadequate understanding of dialogue status.
Approach:	They propose to use a turn-aware context modeling layer to adapt existing models . they propose to model multi-turn contexts from the perspective of sequential relationship, local relationship, and query-alike manner .
Outcome:	The proposed method can be adapted to several advanced response selection models.

Copied to clipboard

Yueqian Wang, Xiaojun Meng, Yuxuan Wang, Jianxin Liang, Jiansheng Wei, Huishuai Zhang, Dongyan Zhao

Visual Data in NLP

Challenge:	Recent studies on video large language models focus on model architectures and training datasets . interaction format between user and model is unsatisfactory for time-sensitive tasks .
Approach:	They propose a video-text duet interaction format that allows for continuous playback of the video . when a text message ends, the video continues to play, similar to the alternative of two performers in a duet.
Outcome:	The proposed format improves performance on time-sensitive tasks with minimal training efforts.

E-Bench: Towards Evaluating the Ease-of-Use of Large Language Models (2025.coling-main)

Copied to clipboard

Zhenyu Zhang, Bingguang Hao, Jinpeng Li, Zekai Zhang, Dongyan Zhao

Translation vs. Dialogue: A Comparative Analysis of Sequence-to-Sequence Modeling (2020.coling-main)

Challenge:	E-Bench is a framework for easy-to-use research on large language models.
Approach:	They propose to evaluate the ease-of-use of large language models and construct an E-Bench . they simulate human use from synonymous and typographical perturbations .
Outcome:	The proposed model is able to resist synonymous expressions and typos and improves performance.

Copied to clipboard

Wenpeng Hu, Ran Le, Bing Liu, Jinwen Ma, Dongyan Zhao, Rui Yan

Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation (2022.acl-long)

Challenge:	Existing models for machine translation and dialogue response generation require a large number of handcrafted features.
Approach:	They propose to interpret a general neural model comparatively by using the seq2seq model in two mainstream NLP tasks.
Outcome:	The proposed model is used in two mainstream NLP tasks and is compared with a standard model.

Copied to clipboard

Mingzhe Li, XieXiong Lin, Xiuying Chen, Jinxiong Chang, Qishen Zhang, Feng Wang, Taifeng Wang, Zhongyi Liu, Wei Chu, Dongyan Zhao, Rui Yan

FAA: Fine-grained Attention Alignment for Cascade Document Ranking (2023.acl-long)

Challenge:	Existing studies focus on contrastive learning on the instance level without discriminating the contribution of each word.
Approach:	They propose a hierarchical contrastive learning mechanism which can unify semantic meaning in the input text.
Outcome:	The proposed model outperforms baselines on storytelling, paraphrasing, dialogue generation, and storytelling tasks.

Copied to clipboard

Zhen Li, Chongyang Tao, Jiazhan Feng, Tao Shen, Dongyan Zhao, Xiubo Geng, Daxin Jiang

Information Retrieval

Challenge:	Contemporary document ranking methods focus on transforming documents into passages to handle long inputs, but intensive query-irrelevant content may lead to harmful distraction and high query latency.
Approach:	They propose a fine-grained attention alignment approach to jointly optimize a cascade document ranking model.
Outcome:	Experiments on MS MARCO and TREC DL show that the proposed method is effective in document ranking tasks.

Modeling discourse cohesion for discourse parsing via memory network (P18-2)

Copied to clipboard

Yanyan Jia, Yuan Ye, Yansong Feng, Yuxuan Lai, Rui Yan, Dongyan Zhao

Do Charge Prediction Models Learn Legal Theory? (2022.findings-emnlp)

Challenge:	Existing approaches to discourse parsing focus on studying the semantic and syntactic aspects of EDU pairs, but they do not address long span dependencies.
Approach:	They propose a new transition-based discourse parser that takes discourse cohesion into account by using memory networks.
Outcome:	The proposed method outperforms traditional features and improves performance on the RST discourse treebank.

Copied to clipboard

Zhenwei An, Quzhe Huang, Cong Jiang, Yansong Feng, Dongyan Zhao

Linguistic Theories

Challenge:	Existing models for charge prediction are sensitive, selective, and presumption of innocence . a recent study has shown that deep learning models can predict the charges accurately, but their reliability and interpretability are still underexplored.
Approach:	They propose that trustworthy charge prediction models should take legal theories into consideration . they propose three principles for trustworthy models to follow in this task .
Outcome:	The proposed framework evaluates whether existing models learn legal theories . it shows that models meet selective and presumption of innocence principles .

Why Machine Reading Comprehension Models Learn Shortcuts? (2021.findings-acl)

Copied to clipboard

Yuxuan Lai, Chen Zhang, Yansong Feng, Quzhe Huang, Dongyan Zhao

Machine Reading Comprehension

Challenge:	Existing studies show that many MRC models learn shortcuts to outwit benchmarks, but the performance is unsatisfactory in real-world applications.
Approach:	They propose to use shortcut questions to analyze learning difficulty of MRC models . they propose to analyze the learning difficulty regarding shortcut and challenging questions .
Outcome:	The proposed methods show that a large proportion of shortcut questions in training data make models rely on shortcut tricks excessively.

Neighborhood Matching Network for Entity Alignment (2020.acl-main)

Copied to clipboard

Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, Dongyan Zhao

SMASH: Improving SMAll Language Models’ Few-SHot Ability with Prompt-Based Distillation (2022.findings-emnlp)

Challenge:	Structural heterogeneity between knowledge graphs is an outstanding challenge for entity alignment.
Approach:	They propose a framework for entity alignment that uses a neighborhood matching module to combine neighborhood differences.
Outcome:	The proposed framework outperforms existing methods on three datasets.

Copied to clipboard

Yueqian Wang, Chang Liu, Kai Chen, Xi Wang, Dongyan Zhao

Smart Word Suggestions for Writing Assistance (2023.findings-acl)

Challenge:	Large-scale language models with prompts have shown remarkable performance on few-shot learning.
Approach:	They propose an approach to improve SMAll language models’ few-SHot ability by training on intermediate tasks before prompt-based fine-tuning on downstream tasks.
Outcome:	The proposed model improves on sentence-pair and sentiment classification tasks by training on intermediate tasks before fine-tuning on downstream tasks.

Copied to clipboard

Chenshuo Wang, Shaoguang Mao, Tao Ge, Wenshan Wu, Xun Wang, Yan Xia, Jonathan Tien, Dongyan Zhao

Challenge:	Using word suggestions, writing assistance is a widely used application of natural language processing (NLP) . a task is performed to identify words or phrases that require improvement and provide substitution suggestions for each improvable target.
Approach:	They propose a task and benchmark to help writers improve word usage . they use human-labeled data and a distantly supervised dataset for testing .
Outcome:	The proposed task and benchmark aims to improve word usage in writing aids.

QAEval: Mixture of Evaluators for Question-Answering Task Evaluation (2025.acl-long)

Copied to clipboard

Tan Yue, Rui Mao, Xuzhao Shi, Shuo Zhan, Zuhao Yang, Dongyan Zhao

Modeling Personalization in Continuous Space for Response Generation via Augmented Wasserstein Autoencoders (D19-1)

Challenge:	Existing QA evaluation methods struggle with open-ended and unstructured responses.
Approach:	They propose a hybrid framework that combines rule-based reliability with LLM-based adaptability to overcome these challenges.
Outcome:	The proposed framework outperforms existing models like GPT-4o and Claude-3 in accuracy and cost.

Copied to clipboard

Zhangming Chan, Juntao Li, Xiaopeng Yang, Xiuying Chen, Wenpeng Hu, Dongyan Zhao, Rui Yan

Extract, Integrate, Compete: Towards Verification Style Reading Comprehension (2021.findings-emnlp)

Challenge:	Existing work on variableal autoencoders and waterstein autoencoding models has shown significant progress in open-domain response generation.
Approach:	They propose to embed user-level and utterance-level information into two multimodal distributions and combine them into a mixed distribution.
Outcome:	The proposed model outperforms state-of-the-art models on a large-scale real-world dataset.

Copied to clipboard

Chen Zhang, Yuxuan Lai, Yansong Feng, Dongyan Zhao

Reasoning

Challenge:	VGaokao is a verification style reading comprehension dataset for Chinese language tests requiring advanced language understanding skills.
Approach:	They propose a new extract-integration-compete approach to extract complementary evidence from Chinese Language tests of Gaokao and a pairwise competition to push models to learn the subtle difference between similar text pieces.
Outcome:	The proposed approach outperforms baselines on VGaokao with retrieved complementary evidence while having the merits of efficiency and explainability.

Exploring Distantly-Labeled Rationales in Neural Network Models (2021.acl-long)

Copied to clipboard

Quzhe Huang, Shengqi Zhu, Yansong Feng, Dongyan Zhao

Text Classification

Challenge:	Existing methods focus on distantly-labeled rationales, ignoring the potential important non-rationale words and not distinguishing the importance of different rationale words.
Approach:	They propose two novel auxiliary loss functions to make better use of distantly-labeled rationales, which encourage models to maintain their focus on important words beyond labeled rationals (PINs) and alleviate redundant training on non-helpful rationale (NoIRs).
Outcome:	The proposed methods outperform existing methods on two representative classification tasks while maintaining the ability to spread focus to other unlabeled important words.

RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank (2023.acl-long)

Copied to clipboard

Jiduan Liu, Jiahao Liu, Qifan Wang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Kai Chen, Rui Yan

Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression (2023.findings-emnlp)

Challenge:	Unsupervised sentence representation learning is one of the fundamental problems in natural language processing . contrastive learning methods fail to capture fine-grained ranking information among the sentences .
Approach:	They propose a novel approach for unsupervised sentence representation learning that integrates ranking consistency and ranking distillation with contrastive learning into a unified framework.
Outcome:	The proposed approach performs better over state-of-the-art models on STS and TR tasks.

Copied to clipboard

Jiduan Liu, Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai, Dongyan Zhao, Ran Wang, Rui Yan

Efficient Temporal Extrapolation of Multimodal Large Language Models with Temporal Grounding Bridge (2024.emnlp-main)

Challenge:	Large-scale pre-trained language models (LLMs) have demonstrated exceptional performance in various natural language processing (NLP) tasks.
Approach:	They propose a new compression paradigm that extracts knowledge from pre-trained language models to construct a knowledge store from which the model can leverage it for effective inference.
Outcome:	The proposed model extracts knowledge from LLMs to construct a knowledge store, which the model can leverage for effective inference.

Copied to clipboard

Yuxuan Wang, Yueqian Wang, Pengfei Wu, Jianxin Liang, Dongyan Zhao, Yang Liu, Zilong Zheng

From the One, Judge of the Whole: Typed Entailment Graph Construction with Predicate Generation (2023.acl-long)

Challenge:	despite advances in multimodal large language models, the challenge of interpreting long-form videos remains a challenge . despite advancements in video-language benchmarks, the inefficiency in temporal grounding and limited pre-trained context window size remains .
Approach:	They propose a framework that bootstraps MLLMs with advanced temporal grounding capabilities and broadens their contextual scope.
Outcome:	The proposed framework significantly enhances the temporal capabilities of existing MLLMs.

Copied to clipboard

Zhibin Chen, Yansong Feng, Dongyan Zhao

PPTC Benchmark: Evaluating Large Language Models for PowerPoint Task Completion (2024.findings-acl)

Challenge:	Existing methods to construct entailment graphs suffer from severe sparsity issues due to limited corpora and the long-tail phenomenon of predicate distributions.
Approach:	They propose a multi-stage method to generate entailment graphs by generating new predicates and detecting enanglement relations among seed predicats.
Outcome:	The proposed method can generate high-quality graphs with high precision over state-of-the-art methods and boost the performance of down-stream inference tasks.

Copied to clipboard

Yiduo Guo, Zekai Zhang, Yaobo Liang, Dongyan Zhao, Nan Duan

Rethinking Dictionaries and Glyphs for Chinese Language Pre-training (2023.findings-acl)

Challenge:	Recent evaluations of Large Language Models (LLMs) focus on their zero-shot/few-shot capabilities for basic natural language tasks and their ability to translate instructions into tool APIs.
Approach:	They propose a PowerPoint Task Completion benchmark to assess LLMs’ ability to create and edit PPT files based on user instructions.
Outcome:	The proposed system outperforms open-source and closed LLMs with 75.1% accuracy in single-turn dialogue testing but only achieves 6% session accuracy.

Copied to clipboard

Yuxuan Wang, Jack Wang, Dongyan Zhao, Zilong Zheng

From Simple to Complex: A Progressive Framework for Document-level Informative Argument Extraction (2023.findings-emnlp)

Challenge:	Large-scale pre-trained language models (PLMs) such as BERT and GPT have revolutionized various research fields in natural language processing (NLP)
Approach:	They propose a new learning paradigm that enhances the semantics understanding ability of Chinese PLMs with dictionary knowledge and structure of Chinese characters.
Outcome:	The proposed model improves on both modern Chinese understanding benchmark CLUE and ancient Chinese understanding.

Copied to clipboard

Quzhe Huang, Yanxi Zhang, Dongyan Zhao

Shorten After You’re Right: Lazy Length Penalties for Reasoning RL (2026.findings-acl)

Challenge:	Existing methods for document-level event argument extraction use memory to store the results of already predicted events.
Approach:	They propose a simple-to-complex progressive framework for document-level event argument extraction . they first calculate the difficulty of each event and then conduct the extraction following a simpler order .
Outcome:	The proposed model outperforms previous methods by 1.4% in the document-level EAE task.

Copied to clipboard

Danlong Yuan, Tian Xie, Shaohan Huang, Huishuai Zhang, Zhuocheng Gong, Chong Luo, Furu Wei, Dongyan Zhao

Reasoning

Challenge:	Existing shortening methods for long reasoning models rely on additional supervision or multi-stage post-training.
Approach:	They propose a lazy length penalty that imposes length pressure on models without extra training stages.
Outcome:	The proposed method significantly reduces response length without extra training stages while maintaining or improving performance.

Entailment Graph Learning with Textual Entailment and Soft Transitivity (2022.acl-long)

Copied to clipboard

Zhibin Chen, Yansong Feng, Dongyan Zhao

Class-Incremental Learning based on Label Generation (2023.acl-short)

Challenge:	Typed entailment graphs suffer from severe sparsity and unreliability of distributional similarity . enlargement relation is critical to semantic understanding and natural language inference .
Approach:	They propose a method to learn local entailment relations by recognizing textual enanglement between template sentences formed by typed CCG-parsed predicates.
Outcome:	The proposed method can model transitivity in entailment graphs to alleviate sparsity and improve performance over current methods.

Copied to clipboard

Yijia Shao, Yiduo Guo, Dongyan Zhao, Bing Liu

VMSMO: Learning to Generate Multimodal Summary for Video-based News Articles (2020.emnlp-main)

Challenge:	Existing studies on pre-trained language models focus on task-incremental learning (TIL) but they perform poorly in a more challenging setting of class-incremental learning.
Approach:	They propose a method which solves CIL based on label generation by using sparse vocabulary and creates pseudo-replay samples by using label semantics.
Outcome:	The proposed method outperforms baseline models by a large margin in the class-incremental learning setting.

Copied to clipboard

Mingzhe Li, Xiuying Chen, Shen Gao, Zhangming Chan, Dongyan Zhao, Rui Yan

Are Training Samples Correlated? Learning to Generate Dialogue Responses with Multiple References (P19-1)

Challenge:	Existing studies show that multimodal news can significantly improve users' sense of satisfaction for informativeness.
Approach:	They propose a task of Video-based Multimodal Summarization with Multimodal Output to solve this problem.
Outcome:	The proposed method can generate multimodal summaries with a single input . it can model the temporal dependency of video with semantic meaning of article .

Copied to clipboard

Lisong Qiu, Juntao Li, Wei Bi, Dongyan Zhao, Rui Yan

Enhancing the Open-Domain Dialogue Evaluation in Latent Space (2021.findings-acl)

Challenge:	Existing approaches to open-domain dialogue generation ignore the nature of 1-to-1 mapping that there may exist multiple valid responses corresponding to the same query.
Approach:	They propose to model open-domain dialogue generation using 1-to-1 mapping . they first extract common features of different responses and then combine them with distinctive features to generate multiple diverse and appropriate responses.
Outcome:	The proposed model outperforms existing models on automatic and human evaluations.

Copied to clipboard

Zhangming Chan, Lemao Liu, Juntao Li, Haisong Zhang, Dongyan Zhao, Shuming Shi, Rui Yan

Learning to Express in Knowledge-Grounded Conversation (2022.naacl-main)

Challenge:	Existing methods to evaluate opendomain dialogues are limited due to the one-to-many nature of dialogues.
Approach:	They propose a self-supervised setting to obtain a smooth latent space that captures discourse-level context information and implicitly models more references in latent spaces.
Outcome:	The proposed method outperforms baseline methods on two real-world dialogue datasets.

Copied to clipboard

Xueliang Zhao, Tingchen Fu, Chongyang Tao, Wei Wu, Dongyan Zhao, Rui Yan

How to Write Summaries with Patterns? Learning towards Abstractive Summarization through Prototype Editing (D19-1)

Challenge:	Existing models focus on synthesizing a dialogue with proper knowledge, but neglect that the same knowledge could be expressed differently even under the same context.
Approach:	They propose a model that ground dialogue generation by extra knowledge by analyzing the structure of the response and the content style of each part.
Outcome:	The proposed model can learn the structure style defined by a few examples and generate responses in desired content style.

Copied to clipboard

Shen Gao, Xiuying Chen, Piji Li, Zhangming Chan, Dongyan Zhao, Rui Yan

Efficient Domain Continual pretraining by Mitigating the Stability Gap (2025.acl-long)

Challenge:	Extensive experiments on a large-scale real-world text summarization dataset show that PESG achieves the state-of-the-art performance in terms of both automatic metrics and human evaluations.
Approach:	They propose a model that learns summary patterns and prototype facts from a prototype document . they use a fact checker to estimate mutual information between the input document and generated summary .
Outcome:	Experiments on a large-scale real-world text summarization dataset show that PESG achieves state-of-the-art performance.

Copied to clipboard

Yiduo Guo, Jie Fu, Huishuai Zhang, Dongyan Zhao

Semi-supervised Text Style Transfer: Cross Projection in Latent Space (D19-1)

Challenge:	Continual pretraining is an important approach for Large Language Models to improve their performance in target domains, learn new topics and languages, and even boost their general capabilities.
Approach:	They propose a training strategy that mitigates instability by increasing the number of epochs, along with two data sampling strategies targeting data domain relevance and corpus distribution.
Outcome:	The proposed training strategy improves the average medical task performance of the OpenLlama-3B model from 36.2% to 40.7% using only 40% of the original training budget, while also enhancing general task performance without causing forgetting.

Copied to clipboard

Mingyue Shang, Piji Li, Zhenxin Fu, Lidong Bing, Dongyan Zhao, Shuming Shi, Rui Yan

SCALE: Synergized Collaboration of Asymmetric Language Translation Engines (2024.findings-acl)

Challenge:	Text style transfer task has long suffered from the shortage of parallel data .
Approach:	They propose a semi-supervised text style transfer model that combines parallel data with large-scale nonparallel data to train it.
Outcome:	The proposed model can transfer a sentence of one style to another while retaining its original content meaning while preserving its original meaning.

Copied to clipboard

Xin Cheng, Xun Wang, Tao Ge, Si-Qing Chen, Furu Wei, Dongyan Zhao, Rui Yan

Multilinguality

Challenge:	In this paper, we introduce SCALE, a collaborative framework that connects a compact Specialized Translation Model (STM) and a general-purpose Large Language Model (LLM) as one unified translation engine.
Approach:	They propose a collaborative framework that connects a Specialized Translation Model (STM) and a general-purpose Large Language Model (LLM) as one unified translation engine.
Outcome:	The proposed framework outperforms both LLMs and supervised models in high-resource or challenging low-resourced settings.

VSTAR: A Video-grounded Dialogue Dataset for Situated Semantic Understanding with Scene and Topic Transitions (2023.acl-long)

Copied to clipboard

Yuxuan Wang, Zilong Zheng, Xueliang Zhao, Jinpeng Li, Yueqian Wang, Dongyan Zhao

Learning Dynamic Representations for Discourse Dependency Parsing (2023.findings-emnlp)

Challenge:	Existing benchmarks for video-grounded dialogues neglect the intrinsic attributes of multimodal dialogues, such as scene and topic transitions.
Approach:	They propose to use a large scale video-grounded scene&topic AwaRe dialogue dataset to study video-based dialogue understanding.
Outcome:	The proposed dataset shows that multimodal information and segments are important in video-grounded dialogue understanding and generation.

Copied to clipboard

Tianyi Liu, Yansong Feng, Dongyan Zhao

PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models (2023.findings-acl)

Challenge:	Existing models characterize transition states by examining a certain number of elementary discourse units (EDUs) Existing work neglects the arcs obtained from the transition history.
Approach:	They propose to employ GAT-based encoder to learn dynamic representations for sub-trees constructed in previous transition steps.
Outcome:	The proposed model retains access to parsed EDUs through the obtained arcs, especially when handling lengthy text spans with complex structures.

Copied to clipboard

Zhuocheng Gong, Jiahao Liu, Qifan Wang, Yang Yang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Rui Yan

Curriculum Learning based Hierarchical Scoring and Analysis Framework for Question Answering Task Evaluation (2026.findings-acl)

Challenge:	Quantization is a viable solution for pre-trained language models, but most existing methods are task-specific and require customized training and quantization with a large number of trainable parameters.
Approach:	They propose a "quantize before fine-tuning" framework that allows for quantization with a large number of trainable parameters on each individual task.
Outcome:	The proposed framework is compatible with quantization-aware training and post-training quantization and corrects quantization errors.

Copied to clipboard

Qiong Wu, Tan Yue, Jianxin Liang, Zhen Li, Kai He, Shuai Zhao, Dongyan Zhao

Question Answering

Challenge:	Existing evaluation methods rely on rule-based matching with shallow semantic understanding or adopt LLM-as-a-Judge approaches that incur high cost and latency while offering limited error interpretability.
Approach:	They propose a curriculum learning based hierarchical framework for QA task evaluation that supports quick scoring and fine-grained error analysis.
Outcome:	The proposed framework outperforms baseline methods on quick scoring and error analysis tasks while being 25 faster.

Jointly Learning Entity and Relation Representations for Entity Alignment (D19-1)

Copied to clipboard

Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, Dongyan Zhao

Stick to the Facts: Learning towards a Fidelity-oriented E-Commerce Product Description Generation (D19-1)

Challenge:	Entity alignment is a viable method for integrating heterogeneous knowledge among different knowledge graphs (KGs).
Approach:	They propose a Graph Convolutional Network-based framework for learning relation representations by embedding relation seeds into entities and incorporating relation approximation into entities to iteratively improve alignment.
Outcome:	The proposed approach outperforms state-of-the-art methods on three real-world cross-lingual datasets.

Copied to clipboard

Zhangming Chan, Xiuying Chen, Yongliang Wang, Juntao Li, Zhiqiang Zhang, Kun Gai, Dongyan Zhao, Rui Yan

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models (2026.acl-long)

Challenge:	Existing models for product description generation do not take the product attribute information into account.
Approach:	They propose a model that takes the embedding and the entity label of each word into account . they establish a keyword memory that stores the entity labels as keys and keywords as values .
Outcome:	The proposed model increases the fidelity of the generated descriptions by 25%.

Copied to clipboard

Xin Cheng, Wangding Zeng, Damai Dai, Qinyu Chen, Bingxuan Wang, Zhenda Xie, Kezhao Huang, Xingkai Yu, Zhewen Hao, Han Zhang, Yu-Kun Li, Huishuai Zhang, Dongyan Zhao, Wenfeng Liang

Things not Written in Text: Exploring Spatial Commonsense from Visual Signals (2022.acl-long)

Challenge:	Mixture-of-Experts (MoE) scales capacity via conditional computation, but lacks knowledge lookup primitive.
Approach:	They propose a conditional memory instantiated via Deep Sparse Embedding (DSE) they propose 'u-shaped scaling law' that identifies optimal balance between MoE experts and DSE memory .
Outcome:	The proposed model outperforms an iso-parameter and isoFLOPs MoE baseline across knowledge and reasoning benchmarks and is infrastructure-efficient.

Copied to clipboard

Xiao Liu, Da Yin, Yansong Feng, Dongyan Zhao

Bi-Directional Multi-Granularity Generation Framework for Knowledge Graph-to-Text with Large Language Model (2024.acl-short)

Challenge:	Pretrained language models fail in many NLP tasks, but are ineffective in spatial commonsense reasoning.
Approach:	They propose a spatial commonsense benchmark that focuses on relative scales of objects and the positional relationship between people and objects under different actions.
Outcome:	The proposed framework outperforms pretrained models in answering spatial questions.

Copied to clipboard

Haowei Du, Chen Li, Dinghao Zhang, Dongyan Zhao

Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocRED (2022.acl-long)

Challenge:	Existing methods generate whole text based on all KG triples at once and may incorporate incorrect KG Triples for each sentence.
Approach:	They propose a bi-directional multi-granularity generation framework that generates graph-level sentences based on KG triples instead of the whole text at a time.
Outcome:	The proposed framework achieves state-of-the-art in benchmark dataset WebNLG and further analysis shows the efficiency of different modules.

Copied to clipboard

Quzhe Huang, Shibo Hao, Yuan Ye, Shengqi Zhu, Yansong Feng, Dongyan Zhao

Challenge:	Document-level relation extraction is a challenging task as it requires reasoning across multiple sentences.
Approach:	They propose to use a recommend-revise scheme to reduce the workload of annotators by providing them with candidate relation instances from distant supervision to supplement and remove relational facts.
Outcome:	The proposed dataset is the first large-scale and human-annotated dataset for relation extraction.

ProphetChat: Enhancing Dialogue Generation with Simulation of Future Conversation (2022.acl-long)

Copied to clipboard

Chang Liu, Xu Tan, Chongyang Tao, Zhenxin Fu, Dongyan Zhao, Tie-Yan Liu, Rui Yan

Attend, Select and Eliminate: Accelerating Multi-turn Response Selection with Dual-attention-based Content Elimination (2023.findings-acl)

Challenge:	generative dialogue models use dialogue histories to generate the response . however, generating a response based on the historical information is not easy .
Approach:	They propose a framework that utilizes simulated dialogue futures to enhance response generation.
Outcome:	The proposed framework can generate better responses over strong baselines on two open-domain dialogue datasets.

Copied to clipboard

Jianxin Liang, Chang Liu, Chongyang Tao, Jiazhan Feng, Dongyan Zhao

Lattice-BERT: Leveraging Multi-Granularity Representations in Chinese Pre-trained Language Models (2021.naacl-main)

Challenge:	Pre-trained language models can be used to perform multi-turn response selection, but they can be expensive.
Approach:	They propose a framework and a strategy that progressively selects and eliminates unimportant content under context-response dual-attention.
Outcome:	The proposed method can effectively speed-up SOTA models without much performance degradation and shows a better trade-off between speed and performance than previous methods.

Copied to clipboard

Yuxuan Lai, Yijia Liu, Yansong Feng, Songfang Huang, Dongyan Zhao