Papers by Dawei Yin
Data Manipulation: Towards Effective Instance Learning for Neural Dialogue Generation via Learning to Augment and Reweight (2020.acl-main)
Copied to clipboard
| Challenge: | Current state-of-the-art neural dialogue models learn from human conversations . however, due to the open-ended nature of human conversations, the quality of training data varies . |
| Approach: | They propose a data manipulation framework to augment and highlight effective training samples . they also propose to increase its manipulation skills through gradient descent with validation samples a reshaping framework to proactively restructure the data distribution towards reliable samples is also proposed . |
| Outcome: | The proposed framework improves the performance of open-domain neural dialogue models with respect to evaluation metrics and human judgments. |
Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures (P18-1)
Copied to clipboard
| Challenge: | Existing solutions to task-oriented dialogue systems follow pipeline designs which introduces complexity and fragility. |
| Approach: | They propose a novel sequence-to-sequence (seq2sequ) model which tracks dialogue believes and a two stage copynet instantiation which emonstrates good scalability. |
| Outcome: | The proposed framework outperforms state-of-the-art pipeline-based methods on large datasets and retains satisfactory entity match rate on out-of vocabulary (OOV) cases where pipeline-designed competitors totally fail. |
Retrieval Models Aren’t Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) suffer from inherent inabilities to interact with the physical world and access vast, up-to-date knowledge. |
| Approach: | They propose a tool retrieval benchmark for large language models (LLMs) that includes 7.6k diverse retrieval tasks and a corpus of 43k tools. |
| Outcome: | The proposed model performs poorly on the heterogeneous tool retrieval benchmark, resulting in low pass rate and low retrieval quality. |
Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Reasoning Models (LLMs) have demonstrated impressive performances across diverse domains, but how their safety benefits from enhanced reasoning capabilities against jailbreak queries remains unexplored. |
| Approach: | They propose a safety-aware reasoning paradigm that integrates a pivot token-based safety-based reasoning mechanism into LLMs’ generation process. |
| Outcome: | The proposed model improves the safety of large language models against jailbreak queries while minimizing attacks and maintaining the original performance. |
Reinforced Efficient Reasoning via Semantically Diverse Exploration (2026.acl-long)
Copied to clipboard
Ziqi Zhao, Zhaochun Ren, Jiahong Zou, Liu Yang, Zhiwei Xu, Xuri Ge, Zhumin Chen, Xinyu Ma, Daiting Shi, Shuaiqiang Wang, Dawei Yin, Xin Xin
| Challenge: | Existing methods for reinforcement learning with verifiable rewards suffer from limited exploration diversity and inefficient reasoning. |
| Approach: | They propose a method that rewards concise and correct reasoning while penalizing unnecessarily long reasoning chains. |
| Outcome: | Extensive experiments on Qwen and Llama models validate the effectiveness and efficiency of ROSE. |
CoRanking: Collaborative Ranking with Small and Large Ranking Agents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Listwise ranking based on Large Language Models (LLMs) has achieved state-of-the-art performance in Information Retrieval (IR) however, their effectiveness often depends on LLMs with massive parameter scales and computationally expensive sliding window processing, leading to substantial efficiency bottlenecks. |
| Approach: | They propose a Collaborative Ranking framework (CoRanking) for LLM-based listwise ranking based on large language models with massive parameter scales and computationally expensive sliding window processing. |
| Outcome: | The proposed framework reduces ranking latency by approximately 70% while improving effectiveness compared to the standalone large reranker. |
Enhancing Retrieval-Augmented Generation via Evidence Tree Search (2025.acl-long)
Copied to clipboard
| Challenge: | Evidence retrieval is used to enhance Large Language Models (LLMs) but in real-world applications, it often returns lengthy documents with redundant or irrelevant content, confusing downstream readers. |
| Approach: | They propose a framework that reformulates evidence retrieval as a dynamic tree expansion process. |
| Outcome: | The proposed framework outperforms existing methods on five datasets. |
The Mirage of Model Editing: Revisiting Evaluation in the Wild (2025.acl-long)
Copied to clipboard
| Challenge: | despite near-perfect results, effectiveness of model editing in real-world applications remains unclear. |
| Approach: | They propose QAEdit and WILD to better reflect real-world use of model editing . they propose a benchmark aligned with widely used question answering datasets and a task-agnostic evaluation framework . |
| Outcome: | The proposed QAEdit benchmark and WILD evaluation framework show that current models perform worse than previously reported. |
An Efficient Framework for Whole-Page Reranking via Single-Modal Supervision (2026.acl-industry)
Copied to clipboard
Zishuai Zhang, Sihao Yu, null Xiewenyi, Ying Nie, Junfeng Wang, Zhiming Zheng, Dawei Yin, Hainan Zhang
| Challenge: | Existing whole-page reranking methods require large-scale expert annotations to achieve high-quality results. |
| Approach: | They propose a whole-page reranking framework that converts single-modal rankers into page-level guidance by constructing budget-aware candidates for cross-modal annotations and distilling intra-modality preferences to align relevance scales across modalities. |
| Outcome: | The proposed framework reduces annotation costs by 70-90% while outperforming fully-annotated reranking baselines. |
Learning to Use Tools via Cooperative and Interactive Agents (2024.findings-emnlp)
Copied to clipboard
Zhengliang Shi, Shen Gao, Xiuyi Chen, Yue Feng, Lingyong Yan, Haibo Shi, Dawei Yin, Pengjie Ren, Suzan Verberne, Zhaochun Ren
| Challenge: | Existing methods for large language models (LLMs) use one agent to iterate and execute tools, but they suffer from performance degradation when addressing practical tasks. |
| Approach: | They propose a tool learning framework that coordinates three specialized agents for tool selection, tool execution, and action calibration separately. |
| Outcome: | The proposed framework outperforms baseline models on three datasets with 14% higher success rate. |
Adversarial Yet Cooperative: Multi-Perspective Reasoning in Retrieved-Augmented Language Models (2026.findings-acl)
Copied to clipboard
Can Xu, Lingyong Yan, Jiayi Wu, Haosen Wang, Shuaiqiang Wang, Yuchen Li, Jizhou Huang, Dawei Yin, Xiang Li
| Challenge: | Existing training paradigms rely on outcome-oriented rewards, which provide insufficient signal for shaping the complex, multi-step reasoning process. |
| Approach: | They propose a framework that integrates large reasoning models with retrieval-augmented generation to improve reasoning fidelity and verification rigor. |
| Outcome: | Experiments on multiple benchmarks demonstrate the effectiveness of the proposed framework. |
MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansion (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for query expansion lack corpus-specific knowledge and cost. |
| Approach: | They propose a query-query-document generation method that leverages large language models for mutual verification to produce diverse sub-queries and corresponding documents. |
| Outcome: | The proposed method is fully zero-shot and extensive experiments on three public benchmark datasets demonstrate its effectiveness over existing methods. |
RACQC: Advanced Retrieval-Augmented Generation for Chinese Query Correction (2025.findings-emnlp)
Copied to clipboard
Jinbo Su, Lingzhe Gao, Wei Li, Shihao Liu, Haojie Lei, Xinyi Wang, Yuanzhao Guo, Ke Wang, Daiting Shi, Dawei Yin
| Challenge: | Large language models (LLMs) exhibit remarkable capabilities across many tasks, but face critical challenges in the CSC scenario: (1) poor generalization to rare entities in open-domain searches; and (2) failure to adapt to temporal entity variations due to static parameters, resulting in serious over-correction issues. |
| Approach: | They propose a Chinese Spelling Check system with RAG and multi-task learning that integrates dynamic knowledge retrieval and entity-centric RAG to address rare entities. |
| Outcome: | The proposed system outperforms existing baselines in the CSC task and achieves a maximum improvement of +9.92% on the search scenario benchmark and +3.2% on the general-domain dataset. |
ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (2026.acl-long)
Copied to clipboard
| Challenge: | Existing rerankers perform poorly in complex ranking scenarios due to the scarcity of reasoning-intensive training data. |
| Approach: | They propose an automated reasoning-intensive training framework which generates high-quality training labels from training queries and passages. |
| Outcome: | The proposed model outperforms baselines significantly and achieves much lower latency than the pointwise reranker. |
Are Message Passing Neural Networks Really Helpful for Knowledge Graph Completion? (2023.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge graphs are far from complete with large portions of triplets missing. |
| Approach: | They propose to use Graph Neural Networks to learn powerful embeddings to improve model performance. |
| Outcome: | The proposed models achieve comparable performance to MLP models, suggesting that MP may not be as crucial as previously thought. |
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023.emnlp-main)
Copied to clipboard
Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, Zhaochun Ren
| Challenge: | Existing work utilizes generative LLMs for Information Retrieval (IR) rather than direct passage ranking. |
| Approach: | They investigate generative LLMs such as ChatGPT and GPT-4 for relevance ranking in IR and use a test set to verify the model’s ability to rank unknown knowledge. |
| Outcome: | The proposed model outperforms a 3B supervised model on the BEIR benchmark. |
PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to optimize RAG generators fail to align with RAG requirements thoroughly. |
| Approach: | They propose a method for optimizing the RAG generator from multiple preference perspectives to align with RAG requirements comprehensively. |
| Outcome: | The proposed method improves the performance of RAG generators by incorporating retrieved documents into the prompt. |
TP-RAG: Benchmarking Retrieval-Augmented Large Language Model Agents for Spatiotemporal-Aware Travel Planning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on large language models (LLMs) focus on basic plan validity, but neglect critical aspects such as route efficiency, POI appeal, and real-time adaptability. |
| Approach: | They propose a benchmark for retrieval-augmented, spatiotemporal-aware travel planning that integrates retrieved trajectories with LLMs’ intrinsic reasoning. |
| Outcome: | The proposed framework improves spatial efficiency and POI rationality while challenging universality and robustness due to conflicting references and noisy data. |
Task Knowledge Injection via Interpolations and Reinstatement for Large Language Model Generalization (2025.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained large language models have been widely adopted to elicit their superior performance on downstream tasks, but instruction tuning may overfit them to specific task formats, compromising their generalization on unseen tasks. |
| Approach: | They propose to inject latent task adaptation and knowledge reinstatement into large language models to mitigate spurious correlations between inputs and targets. |
| Outcome: | The proposed method improves generalization on in-domain and out-of-domain unseen tasks. |
AdaSwitch: Adaptive Switching between Small and Large Agents for Effective Cloud-Local Collaborative Learning (2024.emnlp-main)
Copied to clipboard
Hao Sun, Jiayi Wu, Hengyi Cai, Xiaochi Wei, Yue Feng, Bo Wang, Shuaiqiang Wang, Yan Zhang, Dawei Yin
| Challenge: | Recent advances in large language models have been remarkable . users face a choice between using cloud-based LLMs for generation quality or local-based ones for lower computational cost . |
| Approach: | They propose a new LLM utilization paradigm that facilitates collaborative operation . they evaluate AdaSwitch across 7 benchmarks and compare it to other LLMs . |
| Outcome: | The proposed model improves performance of local and cloud agents across 7 benchmarks . it achieves competitive results compared to the cloud agent while utilizing less computational overhead. |
Knowledge Diffusion for Neural Dialogue Generation (P18-1)
Copied to clipboard
| Challenge: | End-to-end neural dialogue generation does not employ knowledge to guide the generation. |
| Approach: | They propose a neural knowledge diffusion model to introduce knowledge into dialogue generation. |
| Outcome: | The proposed model outperforms baseline models on a real-world dataset. |
Exploring Memorization in Fine-tuned Language Models (2024.acl-long)
Copied to clipboard
Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu, Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, Dawei Yin
| Challenge: | Existing studies have shown that pre-trained langauge models tend to memorize and regenerate segments of their pre-training corpus when prompted appropriately. |
| Approach: | They conduct the first comprehensive analysis to explore language models’ memorization during fine-tuning across tasks. |
| Outcome: | The proposed analysis shows that memorization presents a strong disparity among different fine-tuning tasks. |
The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG) (2024.findings-acl)
Copied to clipboard
Shenglai Zeng, Jiankun Zhang, Pengfei He, Yiding Liu, Yue Xing, Han Xu, Jie Ren, Yi Chang, Shuaiqiang Wang, Dawei Yin, Jiliang Tang
| Challenge: | Retrieval-augmented generation (RAG) is a powerful technique to facilitate language model generation with proprietary and private data, where data privacy is . a privacy issue that is currently under-explored, is posed by RAG. |
| Approach: | They propose to use retrieval-augmented generation (RAG) to facilitate language model generation with proprietary and private data where data privacy is a pivotal concern. |
| Outcome: | The proposed attack methods demonstrate that RAG can mitigate the old risks, i.e., leakage of the LLMs’ training data. |
Agentic-R: Learning to Retrieve for Agentic Search (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing retrievers for single-turn retrieval-augmented generation (RAG) rely on similarity-based retrievers, but similar passages are not always useful for final answer generation. |
| Approach: | They propose a retrieval-augmented-generation retriever that integrates reasoning with retrieval . they use local query-passage relevance and global answer correctness to measure passage utility . |
| Outcome: | The proposed retriever outperforms existing retrievers on QA benchmarks on seven single-hop and multi-hop searches. |
Original Content Is All You Need! an Empirical Study on Leveraging Answer Summary for WikiHowQA Answer Selection Task (2022.coling-1)
Copied to clipboard
Liang Wen, Juan Li, Houfeng Wang, Yingwei Luo, Xiaolin Wang, Xiaodong Zhang, Zhicong Cheng, Dawei Yin
| Challenge: | Existing answer selection approaches for community question answering lack additional answer summaries due to redundancy and lengthiness issues of crowdsourced answers. |
| Approach: | They constructed a dataset which contains a corresponding reference summary for each original lengthy answer. |
| Outcome: | The proposed model improves the performance of a question and candidate answer on a WikiHowQA dataset. |
Adaptive Parameterization for Neural Dialogue Generation (D19-1)
Copied to clipboard
| Challenge: | Existing models of open-domain dialogue generate responses based on sequence-to-sequence paradigms. |
| Approach: | They propose an Adaptive Neural Dialogue generation model which manages various conversations with conversation-specific parameterization. |
| Outcome: | The proposed model performs better on a large-scale conversational dataset. |
Igniting Creative Writing in Small Language Models: LLM-as-a-Judge versus Multi-Agent Refined Rewards (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for enhancing Large Language Models (LLMs) struggle with novelty and Reinforcement Learning from human feedback (RLHF) is costly. |
| Approach: | They propose to use a Reward Model (RM) and a principle-guided LLM-as-a-Judge to enhance creative output over baselines. |
| Outcome: | The proposed approach significantly enhances creative output over baselines, but the principle-guided LLM-as-a-Judge yields superior generation quality. |
Towards Verifiable Text Generation with Evolving Memory and Self-Reflection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) often produce factually incorrect information, also known as hallucination. |
| Approach: | They propose a framework for verifiable text generation with evolving memory and self-reflection that incorporates long-term memory to retain documents and recent documents. |
| Outcome: | The proposed framework outperforms baselines on five datasets across three knowledge-intensive tasks. |
Improving the Robustness of Large Language Models via Consistency Alignment (2024.lrec-main)
Copied to clipboard
Yukun Zhao, Lingyong Yan, Weiwei Sun, Guoliang Xing, Shuaiqiang Wang, Chong Meng, Zhicong Cheng, Zhaochun Ren, Dawei Yin
| Challenge: | Large language models have shown tremendous success in following user instructions and generating helpful responses, but their robustness is still far from optimal. |
| Approach: | They propose a two-stage training framework that helps a model generalize on following instructions via similar instruction augmentations. |
| Outcome: | The proposed training framework improves diversity and aligns the model with human expectations by differentiating subtle differences in similar responses. |
Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for listwise passage ranking use sliding window approach, which is inefficient as it requires repetitive and serialized processing. |
| Approach: | They propose a listwise label construction approach and importance-aware learning objective for full ranking. |
| Outcome: | The proposed method outperforms existing methods in listwise ranking tasks. |
A Robust Semantics-based Watermark for Large Language Model against Paraphrasing (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to detect LLM-generated content use simple hashes of precedent tokens to partition vocabulary. |
| Approach: | They propose a semantics-based watermark framework to enhance the robustness against paraphrase. |
| Outcome: | The proposed framework is robust under different paraphrases and the semantic meaning of the sentences will be likely preserved under paraphrase. |
LawBench: Benchmarking Legal Knowledge of Large Language Models (2024.emnlp-main)
Copied to clipboard
Zhiwei Fei, Xiaoyu Shen, Dawei Zhu, Fengzhe Zhou, Zhuo Han, Alan Huang, Songyang Zhang, Kai Chen, Zhixin Yin, Zongwen Shen, Jidong Ge, Vincent Ng
| Challenge: | LegalBench evaluated 20 LLMs in 162 legal tasks in 20 countries and jurisdictions. |
| Approach: | They present a comprehensive evaluation of 21 popular Large Language Models and the first comparative analysis of the empirical results. |
| Outcome: | The proposed benchmarks are based on the Bloom’s cognitive taxonomy and are compared to 21 popular LLMs. |
The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse (2024.findings-acl)
Copied to clipboard
| Challenge: | Even a single edit can trigger model collapse, manifesting as significant performance degradation in various benchmark tasks. |
| Approach: | They propose to use perplexity as a surrogate metric to determine whether an edited model's performance is affected by a single edit. |
| Outcome: | The proposed method shows that even a single edit can cause model collapse, manifesting as significant performance degradation in various benchmark tasks. |
MatchTIR: Fine-Grained Supervision for Tool-Integrated Reasoning via Bipartite Matching (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reinforcement learning methods rely on outcome- or trajectory-level rewards, assigning uniform advantages to all steps within a trajectory. |
| Approach: | They propose a framework that introduces fine-grained supervision via bipartite matching-based turn-level reward assignment and dual-level advantage estimation. |
| Outcome: | The proposed framework surpasses the majority of 8B competitors on three benchmarks. |
Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method (2024.naacl-long)
Copied to clipboard
Yukun Zhao, Lingyong Yan, Weiwei Sun, Guoliang Xing, Chong Meng, Shuaiqiang Wang, Zhicong Cheng, Zhaochun Ren, Dawei Yin
| Challenge: | Recent literature reveals that Large Language Models (LLMs) hallucinate intermittently, which impedes their reliability for further utilization. |
| Approach: | They propose a self-detection method to detect which questions an LLM does not know by combining the two components to identify whether the model generates a non-factual response to the question. |
| Outcome: | The proposed method can detect which questions an LLM does not know across factoid question-answering, arithmetic reasoning, and commonsense reasoning tasks. |
Uplift-RAG: Uplift-Driven Knowledge Preference Alignment for Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing efforts to estimate document utility rely on downstream generation performance, which conflates the influence of external documents with the intrinsic knowledge of the LLM. |
| Approach: | They propose an uplift-based definition of document utility that quantifies each document’s marginal benefit over the LLM’s internal knowledge. |
| Outcome: | The proposed framework improves the performance of the LLM by incorporating external retrieved documents into the model. |
CTR-Guided Generative Query Suggestion in Conversational Search (2025.emnlp-industry)
Copied to clipboard
Erxue Min, Hsiu-Yuan Huang, Xihong Yang, Min Yang, Xin Jia, Yunfang Wu, Hengyi Cai, Junfeng Wang, Shuaiqiang Wang, Dawei Yin
| Challenge: | Generating effective query suggestions requires aligning model outputs with user click preferences. |
| Approach: | They propose a generative framework that leverages click modeling to denoise implicit feedback and enables reliable preference optimization for improving real-world user engagement. |
| Outcome: | The proposed framework outperforms strong baselines in CTR, relevance, diversity and diversity. |
DORA: A Dual-Objective Reinforcement Learning Framework for Effective and Efficient Multimodal Agentic Search (2026.acl-long)
Copied to clipboard
Guangming Qin, Yuhao Deng, Yukun Zhao, Zhenyang Li, Junfeng Wang, Dawei Yin, Ye Yuan, Guoren Wang, Yizhou Yan, Chengliang Chai, Lei Cao
| Challenge: | Existing methods to train large language models overlook quality of intermediate search results . existing methods often invoke search calls during reasoning, making inference inefficient . |
| Approach: | They propose a dual-objective reinforcement learning framework to improve search strategies of MLLMs . DORA outperforms state-of-the-art methods, achieving up to 8.4% higher accuracy . |
| Outcome: | The proposed model outperforms state-of-the-art methods while reducing search calls by 9.7%. |
Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization (2025.emnlp-main)
Copied to clipboard
Jiulong Wu, Zhengliang Shi, Shuaiqiang Wang, Jizhou Huang, Dawei Yin, Lingyong Yan, Min Cao, Min Zhang
| Challenge: | Existing preference alignment methods focus on aligning model responses with human preferences while neglecting image-text modality alignment. |
| Approach: | They propose Entity-centric Multimodal Preference Optimization to improve modality alignment . they use open-source instruction datasets to automatically construct high-quality preference data . |
| Outcome: | The proposed approach reduces hallucination rates by 80.4% on Object HalBench and 52.6% on MM HalBech. |
DiQAD: A Benchmark Dataset for Open-domain Dialogue Quality Assessment (2023.findings-emnlp)
Copied to clipboard
Yukun Zhao, Lingyong Yan, Weiwei Sun, Chong Meng, Shuaiqiang Wang, Zhicong Cheng, Zhaochun Ren, Dawei Yin
| Challenge: | Existing studies on dialogue quality assessment are uncapable of providing an end-to-end and human-epistemic assessment dataset . open-domain dialogue assessment is complicated and costly, but it can be done by recruiting human evaluators. |
| Approach: | They propose a large-scale dialogue quality assessment dataset for automatically assessing open-domain dialogue quality. |
| Outcome: | The proposed dataset is openly accessible at https://github.com/yukunZhao/Dialogue_quality_evaluation. |
PILE: Pairwise Iterative Logits Ensemble for Multi-Teacher Labeled Distillation (2022.emnlp-industry)
Copied to clipboard
Lianshang Cai, Linhao Zhang, Dehong Ma, Jun Fan, Daiting Shi, Yi Wu, Zhicong Cheng, Simiu Gu, Dawei Yin
| Challenge: | Pre-trained language models have been a key part of ranking systems . knowledge distillation is widely used to maintain high performance while keeping efficient computations. |
| Approach: | They propose an algorithm to combine knowledge from multi-teachers and label information to achieve competitive performance in offline and online experiments. |
| Outcome: | The proposed method has been deployed in a real-world commercial search system. |
Boosting Event Extraction with Denoised Structure-to-Text Augmentation (2023.findings-acl)
Copied to clipboard
Bo Wang, Heyan Huang, Xiaochi Wei, Ge Shi, Xiao Liu, Chong Feng, Tong Zhou, Shuaiqiang Wang, Dawei Yin
| Challenge: | Existing methods for event extraction neglect grammatical incorrectness, structure misalignment, and semantic drifting . et al., 2004; Ahn, 2006) show that the proposed method generates more diverse text representations for event extracting compared with the state-of-the-art. |
| Approach: | They propose a framework for event extraction that generates additional training data and iteratively selects the effective subset from the generated training data. |
| Outcome: | The proposed method generates more diverse representations of training data and achieves comparable results with the state-of-the-art. |
Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendation (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have produced significant advances in the field of recommender systems. |
| Approach: | They propose to retrieve up-to-date structure information from the knowledge graph to augment recommendations by leveraging external knowledge sources. |
| Outcome: | Experiments on a large dataset show that the proposed method is effective in enhancing LLM-based recommendations. |
Proactive Guidance of Multi-Turn Conversation in Industrial Search (2025.acl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have advanced multi-turn conversation systems, emphasizing the need for proactive guidance to enhance users’ interactions. |
| Approach: | They propose a goal-adaptive supervised fine-tuning framework that generates proactive guidance for users to click for the next turn of the conversation. |
| Outcome: | The proposed framework achieves 86.10% accuracy in offline evaluation (+23.95% over baseline) and 25.28% CTR in online deployment (149.06% relative improvement). |
Attribute-aware Sequence Network for Review Summarization (D19-1)
Copied to clipboard
| Challenge: | Existing review summarization systems generate summary only based on review content and neglect the authors’ attributes (e.g., gender, age, and occupation). |
| Approach: | They propose an Attribute-aware Sequence Network (ASN) to take the aforementioned users’ characteristics into account by encoding their attributes over the words. |
| Outcome: | The proposed model outperforms existing systems on tripAtt and human evaluation by taking the authors' attributes into account and incorporating attribute embedding and word-using habits into word prediction. |
UEGP: Unified Expert-Guided Pre-training for Knowledge Rekindle (2024.findings-naacl)
Copied to clipboard
Yutao Mou, Kexiang Wang, Jianhe Lin, Dehong Ma, Jun Fan, Daiting Shi, Zhicong Cheng, Gu Simiu, Dawei Yin, Weiran Xu
| Challenge: | Existing paradigms for pre-training and fine-tuning have limitations . knowledge rekindle aims to break through performance upper bounds of experts without introducing additional annotated data. |
| Approach: | They propose a new paradigm for pre-training and fine-tuning that aims to re-incorporate the fine- tuned expert model into the training cycle and break through performance upper bounds of experts. |
| Outcome: | The proposed model breaks through performance upper bounds of experts without additional annotated data. |
LLMs + Persona-Plug = Personalized LLMs (2025.acl-long)
Copied to clipboard
Jiongnan Liu, Yutao Zhu, Shuting Wang, Xiaochi Wei, Erxue Min, Yu Lu, Shuaiqiang Wang, Dawei Yin, Zhicheng Dou
| Challenge: | Large language models (LLMs) have demonstrated extraordinary capabilities in natural language understanding, generation, and reasoning. |
| Approach: | They propose a plug-and-play LLM model that embeds a user-specific embedding for each individual by modeling her historical contexts through a lightweight plug-in user embedder module. |
| Outcome: | Experiments on various tasks in the language model personalization (LaMP) benchmark show that the proposed model significantly outperforms existing personalized LLM approaches. |
ATM: Adversarial Tuning Multi-agent System Makes a Robust Retrieval-Augmented Generator (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are proven to benefit a lot from retrieval-augmented generation (RAG) due to noisy and fabricating content, it is inevitable that RAG systems are vulnerable to these noises and prone to respond incorrectly. |
| Approach: | They propose to optimize retrieval-augmented generation (RGG) with an Adversarial Tuning Multi-agent system (ATM) ATM steers the Generator to have a robust perspective of useful documents for question answering with the help of an auxiliary Attacker agent. |
| Outcome: | The proposed system improves the retrieval-augmented generator with an auxiliary Attacker agent and can discriminate useful documents amongst fabrications. |
KnowTuning: Knowledge-aware Fine-tuning for Large Language Models (2024.emnlp-main)
Copied to clipboard
Yougang Lyu, Lingyong Yan, Shuaiqiang Wang, Haibo Shi, Dawei Yin, Pengjie Ren, Zhumin Chen, Maarten Rijke, Zhaochun Ren
| Challenge: | Large language models (LLMs) are a default solution for many natural language processing tasks. |
| Approach: | They propose a knowledge-aware fine-tuning method to improve LLMs' knowledge awareness . they propose augmentation and comparison stages to improve accuracy and reliability . |
| Outcome: | The proposed method generates more facts with less factual error rate under fine-grained facts evaluation. |
GOVERN: Gradient Orientation Vote Ensemble for Multi-Teacher Reinforced Distillation (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Pre-trained language models have achieved remarkable performance in OpenQA, but for practical deployment, knowledge distillation is crucial to maintain high performance while operating under computational constraints. |
| Approach: | They propose an algorithm to perform unsupervised knowledge distillation without the guidance of labels to achieve 99.5% of performance. |
| Outcome: | The proposed algorithm achieves 99.5% of performance in a commercial question-answering system. |
The Fall of ROME: Understanding the Collapse of LLMs in Model Editing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have found that model editing methods can cause large language models to collapse with just a single edit. |
| Approach: | They propose a method that uses prefixed keys and adds prefixes during testing to prevent model collapse. |
| Outcome: | The proposed method prevents model collapse while maintaining effectiveness, the authors show . Rank-One Model Editing (ROME) has been found to cause model collapse with just a single edit . |
On Length Divergence Bias in Textual Matching Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing deep models have been successful in textual matching tasks, but it is unclear whether they understand language or measure semantic similarity of texts. |
| Approach: | They propose an adversarial evaluation scheme which invalidates the length divergence bias in TM datasets. |
| Outcome: | The proposed method improves the robustness and generalization ability of models at the same time. |
MAIR: A Massive Benchmark for Evaluating Instructed Retrieval (2024.emnlp-main)
Copied to clipboard
Weiwei Sun, Zhengliang Shi, Wu Long, Lingyong Yan, Xinyu Ma, Yiding Liu, Min Cao, Dawei Yin, Zhaochun Ren
| Challenge: | Existing IR benchmarks focus on a limited scope of tasks, making them insufficient for evaluating the latest IR models. |
| Approach: | They propose a multi-task instruction-tuned IR benchmark that includes 126 distinct IR tasks across 6 domains. |
| Outcome: | The proposed model performs better on instruction-tuned models than non-instruction-tunned models on MAIR. |
Divide-Then-Aggregate: An Efficient Tool Learning Method via Parallel Tool Invocation (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) demonstrate remarkable capabilities but their ability to autonomously execute complex real-world tasks remains limited. |
| Approach: | They propose a parallel tool invocation framework that decomposes tasks into parallel tool-using subtasks while aggregating results for subsequent decisions. |
| Outcome: | The proposed method significantly improves task performance while reducing token consumption and inference time. |
VisLingInstruct: Elevating Zero-Shot Learning in Multi-Modal Language Models with Autonomous Instruction Optimization (2024.naacl-long)
Copied to clipboard
Dongsheng Zhu, Daniel Tang, Weidong Han, Jinghui Lu, Yukun Zhao, Guoliang Xing, Junfeng Wang, Dawei Yin
| Challenge: | Current MMLMs show impressive zero-shot abilities in multi-modal tasks, but their performance depends heavily on the quality of instructions. |
| Approach: | They propose a novel approach to advancing multi-modal language models in zero-shot learning by evaluating and optimizing instructional texts through In-Context Learning. |
| Outcome: | The proposed approach improves zero-shot performance in multi-modal tasks by evaluating and optimizing instructional texts. |
Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation (2025.emnlp-main)
Copied to clipboard
Hengran Zhang, Minghao Tang, Keping Bi, Jiafeng Guo, Shihao Liu, Daiting Shi, Dawei Yin, Xueqi Cheng
| Challenge: | Existing studies on large language models for document utility annotations have shown that they improve retrieval performance and RAG outcomes compared to models trained on human annotations. |
| Approach: | They propose a model that maximizes their summed marginal likelihood to annotate document utility on multiple positive samples per query. |
| Outcome: | The proposed model maximizes the marginal likelihood of multiple positive samples per query. |