Papers by Di He
Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder (2021.emnlp-main)
Copied to clipboard
Shuqi Lu, Di He, Chenyan Xiong, Guolin Ke, Waleed Malik, Zhicheng Dou, Paul Bennett, Tie-Yan Liu, Arnold Overwijk
| Challenge: | Dense retrieval requires high-quality text sequence embeddings to support effective search in the representation space. |
| Approach: | They propose a self-learning method that pre-trains the autoencoder using a weak decoder to push the encoder to provide better sequence representations. |
| Outcome: | The proposed model significantly boosts the effectiveness and few-shot ability of dense retrieval models on web search, news recommendation, and open domain question answering. |
Augmenting Compliance-Guaranteed Customer Service Chatbots: Context-Aware Knowledge Expansion with Large Language Models (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Retrieval-based chatbots leverage human-verified Q&A knowledge to deliver accurate, verifiable responses. |
| Approach: | They propose a similar question generation task for LLM training and inference to enable comprehensive semantic exploration and enhanced alignment with source question-answer relationships. |
| Outcome: | The proposed methods achieve 92% user satisfaction rate in a deployed chatbot system, reflecting an 18% improvement over the baseline. |
ESGenius: Benchmarking LLMs on Environmental, Social, and Governance (ESG) and Sustainability Knowledge (2025.emnlp-main)
Copied to clipboard
Chaoyue He, Xin Zhou, Yi Wu, Xinjia Yu, Yan Zhang, Lei Zhang, Di Wang, Shengfei Lyu, Hong Xu, Wang Xiaoqiao, Wei Liu, Chunyan Miao
| Challenge: | ESGenius is a comprehensive benchmark for evaluating Large Language Models on ESG and sustainability knowledge. |
| Approach: | They introduce ESGenius, a benchmark for evaluating and enhancing ESG proficiency . they use a rigorous two-stage evaluation protocol and a repository of foundational frameworks . |
| Outcome: | ESGenius is a benchmark for evaluating and enhancing the proficiency of Large Language Models (LLMs) in ESG and sustainability-focused question answering. |
Beyond Online Sampling: Bridging Offline-to-Online Alignment via Dynamic Data Transformation for LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Direct Preference Optimization (DPO) eliminates complex reward modeling in aligning large language models with human preferences, but its online variant faces significant efficiency bottlenecks due to costly real-time preference sampling and the reward model annotation. |
| Approach: | They propose a framework that transforms static datasets into dynamically adaptive equivalents without the need for an explicit reward model. |
| Outcome: | The proposed approach matches or exceeds the performance of a fully online DPO. |
Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models (2025.findings-emnlp)
Copied to clipboard
Haonan He, Yuchen Ren, Yining Tang, Ziyang Xu, Junxian Li, Minghao Yang, Di Zhang, Yuan Dong, Tao Chen, Shufei Zhang, Yuqiang Li, Nanqing Dong, Wanli Ouyang, Dongzhan Zhou, Peng Ye
| Challenge: | Biology-Instructions is the first large-scale instruction-tuning dataset for multi-omics biological sequences. |
| Approach: | They propose a large-scale instruction-tuning dataset for multi-omics biological sequences . they propose 'chatMultiOmics' to overcome limitations of current LLMs on multi-ome tasks . |
| Outcome: | The proposed dataset bridges LLMs and complex biological sequence-related tasks while maintaining conversational fluency. |
Enhancing Auto-regressive Chain-of-Thought through Loop-Aligned Reasoning (2026.eacl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought prompting is a powerful technique for enhancing language model’s reasoning capabilities, but generating long and correct CoT trajectories is challenging. |
| Approach: | They propose to align the steps of Chain-of-Thought reasoning with loop iterations and apply intermediate supervision during the training of Looped Transformers. |
| Outcome: | The proposed method generates accurate reasoning chains for complex problems exceeding training length, and improves performance of the auto-regressive model. |
AgentFactory: A Self-Evolving Framework Through Executable Subagent Accumulation and Reuse (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing frameworks for building LLM-based agents treat agent behavior as static-knowledge gained during execution is not preserved for future use. |
| Approach: | They propose a new paradigm that preserves successful task solutions as executable subagent code rather than textual experience. |
| Outcome: | The proposed agent-based agent-driven paradigm preserves successful tasks as executable subagent code rather than textual experience. |
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation (P19-3)
Copied to clipboard
Zhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Sachan, Eric Xing
| Challenge: | Texar is an open-source text generation toolkit that supports a broad set of text generation tasks. |
| Approach: | They introduce Texar, an open-source text generation toolkit that supports text generation tasks. |
| Outcome: | Texar supports machine translation, summarization, dialog, content manipulation, and more. |
Interaction-Aware Topic Model for Microblog Conversations through Network Embedding and User Attention (C18-1)
Copied to clipboard
| Challenge: | Existing topic models ignore that one discusses diverse topics when dynamically interacting with different people. |
| Approach: | They propose an Interaction-Aware Topic Model (IATM) for microblog conversations by integrating network embedding and user attention. |
| Outcome: | The proposed model is based on three real-world microblog datasets. |
TMFN: A Target-oriented Multi-grained Fusion Network for End-to-end Aspect-based Multimodal Sentiment Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for multimodal aspect-based sentiment analysis focus on fusing image regional information and textual words. |
| Approach: | They propose a multimodal aspect-based sentiment analysis method that integrates regional and global image information with global image data. |
| Outcome: | Experiments show that the proposed method outperforms state-of-the-art methods on two benchmark datasets. |
D-Artemis: A Deliberative Cognitive Framework for Mobile GUI Multi-Agents (2026.findings-acl)
Copied to clipboard
Hongze Mi, Yibo Feng, WenJie Lu, Yuqi Wang, Jinyuan Li, Song Cao, He Cui, Tengfei Tian, Xuelin Zhang, Haotian Luo, Di Sun, Jun Fang, Hua Chai, Naiqiang Tan, Gang Pan
| Challenge: | Graphical User Interface (GUI) agents aim to automate a wide spectrum of human tasks by emulating user interaction. |
| Approach: | They propose a deliberative framework that leverages a fine-grained tip retrieval mechanism to inform its decision-making process. |
| Outcome: | The proposed framework achieves SOTA among open-source general models on AndroidWorld and ScreenSpot-V2 . it leverages a fine-grained, app-specific tip retrieval mechanism to inform its decision-making process . |
RoleMRC: A Fine-Grained Composite Benchmark for Role-Playing and Instruction-Following (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing role-playing datasets mostly contribute to controlling role style and knowledge boundaries, but overlook role-following in instruction-follower scenarios. |
| Approach: | They propose a fine-grained role-playing and instruction-following composite benchmark, named RoleMRC, which includes multi-turn dialogues between ideal roles and humans, including free chats or discussions upon given passages . |
| Outcome: | The proposed model improves instruction-following without compromising general role-playing and reasoning capabilities. |
Dense Information Flow for Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Recent advances in deep neural networks have improved learning performance for NMT . Residual connections allow features from previous layers to be accumulated to the next layer easily. |
| Approach: | They propose a densely connected NMT architecture that can train more efficiently for NMT. |
| Outcome: | The proposed architecture improves learning performance and attention quality on multiple datasets. |
BERT-MK: Integrating Graph Contextualized Knowledge into Pre-trained Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge representation learning methods do not use graph contextualized knowledge. |
| Approach: | They propose to model subgraphs in a medical KG and integrate it with a pre-trained language model to do knowledge generalization. |
| Outcome: | The proposed model achieves state-of-the-art on several medical NLP tasks . it improves on MedERNIE, and the proposed model is effective . |
REST: Retrieval-Based Speculative Decoding (2024.naacl-long)
Copied to clipboard
| Challenge: | Retrieval-based speculative decoding (REST) is a new language model generation algorithm . it uses existing knowledge to generate draft tokens, allowing for seamless integration and acceleration of any language model. |
| Approach: | They propose a new algorithm that uses a draft language model to generate tokens from existing knowledge. |
| Outcome: | The proposed method achieves a speedup of 1.62 to 2.36 on code or text generation. |
Finding the Dominant Winning Ticket in Pre-Trained Language Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on pre-trained language models show that they can fine-tune parameters but achieve good downstream performance. |
| Approach: | They find that a dominant winning ticket takes up 0.05% of the parameters and is transferable across different tasks. |
| Outcome: | The proposed model can achieve comparable performance with the full-parameter model, the authors show . the dominant winning ticket takes up 0.05% of the parameters, and the model is transferable across tasks, they show - the authors conclude . |
Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies attributed verbosity to biased labels, but new research shows that DPO can be effective in mitigating verboses. |
| Approach: | They propose to use a method to reduce the amount of verbosity in LLMs by using a downsampling approach. |
| Outcome: | The proposed approach overcomes the problem of verbosity by reducing the length reliance of the proposed algorithm. |
SWE-Swiss: A Multi-Task Fine-Tuning and RL Recipe for High-Performance Issue Resolution (2026.findings-acl)
Copied to clipboard
Zhenyu He, Qingping Yang, Wei Shen, Xiaojian Zhong, Kechi Zhang, Chenxin An, Wenlei Shi, Tianle Cai, Di He, Jiaze Chen, Jingjing Xu
| Challenge: | SWE-Swiss-32B demonstrates strong generalization to other common LLM benchmarks. |
| Approach: | They propose a two-phase training recipe that decomposes issue resolution into three core skills: Localization, Repair, and Unit Test Generation. |
| Outcome: | The proposed model achieves a 60.2% score on the SWE-bench Verified benchmark and is in the top-tier performance bracket of much larger models. |
FIPO: Free-form Instruction-oriented Prompt Optimization with Preference Dataset and Modular Fine-tuning Schema (2025.coling-main)
Copied to clipboard
| Challenge: | naive prompts can enhance the task performance of large language models, but they are resource-intensive. |
| Approach: | They propose an automatic prompt optimization method that refines naive prompts according to task outputs from in-box testing models. |
| Outcome: | The proposed method is based on a large-scale dataset and performed fairly across multiple models. |
Implicit Discourse Relation Recognition using Neural Tensor Network with Interactive Attention and Sparse Learning (C18-1)
Copied to clipboard
| Challenge: | Existing methods for implicit discourse relation recognition ignore bidirectional interactions between two arguments and sparsity of pair patterns. |
| Approach: | They propose a neural Tensor network framework with interactive attention and sparse learning for implicit discourse relation recognition. |
| Outcome: | The proposed framework is effective on PDTB and can be used in text summarization, conversation system and so on. |
Robustness-Aware Word Embedding Improves Certified Robustness to Adversarial Word Substitutions (2023.findings-acl)
Copied to clipboard
| Challenge: | Embedding interval bound constraint is important for NLP models to be certified robust, but adversarial examples can be crafted by synonym substitutions. |
| Approach: | They propose a triplet loss to train robustness-aware word embeddings for better certified robustness. |
| Outcome: | The proposed method outperforms state-of-the-art certified defense baselines and generalizes well to unseen substitutions. |
Machine Translation With Weakly Paired Documents (D19-1)
Copied to clipboard
| Challenge: | Recent studies explore the possibility of unsupervised machine translation with monolingual data only. |
| Approach: | They propose a method to mine bilingual sentences from weakly paired documents . they use word distribution-level alignments to constrain word distributions of two weakly-paired documents. |
| Outcome: | The proposed method outperforms previous results on six translation tasks using weakly paired bilingual documents and a large number of bilingual sentences. |
Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation suffers from exposure bias and error propagation problem. |
| Approach: | They conduct a series of analyses to deeply understand the accuracy drop problem . they find that the left part of the translated sentence is often better than its right part . |
| Outcome: | The results show that the left part of the translated sentence is often better than its right part in left-to-right decoding models. |
KMatrix-2: A Comprehensive Heterogeneous Knowledge Collaborative Enhancement Toolkit for Large Language Model (2025.emnlp-demos)
Copied to clipboard
Shun Wu, Di Wu, Wangtao Sun, Ziyang Huang, Xiaowei Yuan, Kun Luo, XueYou Zhang, Shizhu He, Jun Zhao, Kang Liu
| Challenge: | Existing studies on K-LLMs systems focus on declarative knowledge and procedural knowledge (rules) . |
| Approach: | They propose to build a toolkit that supports comprehensive heterogeneous knowledge collaborative enhancement for Large Language Models (LLMs). |
| Outcome: | The proposed toolkit provides unified knowledge integration and joint knowledge retrieval methods to achieve more comprehensive heterogeneous knowledge collaborative enhancement. |
Hint-Based Training for Non-Autoregressive Machine Translation (D19-1)
Copied to clipboard
| Challenge: | AutoRegressive Translation models have to generate tokens sequentially during decoding and thus suffer from high inference latency. |
| Approach: | They propose to use hidden states and word alignments to help train NART models. |
| Outcome: | The proposed model improves on the WMT14 En-De and De-En datasets but is faster in inference than the current models. |
Double Path Networks for Sequence to Sequence Learning (C18-1)
Copied to clipboard
| Challenge: | Existing approaches for Sequence to Sequence learning have been developed . convolutional neural networks and self-attention networks are the most popular . |
| Approach: | They propose to integrate convolutional and self-attention layers into a double path network for sequence to sequence learning. |
| Outcome: | The proposed method significantly improves performance over state-of-the-art systems. |
Ted-Tok: Maintaining an Evolving Vocabulary for Lifelong Learning (2026.acl-long)
Copied to clipboard
| Challenge: | a static tokenizer fragments newly emerging lexical items as language evolves . as language grows, a dynamic tokenizer reduces compression efficiency and performance . |
| Approach: | They propose a Temporal Drift Tokenizer that maintains an evolving vocabulary that adapts to emerging linguistic patterns over time. |
| Outcome: | The proposed tokenizer maintains an evolving vocabulary that adapts to emerging linguistic patterns over time. |
New Terms, New Toxicity: Consensus-based Chinese Neologism Toxicity Detection via Search-Augmented LLMs (2026.acl-long)
Copied to clipboard
Shiyao Cui, QingLin Zhang, Di Wang, Yida Lu, Zhexin Zhang, Jinhua Gao, Jinglin Yang, Min He, Han Qiu, Minlie Huang
| Challenge: | Neologisms can foster new linguistic consensus by stabilizing shared meanings and usage in common communicative norms. |
| Approach: | They propose a taxonomy that captures the origins and consensus-verification criteria of toxic neologisms . they propose 'SeTox' framework that integrates real-time web context for naeologim detection . |
| Outcome: | The proposed framework outperforms large-scale models in detecting neologism toxicity. |
Multilingual Neural Machine Translation with Language Clustering (D19-1)
Copied to clipboard
| Challenge: | Existing work on multilingual neural machine translation has been neglected due to its burdensome training process. |
| Approach: | They develop a framework that clusters languages into different groups and trains one multilingual model for each cluster. |
| Outcome: | The proposed model reduces the cost of training and improves translation accuracy. |
INT: Establishing Information Transfer for Multilingual Intent Detection and Slot Filling (2025.findings-acl)
Copied to clipboard
Di Wu, Liting Jiang, Bohui Mao, Hongyan Xie, Haoxiang Su, Zhongjiang He, Ruiyu Fang, Shuangyong Song, Hao Huang, Xuelong Li
| Challenge: | Existing studies struggle to achieve performance comparable to that on high-resource languages due to inherent linguistic diversity of multilingual SLU tasks. |
| Approach: | They propose a multilingual information transfer network to solve these challenges . they propose to reformulate SF as a span prediction problem and introduce a slot-matching attention mechanism to achieve slot alignment across languages. |
| Outcome: | The proposed model outperforms baseline models on the MASSIVE and MASSIV-UG datasets in overall accuracy across all languages. |
Matching Article Pairs with Graphical Decomposition and Convolutions (P19-1)
Copied to clipboard
| Challenge: | Existing methods for matching sentence pairs do not perform well in longer documents . Existing approaches for matching sentences do not work in longer document understanding tasks . |
| Approach: | They propose to model article pairs by comparing sentences that enclose same concept vertex . they propose to use a concept interaction graph to match articles by encoding sentences . |
| Outcome: | The proposed methods show significant improvements over existing methods . the proposed datasets consist of 30K pairs of breaking news articles . |
How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMs (2025.findings-acl)
Copied to clipboard
Guhao Feng, Kai Yang, Yuntian Gu, Xinyue Ai, Shengjie Luo, Jiacheng Sun, Di He, Zhenguo Li, Liwei Wang
| Challenge: | Despite the success of transformer-based large language models, understanding and enhancing their mathematical capabilities remains a significant challenge. |
| Approach: | They propose to use numerical precision as a key factor that influences LLMs' effectiveness in arithmetical tasks to determine their effectiveness. |
| Outcome: | The proposed models perform better in arithmetic tasks than transformer-based models with standard numerical precision. |