Papers by Xin Dai
DSP: Discriminative Soft Prompts for Zero-Shot Entity and Relation Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompt-based methods have shown their efficacy in transferring general knowledge within pre-trained language models (PLMs) however, when applied to zero-shot entity and relation extraction, they struggle with the limited coverage of verbalizers to labels and the slow inference speed. |
| Approach: | They propose a method which reformulates zero-shot tasks into token discrimination tasks without having to construct verbalizers. |
| Outcome: | The proposed method outperforms baselines on two zero-shot entity recognition datasets with higher inference speed and achieves 7.5% improvement over previous state-of-the-art models on Wiki-ZSL and FewRel. |
Scaling Laws for Code: Every Programming Language Matters (2026.findings-acl)
Copied to clipboard
Jian Yang, Shuyue Guo, Linzheng Chai, Wei Zhang, Aishan Liu, Chuan Hao, Zhoujun Li, Xin Zhao, Xianglong Liu, Weifeng Lv, Bryan Dai
| Challenge: | Existing studies focus on language-agnostic settings, neglecting the inherently multilingual nature of modern software development. |
| Approach: | They propose a proportion-dependent scaling law that prioritizes high-utility languages . they propose PLs to have varying effects during pre-training that affect model performance . |
| Outcome: | The proposed scaling law is based on 1000+ experiments across multiple languages and models. |
BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attacks (2026.acl-long)
Copied to clipboard
| Challenge: | Existing supervised defense methods rely on labeled malicious agents to train a supervised model of malicious behavior. |
| Approach: | They propose an unsupervised defense method that learns without requiring any attack-specific labels or prior knowledge of malicious behaviors. |
| Outcome: | The proposed method detects diverse attack types across MAS with various communication patterns while maintaining superior generalizability compared to baselines. |
mCLIP: Multilingual CLIP via Cross-lingual Transfer (2023.acl-long)
Copied to clipboard
Guanhua Chen, Lu Hou, Yun Chen, Wenliang Dai, Lifeng Shang, Xin Jiang, Qun Liu, Jia Pan, Wenping Wang
| Challenge: | Existing multilingual vision-language pretrained models are biased towards English due to the lack of sufficient non-English image-text pairs. |
| Approach: | They propose to train a retrieval-efficient dual-stream multilingual VLP model by aligning CLIP model and a multilingual text encoder through a novel Triangle Cross-modal Knowledge Distillation method. |
| Outcome: | Empirical results show that mCLIP achieves new state-of-the-art performance for both zero-shot and finetuned multilingual image-text retrieval tasks. |
mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval (2024.emnlp-industry)
Copied to clipboard
Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li, Min Zhang
| Challenge: | Existing models for text retrieval are based on a multi-stage process that involves retrieving documents from a large corpus. |
| Approach: | They propose to build a multilingual text representation model and a cross-encoder reranker from scratch for text retrieval. |
| Outcome: | The proposed models outperform the state-of-the-art models on long-context retrieval benchmarks. |
Mitigate Extrinsic Social Bias in Pre-trained Language Models via Continuous Prompts Adjustment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods of extrinsic bias mitigation rely on manual word lists for sensitive groups . however, these word lists are limited by length and scope, resulting in poor performance. |
| Approach: | They propose a method which generates continuous token lists from the entire vocabulary space and uses them to bridge the gap between outputs and targets in fairness learning process. |
| Outcome: | The proposed method outperforms baseline methods on three NLU tasks. |
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling (2024.acl-long)
Copied to clipboard
Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, Hang Yan, Jie Fu, Tao Gui, Tianxiang Sun, Yu-Gang Jiang, Xipeng Qiu
| Challenge: | Existing language models that use discrete representations for unified processing of various modalities are limited to text generation and do not include multimodal output. |
| Approach: | They propose a multimodal language model that utilizes discrete representations for unified processing of various modalities. |
| Outcome: | The proposed model can be trained stably without any alterations to existing models or training paradigms. |
Revealing the Attention Floating Mechanism in Masked Diffusion Models (2026.findings-acl)
Copied to clipboard
Xin Dai, Pengcheng Huang, Zhenghao Liu, Shuo Wang, Yukun Yan, Chaojun Xiao, Yu Gu, Ge Yu, Maosong Sun
| Challenge: | Masked diffusion models (MDMs) leverage bidirectional attention and a denoising process. |
| Approach: | They investigate the attention behaviors of Masked diffusion models by revealing the phenomenon of Attention Floating. |
| Outcome: | The proposed model doubles the performance of autoregressive models in knowledge-intensive tasks. |
Interpretable and Low-Resource Entity Matching via Decoupling Feature Learning from Decision Making (2021.acl-long)
Copied to clipboard
| Challenge: | Entity Matching (EM) aims at recognizing entity records that denote the same real-world object. |
| Approach: | They propose a novel EM framework that consists of Heterogeneous Information Fusion and Key Attribute Tree Induction to decouple feature representation from matching decision. |
| Outcome: | The proposed framework outperforms SOTA EM models on 6 public datasets and 3 industrial datasets. |
A Lifelong Multilingual Multi-granularity Semantic Alignment Approach via Maximum Co-occurrence Probability (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to mask and predict tokens in multilingual text limit multilingual interaction . |
| Approach: | They propose a lifelong multilingual multi-granularity semantic alignment approach which continuously extracts massive aligned linguistic units from noisy data via a maximum co-occurrence probability algorithm. |
| Outcome: | The proposed approach improves translation performance on WMT14 18 benchmarks in twelve directions. |
A Chinese Corpus for Fine-grained Entity Typing (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing datasets for fine-grained entity typing are limited to English . a corpus of 4,800 mentions is manually labeled with free-form entity types . |
| Approach: | They propose a Chinese fine-grained entity typing task that uses crowdsourcing . they categorize each mention into 10 general types and use a large tag set to predict open set of types . |
| Outcome: | The proposed dataset contains 4,800 mentions manually labeled in Chinese . it also categorizes all the fine-grained types into 10 general types . |
Demystifying Data Organization for Enhanced LLM Training (2026.acl-long)
Copied to clipboard
Yalun Dai, Yangyu Huang, Tongshen Yang, Yonghan Wang, Xin Zhang, Wenshan Wu, Qihao Zhao, Hao Li, Yuanyuan Gao, Kim-Hui Yap, Scarlett Li
| Challenge: | Large Language Models (LLMs) have revolutionized various fields, yet their training efficiency is heavily reliant on effective data curation. |
| Approach: | They propose to reuse pre-computed sample-level scores originally generated for data efficiency and introduce two new data ordering methods to improve LLM training. |
| Outcome: | The proposed methods improve the stability and performance of LLM training. |
Improving Fine-grained Entity Typing with Entity Linking (D19-1)
Copied to clipboard
| Challenge: | Existing methods for fine-grained entity typing require a large tag set and knowledge of the context. |
| Approach: | They propose a deep neural model that uses context and information from entity linking to improve fine-grained entity typing. |
| Outcome: | The proposed model achieves 5% absolute strict accuracy improvement over the state of the art on two datasets. |
Is Multi-Hop Reasoning Really Explainable? Towards Benchmarking Reasoning Interpretability (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for multi-hop reasoning are not able to evaluate their interpretability . a recent study found that many paths are unreasonable . |
| Approach: | They propose a framework to evaluate the interpretability of multi-hop reasoning models . they annotate all possible rules and establish a benchmark . |
| Outcome: | The proposed framework outperforms existing models in terms of performance and interpretability. |
RecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing memory systems invoke LLMs to extract episodic and semantic memory, and this leads to substantial token consumption. |
| Approach: | They propose a method that stores incoming interactions in a subconscious memory layer and encodes them using lightweight embedding models for retrieval. |
| Outcome: | Experiments show that RecMem reduces the memory construction token cost of three SOTA memory systems by up to 87% while exceeding their accuracy. |
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent large-scale vision-language pre-training models are powerful in multimodal classification and retrieval tasks. |
| Approach: | They propose to augment a vision-language pre-training model with a textual pre-trained language model . the model achieves 44.5% zero-shot accuracy on multimodal generation tasks . |
| Outcome: | The proposed model achieves 44.5% zero-shot accuracy on open-ended visual question answering and image captioning tasks. |
SQUIRE: A Sequence-to-sequence Framework for Multi-hop Knowledge Graph Reasoning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for multi-hop knowledge graph reasoning suffer from slow and poor convergence . a transformer model can be used to learn and predict in an end-to-end fashion, giving faster convergence compared to previous methods . |
| Approach: | They propose a Sequence-to-sequence based multi-hop reasoning framework . it uses an encoder-decoder transformer structure to translate the query to a path . |
| Outcome: | The proposed framework can learn and predict in an end-to-end fashion, which gives better and faster convergence. |
On Domain-Adaptive Post-Training for Multimodal Large Language Models (2025.findings-emnlp)
Copied to clipboard
Daixuan Cheng, Shaohan Huang, Ziyu Zhu, Xintong Zhang, Xin Zhao, Zhongzhi Luan, Bo Dai, Zhenliang Zhang
| Challenge: | Adapting general multimodal large language models to specific domains is important for practical applications. |
| Approach: | They investigate domain adaptation of multimodal large language models via post-training . they develop a generate-then-filter pipeline that curates diverse visual instruction tasks . |
| Outcome: | The proposed model outperforms existing models in domain adaptation by combining data from open-source models with training pipelines. |
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models (2026.acl-long)
Copied to clipboard
Xin Cheng, Wangding Zeng, Damai Dai, Qinyu Chen, Bingxuan Wang, Zhenda Xie, Kezhao Huang, Xingkai Yu, Zhewen Hao, Han Zhang, Yu-Kun Li, Huishuai Zhang, Dongyan Zhao, Wenfeng Liang
| Challenge: | Mixture-of-Experts (MoE) scales capacity via conditional computation, but lacks knowledge lookup primitive. |
| Approach: | They propose a conditional memory instantiated via Deep Sparse Embedding (DSE) they propose 'u-shaped scaling law' that identifies optimal balance between MoE experts and DSE memory . |
| Outcome: | The proposed model outperforms an iso-parameter and isoFLOPs MoE baseline across knowledge and reasoning benchmarks and is infrastructure-efficient. |
SAFE-QAQ: End-to-End Slow-Thinking Audio-Text Fraud Detection via Reinforcement Learning (2026.acl-long)
Copied to clipboard
Peidong Wang, Zhiming Ma, Xin Dai, YongKang Liu, Shi Feng, Xiaocui Yang, Wenxing Hu, Zhihao Wang, Mingjun Pan, Li Yuan, Daling Wang
| Challenge: | Existing methods for fraud detection rely on transcribed text, lacking acoustic cues . a proposed framework for audio-based slow-thinking fraud detection eliminates transcription errors . |
| Approach: | They propose a framework for audio-based slow-thinking fraud detection that eliminates transcription errors and rewards slow-thought reasoning by capturing fine-grained audio details. |
| Outcome: | The proposed method improves accuracy, inference efficiency, and real-time processing capabilities. |
Instruction-Tuning Data Synthesis from Scratch via Web Reconstruction (2025.findings-acl)
Copied to clipboard
Yuxin Jiang, Yufei Wang, Chuhan Wu, Xinyi Dai, Yan Xu, Weinan Gan, Yasheng Wang, Xin Jiang, Lifeng Shang, Ruiming Tang, Wei Wang
| Challenge: | Existing methods for generating and curating high-quality instruction-tuning data rely heavily on the quality of seed data or strong assumptions about the structure and content of web documents. |
| Approach: | They propose a fully automated framework for synthesizing high-quality instruction-tuning (IT) data directly from raw web documents with minimal assumptions. |
| Outcome: | The proposed framework outperforms state-of-the-art baselines by 16.65% across four instruction-following benchmarks. |
Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systems (2025.emnlp-main)
Copied to clipboard
| Challenge: | Empirical studies for communication topology design often overlook why and when sparse and dense topologies help or hinder collaboration. |
| Approach: | They propose a topology design approach that balances error suppression and beneficial information propagation by fusing connectivity patterns from dense and sparse graphs. |
| Outcome: | The proposed topology design achieves superior performance across tasks with sparse and dense graphs. |
Towards Text-Image Interleaved Retrieval (2025.acl-long)
Copied to clipboard
Xin Zhang, Ziqi Dai, Yongqi Li, Yanzhao Zhang, Dingkun Long, Pengjun Xie, Meishan Zhang, Jun Yu, Wenjie Li, Min Zhang
| Challenge: | Existing multimodal information retrieval models rely on single-image inputs . current models use a dense retrieval paradigm, but this approach is not effective . |
| Approach: | They propose a text-image interleaved retrieval task where query and document are interleaves . they adapt off-the-shelf retrievers and build a dense baseline by interleaded multimodal large language model . |
| Outcome: | The proposed model achieves significant improvements over the baseline by substantially fewer visual tokens. |