Papers by Yujia Zhang
L2Dir: Integrating L_2-Norm and Directional Alignment for Unsupervised Contrastive Representation Learning in Multimodal Retrieval (2026.acl-long)
Copied to clipboard
Tianyu Zong, Rui Dai, Hongzhu Yi, Yuanxiang Wang, Zhenghao Zhang, Zhenyu Guan, Yujia Yang, Bingkang Shi, Yueyang Ding, Xiangxiang Chu, Kaikui Liu, Jungang Xu
| Challenge: | Existing approaches to multimodal representation learning focus on directional alignment and embedding magnitudes (L2-norm) however, these methods often fail to account for the intrinsic role of L2-norm in the contrastive process. |
| Approach: | They propose a plug-and-play framework that optimizes L2-norm alignment and Directional consistency jointly. |
| Outcome: | The proposed framework achieves consistent and significant performance gains over established baselines across 95 tasks using UniIR and VLM2Vec-V2 frameworks. |
Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse Autoencoders (2026.acl-long)
Copied to clipboard
Xiangchen Song, Aashiq Muhamed, Yujia Zheng, Lingjing Kong, Zeyu Tang, Mona T. Diab, Virginia Smith, Kun Zhang
| Challenge: | Sparse Autoencoders (SAEs) are a tool in mechanistic interpretability (MI) but the aspiration to identify a canonical set of features is challenged by the observed inconsistency of learned SAE features across different training runs. |
| Approach: | They propose to use the Pairwise Dictionary Mean Correlation Coefficient to quantify SAE feature consistency as an evaluation axis alongside reconstruction and sparsity. |
| Outcome: | The proposed measure is based on the pairwise dictionary mean correlation coefficient (PW-MCC) on LLM activations. |
RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation (2024.emnlp-demo)
Copied to clipboard
Qinyu Luo, Yining Ye, Shihao Liang, Zhong Zhang, Yujia Qin, Yaxi Lu, Yesai Wu, Xin Cong, Yankai Lin, Yingli Zhang, Xiaoyin Che, Zhiyuan Liu, Maosong Sun
| Challenge: | Xia et al., 2018) demonstrate that a large language model can generate and maintain high-quality code documentation. |
| Approach: | They propose a large language model powered open-source framework for generating, maintaining, and updating code documentation. |
| Outcome: | The proposed framework generates high-quality documentation for the entire project. |
Knowledge Inheritance for Pre-trained Language Models (2022.naacl-main)
Copied to clipboard
Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
| Challenge: | Existing large-scale pre-trained language models are mainly trained from scratch individually, ignoring that many well-taught PLMs are available. |
| Approach: | They propose a pre-training framework called knowledge inheritance and propose auxiliary supervision to efficiently learn larger PLMs. |
| Outcome: | The proposed framework can be used to train large-scale language models with huge parameters and a large dataset can be adapted to domain adaptation and knowledge transfer. |
An Element is Worth a Thousand Words: Enhancing Legal Case Retrieval by Incorporating Legal Elements (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for legal case retrieval lack the definition of relevance for legal cases . however, the definition goes beyond the common semantic relevance of ad-hoc retrieval. |
| Approach: | They propose a legal element dataset that incorporates legal elements into a semi-automatic method . they propose two models to enhance legal search using legal elements . |
| Outcome: | The proposed models outperform existing methods in enhancing legal search using legal elements. |
LLM Agents in Law: Taxonomy, Applications, and Challenges (2026.acl-long)
Copied to clipboard
Shuang Liu, Ruijia Zhang, Ruoyun Ma, Yujia Deng, Lanyi Zhu, Jiayu Li, Zelong Li, Zhibin Shen, Mengnan Du
| Challenge: | Large language models (LLMs) have improved the legal domain, but deployment of standalone models faces significant limitations regarding hallucination, outdated information, and verifiability. |
| Approach: | They present a survey of LLM agents for legal tasks and analyze their architectures . they analyze the transition from standard legal LLMs to legal agents . |
| Outcome: | The proposed architectures bridge the gap between technical capabilities and domain-specific needs. |
Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents (2024.acl-long)
Copied to clipboard
Cheng Qian, Bingxiang He, Zhong Zhuang, Jia Deng, Yujia Qin, Xin Cong, Zhong Zhang, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Current language model-driven agents lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions. |
| Approach: | They propose a benchmark to inspect users’ implicit intentions through explicit queries and a model expert as the upstream in agent design to enhance user-agent interaction. |
| Outcome: | The proposed approach excels at identifying vague user tasks, recovering and summarizing critical missing information, setting precise and necessary agent execution goals, and minimizing redundant tool usage, thus boosting overall efficiency. |
Incomplete In-context Learning (2026.acl-long)
Copied to clipboard
Wenqiang Wang, Wen Yujia, Yan Xiao, Zhifeng Chen, Yangshijie Zhang, Peng Chen, Mingbo Yang, Xiaochun Cao
| Challenge: | Existing in-context learning assumes the retrieval dataset contains demonstrations for all output label spaces. |
| Approach: | They propose a framework with train-free and train-based variants to address IICL . they propose to integrate a dataset with labeled demonstrations for each output space . |
| Outcome: | The proposed framework outperforms existing methods under incomplete retrieval datasets and even outperformed ICL with complete labels. |
How Do Your Code LLMs perform? Empowering Code Instruction Tuning with Really Good Data (2024.emnlp-main)
Copied to clipboard
Yejie Wang, Keqing He, Dayuan Fu, Zhuoma GongQue, Heyang Xu, Yanxu Chen, Zhexu Wang, Yujia Fu, Guanting Dong, Muxi Diao, Jingang Wang, Mengdi Zhang, Xunliang Cai, Weiran Xu
| Challenge: | Recent research has shown that code pre-trained models improve coding capabilities. |
| Approach: | They propose a code data pruning strategy to identify which datasets are high-quality code instruction data. |
| Outcome: | The proposed model achieves state-of-the-art performance using fewer training data. |
ProPy: Building Interactive Prompt Pyramids upon CLIP for Partially Relevant Video Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for PRVR use unimodal features, but powerful pretrained vision-language models like CLIP are underexplored. |
| Approach: | ProPy is a model with systematic architectural adaptation of CLIP specifically designed for PRVR. |
| Outcome: | ProPy outperforms existing models on three public datasets in terms of performance on the datasets. |
Search-o1: Agentic Search-Enhanced Large Reasoning Models (2025.emnlp-main)
Copied to clipboard
Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, Zhicheng Dou
| Challenge: | Large reasoning models (LRMs) have demonstrated impressive long stepwise reasoning capabilities through large-scale reinforcement learning. |
| Approach: | They propose a framework that enhances large reasoning models with an agentic retrieval-augmented generation mechanism and a Reason-in-Documents module for refining retrieved documents. |
| Outcome: | The proposed framework enhances LRMs with an agentic retrieval-augmented generation mechanism and Reason-in-Documents module for refining retrieved documents. |
Augmenting Multi-Agent Communication with State Delta Trajectory (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multi-agent systems based on large language models (LLMs) have shown to be effective in downstream tasks. |
| Approach: | They propose a protocol that transfers both natural language tokens and token-wise state transition trajectory from one agent to another. |
| Outcome: | The proposed protocol can transfer both natural language tokens and token-wise state transition trajectory from one agent to another. |
TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation (2026.findings-acl)
Copied to clipboard
Bangde Du, Minghao Guo, Songming He, Ziyi Ye, Xi Zhu, Weihang Su, Shuqi Zhu, Yujia Zhou, Yongfeng Zhang, Qingyao Ai, Yiqun Liu
| Challenge: | Existing studies show that advanced LLMs produce text indistinguishable from human writing. |
| Approach: | They propose a benchmark to assess persona simulation across diverse contexts by decomposing the evaluation into six fundamental capabilities including opinion consistency, memory recall, logical reasoning, persona tone, and syntactic style. |
| Outcome: | The proposed model achieves moderate accuracy but falls short of the basic capabilities needed to simulate personas in real-world contexts. |
Observations and Remedies for Large Language Model Bias in Self-Consuming Performative Loop (2026.acl-long)
Copied to clipboard
| Challenge: | Existing synthetic training loops for large language models cause performance drops and induce emerging biases . a large amount of generated content is posted to coding platforms, social media platforms and other platforms on the internet . |
| Approach: | They propose a self-consuming retraining loop where models are trained on their own outputs . they use a control loop to isolate and analyze feedback-driven bias evolution . |
| Outcome: | The proposed model increases preference bias and decreases disparate bias. |
MIND: Towards Immersive Psychological Healing with Multi-Agent Inner Dialogue (2025.findings-emnlp)
Copied to clipboard
Yujia Chen, Changsong Li, Yiming Wang, Tianjie Ju, Qingqing Xiao, Nan Zhang, Zifan Kong, Peng Wang, Binyu Yan
| Challenge: | Mental health issues are worsening in today’s competitive society, such as depression and anxiety. |
| Approach: | They propose a multi-agent inner dialogue paradigm that provides more immersive psychological healing environments. |
| Outcome: | The proposed paradigm provides more immersive psychological healing environments. |
Knowledge Graph Embedding with Atrous Convolution and Residual Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods are complex and require time for training and inference. |
| Approach: | They propose an atrous convolution based knowledge graph embedding method that increases feature interactions by using atrous . they evaluate method on six benchmark datasets with different evaluation metrics . |
| Outcome: | The proposed method achieves better results on six benchmark datasets than state-of-the-art methods on most evaluation metrics. |
Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Direct Preference Optimization (DPO) is a widely used reinforcement learning from human feedback (RLHF) method across various domains. |
| Approach: | They propose an approach that automatically re-weights ambiguous content to reduce ambiguities by calculating semantic similarity from preference pairs. |
| Outcome: | The proposed approach outperforms state-of-the-art approaches in performance across multiple model scales and widely adopted benchmark datasets. |
Chinese Court Simulation with LLM-Based Agents System (2026.findings-acl)
Copied to clipboard
Kaiyuan Zhang, Jiaqi Li, Yueyue Wu, Haitao Li, Cheng Luo, Shaokun Zou, Yujia Zhou, Weihang Su, Yiqun Liu, Qingyao Ai
| Challenge: | Existing studies have neglected the systematic design and procedure evaluation of court simulations, which are critical to the credibility and usage of court simulators in practice. |
| Approach: | They propose a court simulation paradigm based on the real-world procedure structure of Chinese courts and a framework that focuses on both legal judgment prediction and court procedure analysis. |
| Outcome: | The proposed model outperforms judges and lawyers from the real trials in many aspects. |
ECOLA: Enhancing Temporal Knowledge Embeddings with Contextualized Language Representations (2023.findings-acl)
Copied to clipboard
Zhen Han, Ruotong Liao, Jindong Gu, Yao Zhang, Zifeng Ding, Yujia Gu, Heinz Koeppl, Hinrich Schütze, Volker Tresp
| Challenge: | Existing enhancement approaches cannot be applied to temporal knowledge graphs (tKGs) existing enhancement approaches assume knowledge embedding is time-independent, whereas entity embedded in tKG models evolves . |
| Approach: | They propose to use textual data to enhance temporal knowledge embedding by Enhanced Temporal Knowledge Embeddings with Contextualized Language Representations (ECOLA) to evaluate ECOLA, they introduce three new datasets for training and evaluation. |
| Outcome: | The proposed model significantly improves Hits@1 on the link prediction task. |
Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have focused on replacing the reward model in Reinforcement Learning with Human Feedback (RLHF) methods for Large Language Models (LLMs). |
| Approach: | They propose a self-supervised preference optimization framework that replaces the reward model with a preference loss and alignment loss to improve LLMs' ability to understand human preferences. |
| Outcome: | The proposed framework can be integrated with existing preference optimization methods and significantly boost their performance. |
ELLE: Efficient Lifelong Pre-training for Emerging Data (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained language models are typically trained with static data, ignoring that streaming data of various sources may continuously grow. |
| Approach: | They propose to use function preserved model expansion to expand existing PLM's width and depth to improve efficiency of knowledge acquisition. |
| Outcome: | The proposed model improves pre-training efficiency and performance over existing models on BERT and GPT. |
AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learning (2025.emnlp-main)
Copied to clipboard
Yujie Feng, Jian Li, Xiaoyu Dong, Pengfei Xu, Xiaohui Zhou, Yujia Zhang, Zexin Lu, Yasha Wang, Alan Zhao, Xu Chu, Xiao-Ming Wu
| Challenge: | Recent model merging-based methods struggle to effectively manage the trade-off between learning new knowledge and preventing catastrophic forgetting. |
| Approach: | They propose a model merging framework that utilizes learning and forgetting signals from the training trajectory to dynamically monitor the model’s training status. |
| Outcome: | The proposed framework achieves significant performance improvements over existing state-of-the-art methods on three CL benchmarks with various model sizes (from 770M to 13B). |
Mapping Long-term Causalities in Psychiatric Symptomatology and Life Events from Social Media (2024.naacl-long)
Copied to clipboard
Siyuan Chen, Meilin Wang, Minghao Lv, Zhiling Zhang, Juqianqian Juqianqian, Dejiyangla Dejiyangla, Yujia Peng, Kenny Zhu, Mengyue Wu
| Challenge: | Existing studies focus on the semantic content of social media posts, overlooking the evolving nature of mental disorders and symptoms. |
| Approach: | They extract causality between psychiatric symptoms and life events from social media posts and extract temporal attributes to improve diagnosis and treatment planning. |
| Outcome: | The extracted causality features improve diagnostic and treatment planning and improve performance in tasks such as depression and diagnosis point detection. |
Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)
Copied to clipboard
Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains. |
| Approach: | They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries. |
| Outcome: | The proposed system outperforms baselines in the open domain task-solving benchmark. |