Papers by Jiawei Yuan
HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference (2025.findings-acl)
Copied to clipboard
Ping Gong, Jiawei Yi, Shengnan Wang, Juncheng Zhang, Zewen Jin, Ouxiang Zhou, Ruibo Liu, Guanbin Xu, Youhui Bai, Bowen Ye, Kun Yuan, Tong Yang, Gong Zhang, Renhai Chen, Feng Wu, Cheng Li
| Challenge: | Existing top-k attention methods struggle to strike a balance between efficiency and accuracy. |
| Approach: | They propose a top-k attention approach that integrates low-overhead techniques into the Top-k Attention process to achieve 7.2 speedup compared to vanilla full attention. |
| Outcome: | The proposed approach achieves 7.2 speedup compared to current top-k attention methods while maintaining model accuracy. |
MemSearcher: Iterative Memory Integration for Search Agent via End-to-End Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Qianhao Yuan, Jie Lou, Zichao Li, Jiawei Chen, Yaojie Lu, Hongyu Lin, Le Sun, Debing Zhang, Xianpei Han
| Challenge: | Recent LLM-based search agents often concatenate the full interaction history into the context, producing long and noisy inputs and increasing compute cost and memory overhead. |
| Approach: | They propose an agent framework that maintains a compact memory during multi-turn interactions. |
| Outcome: | The proposed framework outperforms strong history-concatenation (ReAct-style) baselines on a range of public datasets while maintaining nearly constant token counts across multi-turn interactions. |
TokenPenalty: Alleviating Attention Sinks and Positional Decay in LVLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal large language models (MLLMs) often hallucinate due to two relevant phenomena: massive activation phenomenon and positional information decay. |
| Approach: | They propose a token-level intervention strategy that dynamically suppresses irrelevant visual tokens while preserving key contextual signals. |
| Outcome: | Experiments show that TokenTruth significantly improves factual consistency across MLLMs on standard image understanding benchmarks. |
ConsistentChat: Building Skeleton-Guided Consistent Multi-Turn Dialogues for Large Language Models from Scratch (2025.emnlp-main)
Copied to clipboard
Jiawei Chen, Xinyan Guan, Qianhao Yuan, Mo Guozhao, Weixiang Zhou, Yaojie Lu, Hongyu Lin, Ben He, Le Sun, Xianpei Han
| Challenge: | Existing instruction data synthesis methods focus on single-turn instructions and neglect cross-turn coherence, resulting in context drift and reduced task completion rates. |
| Approach: | They propose a framework that constrains multi-turn instruction synthesis by explicitly modeling human conversational intent. |
| Outcome: | The proposed framework outperforms existing models trained on single-turn and multi-turn instruction datasets. |
Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting can mitigate hallucinations by encouraging step-by-step reasoning, but its impact on halluciation detection remains underexplored. |
| Approach: | They conduct an empirical evaluation of CoT prompting in Large Language Models (LLMs) to examine their impact on hallucination detection methods. |
| Outcome: | The proposed method significantly affects the internal states and token probability distributions of the LLM. |
PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models (LMs) are a powerful transfer learning approach for knowledge graph (KG) completion. |
| Approach: | They propose a parameter-lite transfer learning approach for pretrained language models for knowledge graph (KG) completion. |
| Outcome: | The proposed model outperforms the state-of-the-art models on a knowledge graph completion benchmark by tuning 1% of the parameters. |
Diffusion Based Counterfactual Augmentation for Dual Sentiment Classification (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to augment sentiment models have failed to mitigate spurious association problem inherent in the original data. |
| Approach: | They propose a framework for enhancing sentiment models using an antonymous paradigm and contrastive learning to generate high-quality samples. |
| Outcome: | The proposed framework achieves state-of-the-art performance on four benchmark datasets. |
CoTD-PO: Chain-of-Thought Distillation with Preference Optimization (2025.findings-emnlp)
Copied to clipboard
Lujie Niu, Haochen Sun, Fangkun Zhao, Sheng Chen, Zimeng Bai, Jiawei Zhang, Caixia Yuan, Xiaojie Wang
| Challenge: | Existing methods for chain-of-thought distillation suffer from a distribution mismatch between teacher-generated training trajectories and the student model's own generative distribution. |
| Approach: | They propose a framework that shifts the training paradigm from passive imitation to active trajectory exploration by allowing students to sample their own answer paths. |
| Outcome: | The proposed method outperforms standard CoT distillation baselines while mitigating mode collapse and preserving semantic diversity. |
D-RAG: Differentiable Retrieval-Augmented Generation for Knowledge Graph Question Answering (2025.emnlp-main)
Copied to clipboard
Guangze Gao, Zixuan Li, Chunfeng Yuan, Jiawei Li, Wu Jianzhuo, Yuehao Zhang, Xiaolong Jin, Bing Li, Weiming Hu
| Challenge: | Existing approaches to Knowledge Graph Question Answering (KGQA) use Retrieval-Augmented Generation (RAG) but subgraph selection process is non-differentiable, preventing end-to-end training of the retriever and the generator. |
| Approach: | They propose a Differentiable RAG approach that optimizes the retriever and the generator for KGQA. |
| Outcome: | The proposed approach outperforms state-of-the-art approaches on WebQSP and CWQ. |
MedCoT: Medical Chain of Thought via Hierarchical Expert (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for medical visual question answering lack robustness and reasoning paths for real-world medical diagnostics. |
| Approach: | They propose a hierarchical expert verification reasoning chain method to enhance interpretability and accuracy in medical visual question answering. |
| Outcome: | The proposed method outperforms existing methods on four standard Med-VQA datasets. |
Generative Data Augmentation with Contrastive Learning for Zero-Shot Stance Detection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for zero-shot stance detection are labor-intensive to train for each new target. |
| Approach: | They propose a generative data augmentation approach to generate training samples containing unseen and seen targets and map them into the same embedding space with contrastive learning. |
| Outcome: | The proposed model achieves state-of-the-art on most topics in the task of zero-shot stance detection. |
Ro-SLM: Onboard Small Language Models for Robot Task Planning and Operation Code Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) provide robots with contextual reasoning abilities to comprehend human instructions. |
| Approach: | They propose a framework that enables reliable SLM-driven robot operation by distilling LLMs’ knowledge and reasoning. |
| Outcome: | The proposed framework enables reliable SLM-driven robot operation by distilling LLMs’ knowledge and reasoning. |
Shallow Focus, Deep Fixes: Enhancing Shallow Layers Vision Attention Sinks to Alleviate Hallucination in LVLMs (2025.emnlp-main)
Copied to clipboard
Xiaofeng Zhang, Yihao Quan, Chen Shen, Chaochen Gu, Xiaosong Yuan, Shaotian Yan, Jiawei Cao, Hao Cheng, Kaijie Wu, Jieping Ye
| Challenge: | Multimodal large language models (MLLMs) demonstrate excellent abilities for understanding visual information, but the hallucination remains a challenging problem. |
| Approach: | They propose a training-free approach to enhance vision attention sinks to facilitate convergence of the image token attention sink within shallow layers. |
| Outcome: | The proposed approach improves the convergence of the image token attention sink within shallow layers and strengthens the layer’s focus on the image itself. |