Papers by Weihao Zeng
The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents (2026.acl-long)
Copied to clipboard
| Challenge: | a fundamental pillar of trustworthiness is calibration, which refers to an agent’s ability to express confidence that reliably reflects its actual performance. |
| Approach: | They propose a reinforcement learning framework that jointly optimizes task accuracy and calibration, supported by a holistic benchmark of reward designs. |
| Outcome: | The proposed framework improves calibration across tool types and shows that trained agents achieve superior calibration and exhibit robust generalization from local training environments to noisy web settings and to distinct domains such as mathematical reasoning. |
Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers (2026.findings-acl)
Copied to clipboard
Qingcheng Zeng, Yuheng Lu, Zeqi Zhou, Heli Qi, Puxuan Yu, Fuheng Zhao, Hitomi Yanaka, Weihao Xuan, Naoto Yokoya
| Challenge: | a new study examines the performance of code-switching IR in monolingual contexts . code-witching is a pervasive linguistic phenomenon in global communication . |
| Approach: | They propose a benchmark to evaluate code-switching IR in monolingual contexts . they propose CS-MTEB, which measures performance declines of up to 27% . |
| Outcome: | The proposed benchmark shows that code-switching performance is degraded by 27% . the proposed benchmark is based on a dataset of mixed-language queries . |
FutureTOD: Teaching Future Knowledge to Pre-trained Language Model for Task-Oriented Dialogue (2023.acl-long)
Copied to clipboard
| Challenge: | Existing pre-trained language models rely on a contrastive framework and are difficult to use in practice. |
| Approach: | They propose a dialogue pre-training model which distills future knowledge to the representation of the previous dialogue context using a self-training framework. |
| Outcome: | The proposed model can be applied to various downstream dialogue tasks. |
Thinking Out Loud: Do Reasoning Models Know When They’re Right? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large reasoning models (LRMs) have recently demonstrated impressive capabilities in complex reasoning tasks by leveraging increased test-time computation and exhibiting behaviors reminiscent of human-like self-reflection. |
| Approach: | They analyze verbalized confidence, how models articulate their certainty, as a lens into the nature of self-reflection in large reasoning models. |
| Outcome: | The proposed model exhibits human-like self-reflection in reasoning tasks, but how this ability interacts with other model behaviors remains underexplored. |
Seen to Unseen: Exploring Compositional Generalization of Multi-Attribute Controllable Dialogue Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing controllable dialogue generation models focus on single attribute and lack generalization capability to out-of-distribution multiple attribute combinations. |
| Approach: | They propose a compositional generalization model that learns from seen attributes and generalizes to unseen combinations. |
| Outcome: | The proposed model can learn from seen attribute values and generalize to unseen combinations. |
DivTOD: Unleashing the Power of LLMs for Diversifying Task-Oriented Dialogue Representations (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing language models pre-trained on general text overlook the one-to-many property of task-oriented dialogues, where multiple responses can be appropriate given the same context. |
| Approach: | They propose a model that pre-trains LLMs to learn diverse task-oriented dialogue representations by removing domain knowledge that contradicts TODs. |
| Outcome: | The proposed model outperforms strong TOD baselines on various downstream dialogue tasks and learns the intrinsic diversity of task-oriented dialogues. |
The Pragmatic Mind of Machines: Tracing the Emergence of Pragmatic Competence in Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Current large language models (LLMs) have demonstrated emerging capabilities in social intelligence tasks, including implicature resolution and theory-of-mind reasoning. |
| Approach: | They introduce a dataset grounded in the pragmatic concept of alternatives to evaluate whether large language models can accurately infer nuanced speaker intentions. |
| Outcome: | The proposed model can infer nuanced speaker intentions by inferring the speaker’s intended meaning and explaining when and why a speaker would choose one utterance over its alternative. |
On the Perception Bottleneck of VLMs for Chart Understanding (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a perception bottleneck in large vision-language models is critical for chart understanding . instruction tuning improves the extraction capability of LVLMs, but the vision encoder remains a bottleneck . |
| Approach: | They propose to decompose the perception bottleneck into two components . the vision encoder bottleneck is where visual representation fails to encapsulate the correct information . |
| Outcome: | The proposed approach significantly mitigates the vision encoder bottleneck and improves the ability of LVLMs to comprehend charts. |
DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning (2024.acl-long)
Copied to clipboard
Yejie Wang, Keqing He, Guanting Dong, Pei Wang, Weihao Zeng, Muxi Diao, Weiran Xu, Jingang Wang, Mengdi Zhang, Xunliang Cai
| Challenge: | Numerous code large language models (LLMs) have been proposed to enhance code generation performance. |
| Approach: | They propose a diverse instruction model DolphCoder with self-evaluating for code generation that learns diverse instruction targets and combines a code evaluation objective to enhance its code generation ability. |
| Outcome: | The proposed model achieves superior performance on the HumanEval and MBPP benchmarks, demonstrating new insights for future code instruction tuning work. |
Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to quantify uncertainty are limited in vision-language models . however, current models display notable miscalibration across diverse tasks and settings . |
| Approach: | They evaluate verbalized confidence in vision-language models using visual reasoning . they propose a prompting strategy that improves confidence alignment in multimodal settings . |
| Outcome: | The proposed method improves confidence alignment across multimodal settings. |
MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)
Copied to clipboard
Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li
| Challenge: | Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities. |
| Approach: | They propose a comprehensive benchmark covering 29 languages, built on an English benchmark. |
| Outcome: | The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark. |
Domain-Oriented Prefix-Tuning: Towards Efficient and Generalizable Fine-tuning for Zero-Shot Dialogue Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for domain adaptation of abstractive dialogue summarization lack generalization ability on new domains. |
| Approach: | They propose a domain-oriented prefix-tuning model that uses a prefix module to alleviate domain entanglement and discrete prompts to guide the model to focus on key contents of dialogues. |
| Outcome: | The proposed model can be generalized to two multi-domain dialogue summarization datasets. |
Give the Truth: Incorporate Semantic Slot into Abstractive Dialogue Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Abstractive dialogue summarization suffers from a lot of factual errors due to scattered salient elements in multi-speaker information interaction process. |
| Approach: | They propose a slot-driven beam search algorithm to give priority to generating salient elements in a limited length by "filling-in-the-blanks". |
| Outcome: | The proposed algorithm improves the slot-driven beam search algorithm on different types of factual errors and human evaluation further verifies the results. |
BootTOD: Bootstrap Task-oriented Dialogue Representations by Aligning Diverse Responses (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models lack diversity and linguistic challenges in task-oriented dialogues. |
| Approach: | They propose a self-bootstrapping dialogue pre-training model called BootTOD . it learns task-oriented dialogue representations via a framework . |
| Outcome: | The proposed model outperforms strong TOD baselines on diverse dialogue tasks. |
Automatic Instruction Evolving for Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Evol-Instruct is an end-to-end framework that evolves instruction datasets without human effort. |
| Approach: | They propose an end-to-end framework that evolves instruction datasets without human effort by analyzing and analyzing evolutionary strategies for the given instruction data. |
| Outcome: | The proposed method outperforms human-designed methods on various benchmarks including MT-Bench, AlpacaEval, GSM8K, and HumanEval. |
MoSLD: An Extremely Parameter-Efficient Mixture-of-Shared LoRAs for Multi-Task Learning (2025.coling-main)
Copied to clipboard
| Challenge: | LoRA is a key technique for fine-tuning large pre-trained models, yet its performance in multi-task learning scenarios often falls short. |
| Approach: | They propose a mixture-of-shared-LoRAs model with a dropout strategy . they propose to share the upper projection matrix among different experts . |
| Outcome: | The proposed model exhibits excellent performance in both single-task and multi-task scenarios with robust out-of-domain generalization capabilities. |
How to Improve LLMs’ Performance on Specific Languages: A Perspective on LLM-Derived Language Similarity (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit uneven performance across languages. |
| Approach: | They propose to use a framework to quantify the similarity within each language pair through both the lenses of language-specific performance patterns and cross-lingual transferability. |
| Outcome: | The proposed approach outperforms traditional linguistic typology and cross-lingual transferability measures on multilingual LLMs. |
Alleviating Performance Degradation Caused by Out-of-Distribution Issues in Embedding-Based Retrieval (2025.findings-emnlp)
Copied to clipboard
Haotong Bao, Jianjin Zhang, Qi Chen, Weihao Han, Zhengxin Zeng, Ruiheng Chang, Mingzheng Li, Hao Sun, Weiwei Deng, Feng Sun, Qi Zhang
| Challenge: | Recent studies reveal query out-of-distribution issues degrading ANN performance . a distribution regularizer is introduced into the encoder training objective to encourage alignment between query and base embeddings. |
| Approach: | They introduce a distribution regularizer into the encoder training objective to encourage alignment between query and base embeddings. |
| Outcome: | The proposed method consistently improves retrieval performance across multiple datasets. |