Papers by Dun Zhang
An Empirical Study of Position Bias in Modern Information Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new evaluation framework is used to assess the extent and impact of position bias in information retrieval. |
| Approach: | They introduce a position-aware retrieval benchmark and a diagnostic metric to quantify position bias . they compare models with BM25, dense embedding models, ColBERT-style late-interaction models . |
| Outcome: | The proposed framework evaluates retrieval models for position bias from a worst-case perspective. |
One Battle After Another: Probing LLMs’ Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for instruction-following in multi-topic dialogues are limited to a fixed number of turns, susceptible to saturation and failing to account for users’ interactive experience. |
| Approach: | They propose a framework featuring a three-layer tracking mechanism and a query synthesis agent to mimic sequential user behaviors. |
| Outcome: | The proposed framework outperforms existing benchmarks in the evaluation of instruction following in multi-topic dialogues and demonstrates deficiencies in failure recovery and fine-grained instruction following. |
Flaming-hot Initiation with Regular Execution Sampling for Large Language Models (2025.findings-naacl)
Copied to clipboard
Weizhe Chen, Zhicheng Zhang, Guanlin Liu, Renjie Zheng, Wenlei Shi, Chen Dun, Zheng Wu, Xing Jin, Lin Yan
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities across various domains since the release of ChatGPT . a key challenge in developing these general capabilities is efficiently sourcing diverse, high-quality data. |
| Approach: | They introduce Flaming-hot Initiation with Regular Execution (FIRE) sampling to efficiently find good responses by promoting diversity. |
| Outcome: | The proposed method enhances inference-time generation quality and benefits training in the alignment stage. |
Graph-Reward-SQL: Execution-Free Reinforcement Learning for Text-to-SQL via Graph Matching and Stepwise Reward (2025.findings-emnlp)
Copied to clipboard
Han Weng, Puzhen Wu, Cui Longjie, Yi Zhan, Boyi Liu, Yuanfeng Song, Dun Zeng, Yingxiang Yang, Qianru Zhang, Dong Huang, Xiaoming Yin, Yang Sun, Xing Chen
| Challenge: | Existing methods to enhance performance of large language models (LLMs) on Text-to-SQL tasks rely on execution-based or LLM-based reward models. |
| Approach: | They propose a reward model framework for RL-based Text-to-SQL that employs the GMNScore outcome reward model. |
| Outcome: | The proposed reward model outperforms existing reward models on standard benchmarks including Spider and BIRD. |
Noise Robust Named Entity Understanding for Voice Assistants (2021.naacl-industry)
Copied to clipboard
Deepak Muralidharan, Joel Ruben Antony Moniz, Sida Gao, Xiao Yang, Justine Kao, Stephen Pulman, Atish Kothari, Ray Shen, Yinying Pan, Vivek Kaul, Mubarak Seyed Ibrahim, Gang Xiang, Nan Dun, Yidan Zhou, Andy O, Yuan Zhang, Pooja Chitkara, Xuan Wang, Alkesh Patel, Kushal Tayal, Roger Zheng, Peter Grasch, Jason D Williams, Lin Li
| Challenge: | Named Entity Recognition and Entity Linking are challenging for voice assistants . utterances are relatively short, so there is not much context to help disambiguate . |
| Approach: | They propose a Named Entity Understanding system that combines NER and EL in a joint reranking module. |
| Outcome: | The proposed framework improves NER accuracy by up to 3.13% and EL accuracy by 3.6% in F1 score . it also leads to better accuracies in other natural language understanding tasks . |