Papers by Emine Yilmaz
A Survey on Asking Clarification Questions Datasets in Conversational Systems (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies on Asking Clarification Questions (ACQs) are incomparable due to inconsistent data, experimental setups and evaluation strategies. |
| Approach: | They analyse the current research status on Asking Clarification Questions (ACQs) and propose a set of evaluation metrics and benchmarks for multiple ACQs-related tasks. |
| Outcome: | The proposed techniques are compared with the available datasets and evaluated against benchmarks. |
Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to model the relations between domains and slots fail to address these issues and can be generalized to unseen domains. |
| Approach: | They propose a Dynamic Schema Graph Fusion Network which generates a dynamic schema graph to explicitly fuse prior slot-domain membership relations and dialogue-aware dynamic slot relations. |
| Outcome: | The proposed model outperforms existing methods on benchmark datasets showing that it can extract users' goals or intentions as dialogue states and keep them updated over the whole dialogue. |
Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewriting (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to rewrite context-dependent queries lack sufficient information for optimal retrieval performance. |
| Approach: | They propose to use large language models (LLMs) as query rewriters to generate informative queries through well-designed instructions. |
| Outcome: | The proposed approach improves performance on the QReCC dataset compared to human rewrites . |
PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistants (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing personalization benchmarks focus on chit-chat, non-conversational tasks, or narrow domains, failing to capture complexities of personalized task-oriented assistance. |
| Approach: | They propose a benchmark to evaluate personalization in task-oriented AI assistants . the benchmark features user profiles equipped with rich preferences and interaction histories . |
| Outcome: | The proposed benchmark features user profiles equipped with rich preferences and interaction histories . it also features a judge agent and user agent that employs the LLM-as-a-Judge paradigm . |
Improving Conversational Recommendation Systems via Bias Analysis and Language-Model-Enhanced Data Augmentation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Conversational Recommendation System (CRS) is a rapidly growing research area, along with advancements in language modelling techniques. |
| Approach: | They propose to use a benchmark dataset to develop CRS models and address biases arising from feedback loop inherent in multi-turn interactions to enhance model performance while mitigating biase. |
| Outcome: | The proposed strategies improve on ReDial and TG-ReDial benchmark datasets and offer additional insights on addressing multiple newly formulated biases. |
EventRAG: Enhancing LLM Generation with Event Knowledge Graphs (2025.acl-long)
Copied to clipboard
Zairun Yang, Yilin Wang, Zhengyan Shi, Yuan Yao, Lei Liang, Keyan Ding, Emine Yilmaz, Huajun Chen, Qiang Zhang
| Challenge: | Existing approaches to text generation often neglect event structures that shape real-world narratives. |
| Approach: | They propose a framework that integrates structured event semantics with iterative retrieval and inference to enhance text generation. |
| Outcome: | Experiments on UltraDomain and MultiHopRAG show that the proposed framework outperforms baseline RAG systems in generation effectiveness, logical consistency, and multi-hop reasoning accuracy. |
Rethinking Semi-supervised Learning with Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a popular setting to make use of unlabelled data . Currently, there are two popular approaches to make effective use of the unlabelled datasets . |
| Approach: | They compare semi-supervised learning (SSL) and task-adaptive pre-training (TAPT) they find TAPT is a stronger and more robust SSL learner, even when using just a few hundred unlabelled samples . |
| Outcome: | The proposed methods improve model performance across different NLP tasks and data sizes. |
Unsupervised Few-Bits Semantic Hashing with Implicit Topics Modeling (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for few-bits hashing cannot be guaranteed due to severe information loss. |
| Approach: | They propose a simple unsupervised neural generative semantic hashing method with a focus on few-bits hash. |
| Outcome: | The proposed method improves on the state-of-the-art methods in few-bits hashing. |
Modeling User Satisfaction Dynamics in Dialogue via Hawkes Process (2023.acl-long)
Copied to clipboard
| Challenge: | Existing estimators measure performance by user satisfaction but ignore satisfaction dynamics across turns. |
| Approach: | They propose to use user satisfaction estimation to estimate performance of dialogue systems by using an estimator to simulate users. |
| Outcome: | The proposed estimator outperforms existing estimators on four benchmark dialogue datasets. |
Clarifying the Path to User Satisfaction: An Investigation into Clarification Usefulness (2024.findings-eacl)
Copied to clipboard
| Challenge: | Poorly formulated questions can lead to user frustration and dissatisfaction . |
| Approach: | They propose to leverage key features that contribute to the classification of clarifying questions, enhancing user satisfaction and system performance. |
| Outcome: | The proposed model improves with a minimum performance boost of 45% in traditional classifiers, especially in large language models. |
Mitigating Context Interference for Reliable and Efficient Search Agents (2026.acl-long)
Copied to clipboard
Boyang Xue, Bin Wu, Shuofei Qiao, Sheng Wang, Rui Wang, Yiming Du, Hongru Wang, Jeff Z. Pan, Emine Yilmaz, Kam-Fai Wong, Aldo Lipani
| Challenge: | Recent research empowers Large Language Models (LLMs) as multi-turn search agents to iteratively retrieve and generate outputs until complex tasks are solved. |
| Approach: | They propose a distill-based context refiner to dynamically mitigate context interference . they also propose RLs that refine contexts to generate outputs . |
| Outcome: | The proposed refiner can mitigate context interference in multi-turn search agents. |
ASSIST: Towards Label Noise-Robust Dialogue State Tracking (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing versions of MultiWOZ 2.0 have been published, but there are still lots of noisy labels in the training set. |
| Approach: | They propose a framework to train dialogue state tracking models from noisy labels instead of improving annotation quality further by using auxiliary models. |
| Outcome: | The proposed framework improves the goal accuracy of DST models by 28.16% on MultiWOZ 2.0 and 8.41% on MultiWoz 2.4, compared to using only the vanilla noisy labels. |
Beyond Static Toolsets: Self-Evolving LLM Tool Agents via Continual Documentation Adaptation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for evaluating tool usage assume static toolsets with fixed APIs and documentation. |
| Approach: | They propose a continual documentation adaptation framework that allows LLM agents to self-evolve by updating tool documentation. |
| Outcome: | The proposed framework improves performance on three evolution patterns on dynamic extensions of StableToolBench and RestBench. |
MetaASSIST: Robust Dialogue State Tracking with Meta Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing dialogue datasets contain lots of noise in their state annotations. |
| Approach: | They propose a framework to train robust dialogue state tracking models by combining pseudo and vanilla labels by a common weighting parameter. |
| Outcome: | The proposed framework achieves state-of-the-art accuracy of 80.10% on multiWOZ 2.4. |
Improving Dialogue State Tracking with Turn-based Loss Function and Sequential Data Augmentation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models rely on a traditional cross-entropy loss function during training, which may not be optimal for improving the joint goal accuracy. |
| Approach: | They propose a Turn-based Loss Function that penalises the model if it inaccurately predicts a slot value at the early turns more so than in later turns to improve joint goal accuracy. |
| Outcome: | The proposed techniques improve the state-of-the-art model by approximately 7-8% relative reduction in error and achieve a new state- of-the art joint goal accuracy with 59.50 and 54.90 on MultiWOZ2.1 and MultiWOz2.2, respectively. |
Towards Self-Improving Error Diagnosis in Multi-Agent Systems (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing diagnostic approaches rely on expensive expert annotations and ”LLM-as-a-judge” paradigms. |
| Approach: | They propose a framework for semantic failure attribution that identifies responsible agents and the originating error step. |
| Outcome: | The proposed framework outperforms baselines in step-level localization and validation. |
A Joint Optimization Framework for Enhancing Efficiency of Tool Utilization in LLM Agents (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing efforts for tool utilization involve an LLM agent that contains instructions on using the description of the available tools to determine and call the tools required to solve the problem. |
| Approach: | They propose to optimize the context of LLM agents by combining the instructions provided in agent prompts and tool descriptions to enhance their interaction. |
| Outcome: | The proposed framework improves both the instructions provided in agent prompt and tool description, enhancing their interaction. |
Machine-generated text detection prevents language model collapse (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly prevalent across the web, resulting in a degenerative process whereby LLMs reinforce their own errors and reduce output diversity. |
| Approach: | They propose to use machine-generated text to reduce model collapse by up-sampling likely human content in training data. |
| Outcome: | The proposed approach prevents model collapse and improves performance compared to training on purely human data. |
Schema-Guided User Satisfaction Modeling for Task-Oriented Dialogues (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies on USM neglect explicitly modeling the user’s task goals fulfillment using the task schema. |
| Approach: | They propose a schema-guided user satisfaction modeling framework that explicitly models the degree to which the user’s preferences regarding task attributes are fulfilled by the system. |
| Outcome: | The proposed framework outperforms existing methods on benchmark datasets and shows that it can interpret and scale well with unseen tasks and can work in low-resource settings. |
Adaptive Retrieval-Augmented Generation for Conversational Systems (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown the effectiveness of retrieving and augmenting external knowledge for informative responses. |
| Approach: | They propose to use a gating model to predict if a conversational system requires retrieval-augmented generation to generate high-quality responses with high confidence. |
| Outcome: | The proposed model can predict if a conversational system requires RAG to generate high-quality responses with high confidence. |