Papers by Nouha Dziri
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning (2023.emnlp-main)
Copied to clipboard
Ximing Lu, Faeze Brahman, Peter West, Jaehun Jung, Khyathi Chandu, Abhilasha Ravichander, Prithviraj Ammanabrolu, Liwei Jiang, Sahana Ramnath, Nouha Dziri, Jillian Fisher, Bill Lin, Skyler Hallinan, Lianhui Qin, Xiang Ren, Sean Welleck, Yejin Choi
| Challenge: | Extreme-scale language models have shown exceptional performance on a variety of language tasks, but the degree of control offered by these models through pure prompting is limited. |
| Approach: | They propose an inference-time policy adapter which tailors a large base model without fine-tuning it. |
| Outcome: | The proposed model outperforms baseline methods on five challenging text generation tasks and even over GPT-4. |
Automatic Dialogue Generation with Expressed Emotions (N18-2)
Copied to clipboard
| Challenge: | a growing interest in neural dialogue generation systems is focusing on generating human-like responses based on past utterances . despite efforts, few consider putting restrictions on the response itself . authors present three models that concatenate the desired emotion with the source input . |
| Approach: | They propose three models that concatenate the desired emotion with the source input or push the emotion in the decoder. |
| Outcome: | The proposed model is more efficient than the previous models, but it lacks the emotion vector. |
What Makes it Ok to Set a Fire? Iterative Self-distillation of Contexts and Rationales for Disambiguating Defeasible Social and Moral Situations (2023.findings-emnlp)
Copied to clipboard
Kavel Rao, Liwei Jiang, Valentina Pyatkin, Yuling Gu, Niket Tandon, Nouha Dziri, Faeze Brahman, Yejin Choi
| Challenge: | Moral or ethical judgments rely heavily on the contexts in which they occur . a student model that produces defeasible contexts with improved validity, diversity, and defasibility is superior to intermediate student models . |
| Approach: | a new study uses a student model to provide contextualizations that make an action morally acceptable . the model is based on a dataset of 115K defeasible moral actions rated highly by human annotators . |
| Outcome: | The proposed model outperforms all intermediate models in a high-quality dataset . the model is based on 1.2M entries of contextualizations and rationales for 115K moral actions . |
Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark (2022.tacl-1)
Copied to clipboard
| Challenge: | Knowledge-grounded dialogue systems powered by large language models often generate responses that, while fluent, are not attributable to a relevant source of information. |
| Approach: | They propose to evaluate the validity of 12k dialogue turns generated by neural dialogue systems trained on three knowledge-grounded dialogue corpora and to use them to analyze eight evaluation metrics. |
| Outcome: | The proposed evaluation metrics rely on spurious correlations, do not reliably distinguish attributable abstractive responses from unattributable ones, and perform substantially worse when the knowledge source is longer. |
Evaluating Coherence in Dialogue Systems using Entailment (N19-1)
Copied to clipboard
| Challenge: | Evaluating open-domain dialogue systems is difficult due to the diversity of possible correct answers. |
| Approach: | They propose a set of metrics for evaluating topic coherence using distributed sentence representations and calculable approximations of human judgment using conversational coherency. |
| Outcome: | The proposed metrics can be used as a surrogate for human judgment based on conversational coherence on large-scale datasets and provide an unbiased estimate for the quality of the responses. |
Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding (2021.emnlp-main)
Copied to clipboard
| Challenge: | Dialogue systems that generate factually incorrect responses are often unfitful and hallucinate factuality invalid. |
| Approach: | They propose a method to improve faithfulness and reduce hallucination of neural dialogue systems to known facts supplied by a Knowledge Graph. |
| Outcome: | The proposed approach improves faithfulness and reduces hallucination of dialogue systems to known facts . it leverages a token-level fact critic to identify plausible sources of hallucinism . |
FaithDial: A Faithful Benchmark for Information-Seeking Dialogue (2022.tacl-1)
Copied to clipboard
| Challenge: | a new benchmark for hallucination-free dialogues is based on knowledge-based conversational models that generate unsupported utterances . a recent study shows that models that are trustworthy generate unverifiable or factually incorrect statements . |
| Approach: | They propose a data-centric solution to edit hallucinated responses in the Wizard of Wikipedia benchmark. |
| Outcome: | The proposed model improves on the Wizard of Wikipedia benchmark while maintaining engaging conversations. |
On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models? (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing knowledge-grounded conversational benchmarks produce factually invalid statements, a phenomenon commonly called hallucination. |
| Approach: | They conduct a human study on knowledge-grounded conversational benchmarks and state-of-the-art models. |
| Outcome: | The findings raise important questions on the quality of existing datasets and models. |
RewardBench: Evaluating Reward Models for Language Modeling (2025.findings-naacl)
Copied to clipboard
Nathan Lambert, Valentina Pyatkin, Jacob Morrison, Lester James Validad Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi
| Challenge: | Evaluating reward models presents an opportunity to understand the opaque technologies used for alignment of language models. |
| Approach: | They present a benchmark dataset and code-base for evaluation of reward models . they use prompt-chosen-rejected trios to benchmark how they perform on queries . |
| Outcome: | The proposed dataset compares RMs with other models on a set of questions. |
Elastic Weight Removal for Faithful and Abstractive Dialogue Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Current-day large language models generate coherent, grammatical, and seemingly meaningful text, but are prone to hallucinating incorrect information. |
| Approach: | They propose to ‘subtract’ parameters of a model trained to hallucinate from a dialogue response generation model to ‘negate’ the contribution of such hallucinatedexamples from it. |
| Outcome: | The proposed method reduces hallucinations and discourages extractive responses, which are often a consequence of reducing hallucines by encouraging copy-pasting of document spans. |
Knowledge Control for Responsible Generative AI: Bridging Academia, Industry, and Society (2026.acl-tutorials)
Copied to clipboard
Zheyuan Liu, Yixin Wan, Kai-Wei Chang, Meng Jiang, Jieyu Zhao, Nouha Dziri, Yuning Mao, Jia-Chen Gu, Jindong Gu
| Challenge: | This tutorial introduces the foundations of post-training knowledge control and showcases recent frontier methods. |
| Approach: | This tutorial introduces the foundations of post-training knowledge control and showcases recent frontier methods. |
| Outcome: | This tutorial introduces the foundations of post-training knowledge control and showcases recent frontier methods . key motivations and failure modes, harmful generation and stereotype reinforcement, are addressed . core methods such as machine unlearning, knowledge editing, and inference-time interventions are also included . |
REL-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluations of large language models' ability to communicate uncertainty and knowledge limitations focus on the behaviors of their human interlocutors. |
| Approach: | They propose an interaction-centered evaluation approach that quantifies whether and how humans rely on LLMs' responses. |
| Outcome: | The proposed approach quantifies whether and how humans rely on LLMs' responses. |
Evaluating Open-Domain Question Answering in the Era of Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation models fail to identify lexical matching failures for open-domain question answering. |
| Approach: | They manually evaluate open-domain QA models by manually evaluating their answers on a popular benchmark. |
| Outcome: | The proposed model performs better on NQ-open than existing models and more than 50% of lexical matching failures are attributed to semantically equivalent answers. |