Papers by Nico Daheim
Book2Dial: Generating Teacher Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbots (2024.findings-acl)
Copied to clipboard
| Challenge: | Educational chatbots are a promising tool for assisting student learning, but high-quality data is difficult to obtain due to privacy concerns. |
| Approach: | They propose a framework for generating synthetic teacher-student interactions grounded in a set of textbooks and propose to open-source their results. |
| Outcome: | The proposed framework captures a key aspect of learning interactions where curious students with partial knowledge ask teachers questions about the material in the textbook. |
From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are often optimized for direct question-answering, but their effectiveness is often undermined by strategically withholding answers. |
| Approach: | They propose an online reinforcement learning-based alignment framework that can quickly adapt LLMs into effective tutors using simulated student-tutor interactions. |
| Outcome: | The proposed model outperforms proprietary models like LearnLM and can be used to enhance interpretability and pedagogical quality. |
A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs (2025.emnlp-main)
Copied to clipboard
Artem Shelmanov, Ekaterina Fadeeva, Akim Tsvigun, Ivan Tsvigun, Zhuohan Xie, Igor Kiselev, Nico Daheim, Caiqi Zhang, Artem Vazhentsev, Mrinmaya Sachan, Preslav Nakov, Timothy Baldwin
| Challenge: | Uncertainty quantification (UQ) is a framework for assessing the reliability of model outputs. |
| Approach: | They introduce pre-trained UQ heads for LLMs that are highly robust and generalized to languages they were not explicitly trained on. |
| Outcome: | The pre-trained heads significantly improve their ability to capture uncertainty compared to unsupervised methods. |
MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors (2025.emnlp-main)
Copied to clipboard
| Challenge: | Evaluating the pedagogical capabilities of AI-based tutoring models is critical for guided progress in the field. |
| Approach: | They propose an open-source benchmark for holistic tutoring model evaluation. |
| Outcome: | The proposed model can discriminate between expert and novice teachers with high accuracy. |
MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems (2023.findings-emnlp)
Copied to clipboard
Jakub Macina, Nico Daheim, Sankalan Chowdhury, Tanmay Sinha, Manu Kapur, Iryna Gurevych, Mrinmaya Sachan
| Challenge: | Existing models for automatic dialogue tutoring fail to provide accurate feedback or reveal solutions to students too early. |
| Approach: | They propose a framework to generate one-to-one teacher-student tutoring dialogues by pairing human teachers with a Large Language Model (LLM) they use scaffolding questions and annotations to fine-tune models to be more effective tutors . |
| Outcome: | The proposed framework can generate 3k one-to-one teacher-student tutoring dialogues grounded in multi-step math reasoning problems. |
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)
Copied to clipboard
Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina Mcmillan-major, Anna Shvets, Ashish Upadhyay, Bernd Bohnet, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez Beltrachini, Leonardo F . R. Ribeiro, Lewis Tunstall, Li Zhang, Mahim Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Štajner, Sebastien Montella, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou
| Challenge: | Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work. |
| Approach: | They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations. |
| Outcome: | The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work. |
Opportunities and Challenges in Neural Dialog Tutoring (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to designing dialog tutors have been challenging . current approaches perform poorly in constrained learning scenarios, authors find . |
| Approach: | They analyze dialog tutoring models using automatic and human evaluations to understand the new opportunities brought by dialog tutors. |
| Outcome: | The proposed models perform poorly in less constrained learning scenarios, the authors show . they find large number of model reasoning errors in 45% of conversations . |
Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for dialog tutoring fail to detect student errors and tailor their feedback to them. |
| Approach: | They propose to build dialog tutoring models to scaffold students' problem-solving and verify student solutions by using automatic and human evaluation. |
| Outcome: | The proposed model improves the quality of the tutor response generation by detecting student errors and adjusting the feedback to the errors. |
Token Weighting for Long-Range Language Modeling (2025.findings-naacl)
Copied to clipboard
| Challenge: | Many applications of large language models (LLMs) require long-context understanding, but models still struggle with such tasks. |
| Approach: | They propose token-weighting schemes that assign different weights to each training token in the loss, generalizing existing works. |
| Outcome: | The proposed methods compare confidences of a long-context and short-concept model and show that non-uniform loss weights improve the long-constability of LLMs. |
Elastic Weight Removal for Faithful and Abstractive Dialogue Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Current-day large language models generate coherent, grammatical, and seemingly meaningful text, but are prone to hallucinating incorrect information. |
| Approach: | They propose to ‘subtract’ parameters of a model trained to hallucinate from a dialogue response generation model to ‘negate’ the contribution of such hallucinatedexamples from it. |
| Outcome: | The proposed method reduces hallucinations and discourages extractive responses, which are often a consequence of reducing hallucines by encouraging copy-pasting of document spans. |
Poor Man’s Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference (2023.eacl-main)
Copied to clipboard
Vilém Zouhar, Shehzaad Dhuliawala, Wangchunshu Zhou, Nico Daheim, Tom Kocmi, Yuchen Eleanor Jiang, Mrinmaya Sachan
| Challenge: | State-of-the-art machine translation quality estimation systems have been achieving remarkable correlations with human judgements yet they require human annotations, which are expensive and computationally heavy. |
| Approach: | They propose a problem where one predicts automated metric scores without the reference. |
| Outcome: | The proposed model can estimate automated metrics at the sentence-level without the reference. |
Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent document-grounded dialog systems have seen an increase in popularity. |
| Approach: | They propose a model for document-grounded response generation in dialog that is decomposed into two components according to Bayes’ theorem and propose different approximate decoding schemes. |
| Outcome: | The proposed model is more factual in terms of automatic factuality metrics than the baseline model and can be combined with a recently proposed method to control factuity in grounded dialog, CTRL. |