Papers by Mohammad Aliannejadi
Learning to Judge: LLMs Designing and Applying Evaluation Rubrics (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models are increasingly used as evaluators for natural language generation . human rubrics are often static and misaligned with how models internally represent language quality. |
| Approach: | They propose to use large language models to generate interpretable and task-aware evaluation dimensions and apply them within models. |
| Outcome: | The proposed model improves the semantic coherence and scoring reliability of LLM-defined criteria and their alignment with human criteria. |
Generating Multi-Aspect Queries for Conversational Search (2026.eacl-long)
Copied to clipboard
| Challenge: | Conversational information seeking (CIS) systems aim to model the user’s information need within the conversational context and retrieve the relevant information. |
| Approach: | They propose a multi-aspect query generation and retrieval framework which uses Large Language Models to break the user utterance into multiple queries. |
| Outcome: | The proposed framework outperforms state-of-the-art query rewriting methods on six widely used CIS datasets and fine-tunes the model on MASQ yields significant improvements. |
ChatR1: Reinforcement Learning for Conversational Reasoning and Retrieval Augmented Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Unlike static ‘rewrite, retrieve, and generate’ pipelines, ChatR1 interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through RL. |
| Approach: | They propose a reasoning framework based on reinforcement learning (RL) for conversational question answering that interleaves search and reasoning across turns and provides turn-level feedback. |
| Outcome: | The proposed framework outperforms competing models on five CQA datasets, measured by different metrics (F1, BERTScore, and LLM-as-judge). |
Clarifying the Path to User Satisfaction: An Investigation into Clarification Usefulness (2024.findings-eacl)
Copied to clipboard
| Challenge: | Poorly formulated questions can lead to user frustration and dissatisfaction . |
| Approach: | They propose to leverage key features that contribute to the classification of clarifying questions, enhancing user satisfaction and system performance. |
| Outcome: | The proposed model improves with a minimum performance boost of 45% in traditional classifiers, especially in large language models. |
Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances on neural approaches to natural language processing have triggered a renaissance in end-to-end neural open-domain chatbots. |
| Approach: | They propose to use offline and online steps to evaluate the quality of clarifying questions in various open-domain dialogues to improve the quality and accuracy of the system response. |
| Outcome: | The proposed pipeline is suitable as a foundation for further research. |
CAUSE: Counterfactual Assessment of User Satisfaction Estimation in Task-Oriented Dialogue Systems (2024.findings-acl)
Copied to clipboard
| Challenge: | a number of studies have evaluated user satisfaction estimation in TOD systems . current benchmarks for user satisfaction estimates are highly skewed towards dialogues for which the user is satisfied. |
| Approach: | They leverage large language models to generate satisfaction-aware counterfactual dialogues to augment original dialogues of a test collection. |
| Outcome: | The proposed models show higher robustness to increase in dissatisfaction labels than fine-tuned models. |
Unlocking Markets: A Multilingual Benchmark to Cross-Market Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Product-related question answering (PQA) involves utilizing product-related resources to provide precise answers to users. |
| Approach: | They propose a task of multilingual cross-market product-based question answering that combines product-related questions with product-specific questions from a multilingual marketplace. |
| Outcome: | The proposed task provides answers to product-related questions in a multilingual marketplace even in fewer languages. |
Eliciting Motivational Interviewing Skill Codes in Psychotherapy with LLMs: A Bilingual Dataset and Analytical Study (2024.lrec-main)
Copied to clipboard
Xin Sun, Jiahuan Pei, Jan de Wit, Mohammad Aliannejadi, Emiel Krahmer, Jos T.P. Dobber, Jos A. Bosch
| Challenge: | Motivational interviewing (MI) is an essential, directive, client-centered counseling technique. |
| Approach: | They propose a bilingual dataset of MI conversations in English and Dutch . they propose an approach to elicit MISC expertise from Large language models . |
| Outcome: | The proposed approach yields results aligned with expert annotations and maintains consistent performance across languages. |
Summarize-Exemplify-Reflect: Data-driven Insight Distillation Empowers LLMs for Few-shot Tabular Classification (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show the promise of large language models for few-shot tabular classification but highlight challenges due to the variability in structured data. |
| Approach: | They propose a framework that distills data into actionable insights to enable robust and effective classification by large language models. |
| Outcome: | The proposed framework integrates rule summarization, strategic exemplification, and insight reflection through deep collaboration between LLMs and data modeling techniques. |
Expand, Highlight, Generate: RL-driven Document Generation for Passage Reranking (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies use large language models to generate training data for ranking models. |
| Approach: | They propose a pipeline that generates synthetic documents from queries using large language models . they propose RL-based reinforcement learning to optimize the pipeline . |
| Outcome: | The proposed pipeline outperforms existing state-of-the-art methods in generating synthetic documents more effectively. |
SOLID: Self-seeding and Multi-intent Self-instructing LLMs for Generating Intent-aware Information-Seeking Dialogs (2025.findings-naacl)
Copied to clipboard
Arian Askari, Roxana Petcu, Chuan Meng, Mohammad Aliannejadi, Amin Abolghasemi, Evangelos Kanoulas, Suzan Verberne
| Challenge: | Existing methods for intent prediction rely on human feedback and are tailored to structured intents. |
| Approach: | They propose a method that generates dialogs turn-by-turn using self-seeding and multi-intent self-instructing strategies. |
| Outcome: | The proposed methods generate dialogs turn-by-turn using self-seeding and multi-intent self-instructing strategies. |
Context Does Matter: Implications for Crowdsourced Evaluation Labels in Task-Oriented Dialogue Systems (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies suggest using only a portion of the dialogue context in the annotation process, but the impact of this limitation on label quality remains unexplored. |
| Approach: | They propose to use large language models to summarize the dialogue context to provide a rich and short description of the dialogue and to examine the impact of doing so on the annotator’s performance. |
| Outcome: | The proposed model reduces the context and produces higher quality ratings but introduces ambiguity in usefulness ratings. |