LUCID: LLM-Generated Utterances for Complex and Interesting Dialogues (2024.naacl-srw)
Copied to clipboard
Joe Stacey, Jianpeng Cheng, John Torr, Tristan Guigue, Joris Driesen, Alexandru Coca, Mark Gaynor, Anders Johannsen
| Challenge: | Existing datasets with limited domain coverage and few challenging conversational phenomena are often unlabelled . Existing data is limited in quality and lacks a robust evaluation process . |
| Approach: | They propose a high quality data generation system that generates high quality dialogues using 4,277 conversations across 100 intents. |
| Outcome: | The proposed system produces high quality dialogue data with high quality labels. |
Similar Papers
Generative Interfaces for Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models are increasingly seen as assistants, copilots, and consultants . however, their linear request-response format often makes interactions inefficient in multi-turn tasks . |
| Approach: | They propose a paradigm in which large language models respond to user queries by generating user interfaces that enable more adaptive and interactive engagement. |
| Outcome: | The proposed paradigm outperforms traditional chat-based interfaces in many tasks and interaction patterns. |
Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Human-LLM Dialogue (2026.findings-acl)
Copied to clipboard
Jonathan Ivey, Shivani Kumar, Jiayu Liu, Hua Shen, Sushrita Rakshit, Rohan Raju, Haotian Zhang, Aparna Ananthasubramaniam, Junghwan Kim, Bowen Yi, Dustin Wright, Abraham Israeli, Anders Giovanni Møller, Lechen Zhang, David Jurgens
| Challenge: | Recent work has sought to use large language models to simulate human-human and human-LLM interactions. |
| Approach: | They use a large-scale dataset to generate a paired LLM-LLM and human-LLm dialogues from the WildChat dataset and quantify how well they align with their human counterparts. |
| Outcome: | The proposed models perform similarly in simulating English, Chinese, and Russian dialogues. |
On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) provide a data-centric solution to alleviate limitations of real-world data with synthetic data generation. |
| Approach: | They propose a generic workflow for LLM-driven synthetic data generation. |
| Outcome: | The proposed workflows highlight gaps in existing research and outline avenues for future studies. |
Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are powerful dialogue agents, but specializing them towards fulfilling a specific function can be prohibitive in terms of feasibility, time, and resources. |
| Approach: | They propose a method for training large language models by enabling "self-talk" they propose supervised fine-tuning of LLMs to improve quality of dialogues . |
| Outcome: | The proposed method generates training data via "self-talk" of LLMs that can be refined and utilized for supervised fine-tuning. |
LLM-REDIAL: A Large-Scale Dataset for Conversational Recommender Systems Created from User Behaviors with LLMs (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing CRS datasets suffer from data inextensibility and semantic inconsistency . |
| Approach: | They introduce the LLM-REDIAL dataset to facilitate the research in CRS by leveraging large language models to generate high-quality dialogues. |
| Outcome: | The proposed dataset is the largest multi-domain CRS dataset which consists of 47.6k multi-turn dialogues with 482.6k utterances across 4 domains. |
Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for data-to-text generation are saturated, and there is no way to test them. |
| Approach: | They propose a tool for collecting structured data from public APIs to analyze the behavior of open large language models on the task of data-to-text generation. |
| Outcome: | The proposed model can generate fluent and coherent texts in zero-shot settings from data in common formats collected with Quintd. |
MAGID: An Automated Pipeline for Generating Synthetic Multi-modal Datasets (2024.naacl-long)
Copied to clipboard
Hossein Aboutalebi, Hwanjun Song, Yusheng Xie, Arshit Gupta, Lijia Sun, Hang Su, Igor Shalyminov, Nikolaos Pappas, Siffi Singh, Saab Mansour
| Challenge: | Existing approaches to augment textual dialogues with retrieved images pose privacy, diversity, and quality constraints. |
| Approach: | They propose a framework to augment text-only dialogues with diverse and high-quality images by using a diffusion model and a feedback loop. |
| Outcome: | The proposed framework is comparable to or better than baselines, with significant improvements in human evaluation, especially against retrieval baselines where the image database is small. |
MEGA: Multilingual Evaluation of Generative AI (2023.emnlp-main)
Copied to clipboard
Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram
| Challenge: | Large Large Models (LLMs) have shown impressive performance on many natural language processing tasks such as language understanding, reasoning, and language generation. |
| Approach: | They present a framework for evaluating generative LLMs in the multilingual setting and provide directions for future progress in the field. |
| Outcome: | The proposed framework evaluates generative models on 16 NLP datasets across 70 typologically diverse languages and compares them to state-of-the-art non-autoregressive models. |
Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis (2025.acl-long)
Copied to clipboard
| Challenge: | Personalized AI assistants are a challenging application that intertwines multiple problems in LLM research. |
| Approach: | They propose a Llama-3.2-based automated evaluation model that matches human preferences to a conversational dataset. |
| Outcome: | HiCUPID provides a conversational dataset tailored for personalization . the evaluation model closely mirrors human preferences, the researchers show . |
PRODIGy: a PROfile-based DIalogue Generation dataset (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing profiles-based dialogue datasets lack explicit profile representations or are difficult to collect. |
| Approach: | They propose a dataset that brings together multiple profiles for each speaker, and then integrates them together to provide a more comprehensive profile dimension set for generative language models. |
| Outcome: | The PRODIGy dataset provides a more comprehensive profile dimension set for each speaker. |