Papers by Lijia Sun
MAGID: An Automated Pipeline for Generating Synthetic Multi-modal Datasets (2024.naacl-long)
Copied to clipboard
Hossein Aboutalebi, Hwanjun Song, Yusheng Xie, Arshit Gupta, Lijia Sun, Hang Su, Igor Shalyminov, Nikolaos Pappas, Siffi Singh, Saab Mansour
| Challenge: | Existing approaches to augment textual dialogues with retrieved images pose privacy, diversity, and quality constraints. |
| Approach: | They propose a framework to augment text-only dialogues with diverse and high-quality images by using a diffusion model and a feedback loop. |
| Outcome: | The proposed framework is comparable to or better than baselines, with significant improvements in human evaluation, especially against retrieval baselines where the image database is small. |
Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are powerful dialogue agents, but specializing them towards fulfilling a specific function can be prohibitive in terms of feasibility, time, and resources. |
| Approach: | They propose a method for training large language models by enabling "self-talk" they propose supervised fine-tuning of LLMs to improve quality of dialogues . |
| Outcome: | The proposed method generates training data via "self-talk" of LLMs that can be refined and utilized for supervised fine-tuning. |
TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization (2024.naacl-long)
Copied to clipboard
Liyan Tang, Igor Shalyminov, Amy Wong, Jon Burnsky, Jake Vincent, Yu’an Yang, Siffi Singh, Song Feng, Hwanjun Song, Hang Su, Lijia Sun, Yi Zhang, Saab Mansour, Kathleen McKeown
| Challenge: | Existing LLMs hallucinate significant amounts of factual errors in the dialogue domain, regardless of the model’s size. |
| Approach: | They propose to evaluate topic-focused dialogue summarization by using large language models (LLMs) they use human annotations to evaluate factual consistency and explain factually inconsistent sentences. |
| Outcome: | The proposed evaluation benchmark on topic-focused dialogue summarization shows that existing LLMs hallucinate significant amounts of factual errors regardless of the model’s size. |