Multilingual Fine-Grained News Headline Hallucination Detection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models to generate news headlines often suffer from the "hallucination" problem, where the produced headline does not fully align with the source article's content. |
| Approach: | They propose to use a multilingual, fine-grained dataset to detect news headlines in 5 languages using supervised fine-tuning techniques and coarse-to-fine prompting to boost the few-shot detection performance. |
| Outcome: | The proposed methods boost the few-shot hallucination detection performance in terms of the example-F1 metric. |
Similar Papers
When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQA (2025.findings-emnlp)
Copied to clipboard
Elisei Rykov, Kseniia Petrushina, Maksim Savkin, Valerii Olisov, Artem Vazhentsev, Kseniia Titova, Alexander Panchenko, Vasily Konovalov, Julia Belikova
| Challenge: | Existing hallucination detection benchmarks operate at the sequence level and are limited to English . Existing methods lacking fine-grained, multilingual supervision are limited in English based on the sequence . |
| Approach: | They propose a large-scale, multilingual dataset annotated with span-level hallucinations across 14 languages. |
| Outcome: | The proposed dataset annotated with span-level hallucinations across 14 languages is scalable and cost-efficient. |
Detecting Hallucinated Content in Conditional Neural Sequence Generation (2021.findings-acl)
Copied to clipboard
Chunting Zhou, Graham Neubig, Jiatao Gu, Mona Diab, Francisco Guzmán, Luke Zettlemoyer, Marjan Ghazvininejad
| Challenge: | Neural sequence models can generate fluent sentences, but they can also hallucinate additional content not supported by the input. |
| Approach: | They propose a task to predict whether each token in the output sequence is hallucinated and collect manually annotated evaluation sets for this task. |
| Outcome: | The proposed method outperforms baseline methods on machine translation and abstractive summarization datasets and achieves significant improvements in both supervised and unsupervised settings. |
Varta: A Large-Scale Headline-Generation Dataset for Indic Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Varta dataset includes more than 41 million pairs of headlines and articles in 14 different Indic languages (and English) |
| Approach: | They present a large-scale multilingual dataset for headline generation in Indic languages. |
| Outcome: | The Varta dataset includes more than 41 million pairs of headlines and articles in 14 different Indic languages (and English) the data can be used to train strong language models that outperform competitive baselines in both NLU and NLG benchmarks. |
German Also Hallucinates! Inconsistency Detection in News Summaries with the Absinth Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made significant progress on a wide range of natural language processing tasks, but they still suffer from hallucinating information in their output. |
| Approach: | They propose to use an annotated dataset to detect hallucinations in german news summarization and open-source it to foster further research on hallucinosity detection in german. |
| Outcome: | The proposed model can detect hallucinations in the output and evaluate the faithfulness of the summaries. |
How Much Do LLMs Hallucinate across Languages? On Realistic Multilingual Estimation of LLM Hallucination (2025.emnlp-main)
Copied to clipboard
| Challenge: | despite LLMs becoming increasingly multilingual, most studies on detecting and quantifying LLM hallucination are English-centric . |
| Approach: | They train a multilingual hallucination detection model and conduct a large-scale study across 30 languages and 6 open-source LLM families. |
| Outcome: | The proposed model is based on an English-centric model and annotates gold data for five high-resource languages. |
Embedding Hallucination for Few-shot Language Fine-tuning (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained language models can cause severe over-fitting. |
| Approach: | They propose an Embedding Hallucination method which generates auxiliary embedding-label pairs to expand the fine-tuning dataset. |
| Outcome: | The proposed method outperforms current fine-tuning methods in a wide range of language tasks. |
Headline Token-based Discriminative Learning for Subheading Generation in News Article (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing models that generate news subheadings rely on topical headline information to capture topical knowledge from the article. |
| Approach: | They propose a model that uses topical headline information to generate news subheadings using masked headline tokens. |
| Outcome: | The proposed model outperforms the comparative models on three news datasets written in two languages and performs robustly on a small dataset and various masking ratios. |
Constructing a Dataset for Hallucination Detection in Japanese Summarization with Fine-grained Faithfulness Labels (2026.eacl-srw)
Copied to clipboard
| Challenge: | Large language models (LLMs) can generate fluent text, but the quality of generated content depends on its consistency with the given input. |
| Approach: | They constructed a Japanese evaluation dataset for hallucination detection in summarization by manually annotating sentence-level faithfulness labels in LLM-generated summaries of Japanese documents. |
| Outcome: | The proposed model can detect hallucinations in Japanese documents by annotating faithfulness labels in Japanese summaries. |
Embedding and Gradient Say Wrong: A White-Box Method for Hallucination Detection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for hallucination detection have attracted more attention from the community. |
| Approach: | They propose to model the distributional distance between the regular conditional output and the unconditional output, which is generated without a given input text. |
| Outcome: | The proposed model achieves state-of-the-art on the hallucination benchmarks HADES and other datasets. |
HAD: HAllucination Detection Language Models Based on a Comprehensive Hallucination Taxonomy (2026.acl-industry)
Copied to clipboard
| Challenge: | relying on large language models for information has raised concerns about reliability and accuracy of outputs. |
| Approach: | They propose a hallucination taxonomy with 11 categories for various NLG tasks and propose HAllucination Detection models which integrate hallucinism detection, span-level identification, and correction into a single inference process. |
| Outcome: | The proposed models outperform baselines on HaluEval, FactCHD, and FaithBench, confirming their robustness and versatility. |