From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have enhanced the capacity of vision-language models to caption visual text. |
| Approach: | They compare standard-format captions and recent GCE processes from the perspectives of gender bias and hallucination. |
| Outcome: | The proposed methods amplify gender bias by 30.9% and increase hallucination by 59.5%. |
Similar Papers
Addressing Bias and Hallucination in Large Language Models (2024.lrec-tutorials)
Copied to clipboard
Nihar Ranjan Sahoo, Ashita Saxena, Kishan Maharaj, Arif A. Ahmad, Abhijit Mishra, Pushpak Bhattacharyya
| Challenge: | This tutorial provides a comprehensive overview of two critical aspects of Large Language Models: bias and hallucination. |
| Approach: | This tutorial provides an overview of two critical aspects of Large Language Models: bias and hallucination. |
| Outcome: | This tutorial delves into the complex dimensions of Large Language Models (LLMs) it outlines ethical considerations pertinent to their development and discusses hallucination, a prevalent issue in generative AI systems such as LLMs. |
ACUEval: Fine-grained Hallucination Evaluation and Correction for Abstractive Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent-proposed evaluation metrics for large language models have a preference-bias . however, such metrics often lack interpretability and only offer a single score . |
| Approach: | They propose a metric that leverages the power of large language models to perform two sub-tasks: decomposing summaries into atomic content units and validating them against the source document. |
| Outcome: | The proposed metric improves faithfulness scores on three summarization evaluation benchmarks by 3% compared to the next-best metric. |
Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation (2025.findings-acl)
Copied to clipboard
| Challenge: | Prior work has shown that finetuning on specialized datasets can mitigate this behavior, and doing so in English can transfer to other languages. |
| Approach: | They propose to fine tune generative large language models to provide safe responses to harmful user input and to use direct preference optimization to mitigate toxicity. |
| Outcome: | The proposed models show that finetuning on specialized datasets reduces biases but also produces fluent and diverse text in non-English languages. |
Enhancing Large Vision-Language Models with Ultra-Detailed Image Caption Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing pipelines for generating high-quality, ultra-detailed image captions are limited by the scarcity of image caption data. |
| Approach: | They propose a pipeline for generating high-quality, ultra-detailed image captions that integrates both pre-processing and post-processor stages. |
| Outcome: | The proposed pipeline improves LVLMs' perception and cognitive abilities across multiple vision-language benchmarks. |
Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) have been proposed to augment LLMs with visual inputs. |
| Approach: | They propose large vision-Language Models to augment LLMs with visual inputs. |
| Outcome: | The proposed models condition generated text on both an input image and a visual prompt, enabling a variety of use cases such as visual question answering and multimodal chat. |
Towards Enhancing Coherence in Extractive Summarization: Dataset and Experiments with LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for extractive summarization lack coherence, despite improvements . a human-annotated dataset is used to improve coherency of extractive summary . |
| Approach: | They propose to use human-annotated datasets to create coherent extractive summaries . they use supervised fine-tuning and natural language user feedback to enhance coherence . |
| Outcome: | The proposed dataset shows that LLMs can produce coherent summaries with human feedback. |
The Impact of Inference Acceleration on Bias of LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent work suggests strategies to increase inference efficiency with LLMs . however, these strategies may inadvertently lead to some side-effects. |
| Approach: | They propose to optimize inference acceleration strategies such as quantization, pruning, and caching to reduce inference cost and latency while maintaining predictive performance. |
| Outcome: | The proposed strategies reduce cost and latency while maintaining predictive performance while preserving the model size. |
Exploring the Potential of Dense Information in Multimodal Alignment (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to enhance captions have limitations such as insufficient detail and excessive hallucinations, resulting in compromised alignment and masking the true potential of dense information. |
| Approach: | They propose a pipeline that generates highly detailed captions for images that facilitates in-depth analysis of the potential for dense information in multimodal alignment. |
| Outcome: | The proposed pipeline significantly improves multimodal alignment and compositional reasoning abilities, surpassing hard negative samples. |
Long Story Short: Disentangling Compositionality and Long-Caption Understanding in Contrastive VLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for vision-language models treat compositionality and long-caption understanding in isolation. |
| Approach: | They analyze when compositional reasoning and long-caption understanding transfer across tasks and when this relationship fails. |
| Outcome: | The proposed model can generalize on poorly grounded captions and with strong visual grounding, while architectural choices can limit compositional learning. |
When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization (2023.eacl-main)
Copied to clipboard
Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, Tatsunori Hashimoto
| Challenge: | Existing studies have shown that large language models contain linguistic and societal biases, but it is unclear how these biase amplify to downstream tasks. |
| Approach: | They investigate how name-nationality bias propagates from pre-training to downstream tasks . they show that these biases manifest themselves as hallucinations in summarization . |
| Outcome: | The proposed model can reduce the rate of hallucinations, but does not change the types of biases that do appear. |