Peering into the Mind of Language Models: An Approach for Attribution in Contextual Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used for question answering . lack of explicit references or attributions hinders ability to verify accuracy of answers . |
| Approach: | They propose a method for attribution in contextual question answering . they use hidden state representations of large language models to identify copied segments . |
| Outcome: | The proposed method performs better than GPT-4 at identifying verbatim copied segments in LLM generations and attributing these segments to their source. |
Similar Papers
On Synthesizing Data for Context Attribution in Question Answering (2025.acl-long)
Copied to clipboard
Gorjan Radevski, Kiril Gashteovski, Shahbaz Syed, Christopher Malon, Sebastien Nicolas, Chia-Chien Hung, Timo Sztyler, Verena Heußer, Wiem Ben Rim, Masafumi Enomoto, Kunihiro Takeoka, Masafumi Oyamada, Goran Glavaš, Carolin Lawrence
| Challenge: | Large Language Models (LLMs) have a tendency to hallucinate, resulting in false or misleading answers. |
| Approach: | They propose a novel generative strategy for synthesizing context attribution data. |
| Outcome: | The proposed approach is highly effective for fine-tuning small LMs for context attribution in different QA tasks and domains. |
Automatic Evaluation of Attribution by Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Generative large language models (LLMs) incorporate external references to generate and support claims. however, evaluating the attribution remains an open problem. |
| Approach: | They investigate automatic evaluation of attribution given by large language models . they define different types of attributed errors and then explore two approaches . |
| Outcome: | The proposed methods highlight promising signals and challenges. |
Evaluating and Modeling Attribution for Cross-Lingual Question Answering (2023.emnlp-main)
Copied to clipboard
Benjamin Muller, John Wieting, Jonathan Clark, Tom Kwiatkowski, Sebastian Ruder, Livio Soares, Roee Aharoni, Jonathan Herzig, Xinyi Wang
| Challenge: | Open-retrieval question answering systems are lacking in attribution for cross-lingual question answering . open-research questions are available in 20 languages, but their raw generation often falls short in factuality . |
| Approach: | They are the first to study attribution for cross-lingual question answering . they collect data in 5 languages to assess the attribution level of a state-of-the-art QA system . |
| Outcome: | The proposed approach improves the attribution level of a state-of-the-art cross-lingual QA system. |
Decomposition-Enhanced Training for Post-Hoc Attributions in Language Models (2026.eacl-long)
Copied to clipboard
Sriram Balasubramanian, Samyadeep Basu, Koustava Goswami, Ryan A. Rossi, Varun Manjunatha, Roshan Santhosh, Ruiyi Zhang, Soheil Feizi, Nedim Lipka
| Challenge: | Existing methods for extractive QA struggle in multi-hop, abstractive, and semi-extractive settings. |
| Approach: | They propose a method that prompts models to produce answer decompositions as intermediate reasoning steps. |
| Outcome: | The proposed method outperforms existing methods and matches or exceeds state-of-the-art frontier models. |
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources. |
| Approach: | They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation . |
| Outcome: | The proposed model outperforms previous approaches by a significant margin in QA tasks over text. |
Exploiting Contextual Knowledge in LLMs through 𝒱-usable Information based Layer Enhancement (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to enhance the context-faithfulness of Large Language Models (LLMs) ignore the fundamental mechanism of how contextual information is processed within LLMs’ internal states. |
| Approach: | They propose a method that enhances the utilization of contextual knowledge within LLMs’ internal representations by employing V-usable information analysis. |
| Outcome: | The proposed method improves context-faithfulness generation in Question-Answering tasks, particularly in scenarios involving unknown or conflicting contextual knowledge. |
Multi-Level Explanations for Generative Language Models (2025.acl-long)
Copied to clipboard
Lucas Monteiro Paes, Dennis Wei, Hyo Jin Do, Hendrik Strobelt, Ronny Luss, Amit Dhurandhar, Manish Nagireddy, Karthikeyan Natesan Ramamurthy, Prasanna Sattigeri, Werner Geyer, Soumya Ghosh
| Challenge: | Large language models (LLMs) are being used for context-grounded tasks like summarizing meetings and answering doctors' questions. |
| Approach: | They propose a technique to provide explanations for context-grounded text generation by assigning scores to parts of the context to quantify their influence on the model output. |
| Outcome: | The proposed framework can provide more faithful explanations of generated output than available alternatives, including LLM self-explanations. |
CoTAR: Chain-of-Thought Attribution Reasoning with Multi-level Granularity (2024.findings-emnlp)
Copied to clipboard
| Challenge: | State-of-the-art QA systems employ Large Language Models (LLMs) however, these models tend to hallucinate information in their responses. |
| Approach: | They propose an attribution-oriented Chain-of-Thought reasoning method to enhance attributions. |
| Outcome: | The proposed method outperforms existing models on context enhanced question-answering datasets and shows that it can be used to improve accuracy. |
Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models lack contextual knowledge, resulting in text with factual inconsistencies or contextually unfaithful content. |
| Approach: | They propose a method that integrates contrastive decoding with adversarial irrelevant passages as negative samples to enhance robust context grounding during generation. |
| Outcome: | The proposed method improves context grounding during generation without training. |
Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts? (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in augmenting Large Language Models (LLMs) with auxiliary information have significantly revolutionized their efficacy in knowledge-intensive tasks. |
| Approach: | They propose a systematic framework to identify whether LLMs’ responses are attributed to either generated or retrieved contexts. |
| Outcome: | The proposed framework identifies whether LLMs’ responses are attributed to either generated or retrieved contexts. |