Lucas Monteiro Paes, Dennis Wei, Hyo Jin Do, Hendrik Strobelt, Ronny Luss, Amit Dhurandhar, Manish Nagireddy, Karthikeyan Natesan Ramamurthy, Prasanna Sattigeri, Werner Geyer, Soumya Ghosh
| Challenge: | Large language models (LLMs) are being used for context-grounded tasks like summarizing meetings and answering doctors' questions. |
| Approach: | They propose a technique to provide explanations for context-grounded text generation by assigning scores to parts of the context to quantify their influence on the model output. |
| Outcome: | The proposed framework can provide more faithful explanations of generated output than available alternatives, including LLM self-explanations. |
Similar Papers
TextGenSHAP: Scalable Post-Hoc Explanations in Text Generation with Long Documents (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are difficult to explain and understand due to long input contexts and autoregressive output generation. |
| Approach: | They propose a post-hoc explanation method which incorporates LLM-specific techniques. |
| Outcome: | The proposed method improves retrieval recall and prediction accuracy significantly on open-domain question answering benchmarks. |
Peering into the Mind of Language Models: An Approach for Attribution in Contextual Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used for question answering . lack of explicit references or attributions hinders ability to verify accuracy of answers . |
| Approach: | They propose a method for attribution in contextual question answering . they use hidden state representations of large language models to identify copied segments . |
| Outcome: | The proposed method performs better than GPT-4 at identifying verbatim copied segments in LLM generations and attributing these segments to their source. |
XplainLLM: A Knowledge-Augmented Dataset for Reliable Grounded Explanations in LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved remarkable success in natural language tasks, yet understanding their reasoning processes remains a significant challenge. |
| Approach: | They propose a dataset that includes 24204 instances where each instance interprets the LLM’s reasoning behavior using knowledge graphs and graph attention networks (GAT). |
| Outcome: | The proposed explanation framework reduces hallucinations and improves grounded explanation generation in large language models. |
Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts? (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in augmenting Large Language Models (LLMs) with auxiliary information have significantly revolutionized their efficacy in knowledge-intensive tasks. |
| Approach: | They propose a systematic framework to identify whether LLMs’ responses are attributed to either generated or retrieved contexts. |
| Outcome: | The proposed framework identifies whether LLMs’ responses are attributed to either generated or retrieved contexts. |
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources. |
| Approach: | They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation . |
| Outcome: | The proposed model outperforms previous approaches by a significant margin in QA tasks over text. |
Distillation with Explanations from Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) like ChatGPT and GPT-4 have made remarkable progress in various NLP tasks while also providing explanations alongside their answers. |
| Approach: | They propose to use Large language models (LLMs) to generate more accurate answers and corresponding free-text explanations by combining ground truth labels and answers-explanations generated by LLMs. |
| Outcome: | The proposed method achieves improved predictive performance and generates explanations that exhibit greater alignment with the model’s task outputs. |
Explain-then-translate: an analysis on improving program translation with self-generated explanations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Using self-generated natural language explanations improves zero-shot performance by 12% on average. |
| Approach: | They propose to use self-generated natural language explanations as an intermediate step for code-to-code translation with language models. |
| Outcome: | The proposed approach improves zero-shot performance by 12% on average . the proposed approach is not evaluated on a broader set of languages including low-resource languages. |
Making Long-Context Language Models Better Multi-Hop Reasoners (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in long-context modeling have enhanced language models for complex tasks, but they struggle with multi-hop reasoning and noisy contexts. |
| Approach: | They propose an approach that prompts LMs to supply attributions for each assertion during reasoning. |
| Outcome: | The proposed model achieves competitive performance on multi-hop reasoning benchmarks, closely paralleling proprietary LMs such as ChatGPT and Claude-instant. |
A Rose by Any Other Name: LLM-Generated Explanations Are Good Proxies for Human Explanations to Collect Label Distributions on NLI (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent research has shown that explanations provide valuable information for understanding human label variation (HLV) Large language models (LLMs) can approximate HJD from a few human-provided label-explanation pairs, but collecting explanations for every label is still time-consuming. |
| Approach: | They propose to use Large Language Models (LLMs) as annotators to generate model explanations for a few given human labels. |
| Outcome: | The proposed models can generate human-provided explanations from human labels, but they are still time-consuming. |
Explaining Language Model Predictions with High-Impact Concepts (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to explain large language models (LLMs) are mostly correlational and lack causal features due to compositional nature of languages. |
| Approach: | They propose a framework to provide impact-aware explanations for large language models that are robust to feature changes and influential to the model’s predictions. |
| Outcome: | The proposed explanations improve on real and synthetic tasks and are robust to feature changes and influential to the model’s predictions. |