Papers with IG
The Impact of Integration Step on Integrated Gradients (2024.eacl-srw)
Copied to clipboard
| Challenge: | Integrated gradients (IG) are a powerful tool for explaining the internal structure of a language model. |
| Approach: | They propose to customize the step count for each instance to minimize the error. |
| Outcome: | The optimal number of steps to maintain minimal error varies from instance to instance. |
A Character-Centric Creative Story Generation via Imagination (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing narrative generation models lack diversity and character depth, but they are inadequate for human creativity. |
| Approach: | They propose a novel story generation framework called CCI that leverages images to create stories that are diverse and creative in their themes and richer in content. |
| Outcome: | The proposed framework significantly improves various aspects of the stories’ creativity. |
AD-KD: Attribution-Driven Knowledge Distillation for Language Model Compression (2023.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge distillation methods focus on the transfer of model-specific knowledge but overlook data-specific information. |
| Approach: | They propose an attribution-driven knowledge distillation approach which explores the token-level rationale behind the teacher model and transfers attribution knowledge to the student model. |
| Outcome: | The proposed method outperforms state-of-the-art methods on the GLUE benchmark and shows that it is more efficient than existing methods. |
Sequential Integrated Gradients: a simple but effective method for explaining language models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing explanation methods such as Integrated Gradients (IG) produce a path for each word of a sentence simultaneously, which can lead to sentences with no clear meaning or a significantly different meaning compared to the original one. |
| Approach: | They propose to use a sequenced integrated gradient method to fix every word in a sentence and move it along a straight path to the word of interest. |
| Outcome: | The proposed method improves on Integrated Gradients (IG) and DIG, but can produce sentences with different meanings than the original one. |
Discretized Integrated Gradients for Explaining Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Integrated Gradients (IG) is widely adopted due to its desirable explanation axioms and the ease of gradient computation. |
| Approach: | They propose an attribution-based explanation algorithm that uses averaging the model's output gradient interpolated along a straight-line path in the input data space. |
| Outcome: | The proposed method is compared with IG on multiple sentiment classification datasets. |
GuessingGame: Measuring the Informativeness of Open-Ended Questions in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models excel at factual recall, arithmetic reasoning, multi-turn dialogue . their capacity as askers, formulating strategic, adaptive, and information-seeking questions, remains less explored . |
| Approach: | They propose a protocol for evaluating large language models as strategic question-askers . they propose entropy-based methods that filter candidates via ConceptNet and Bayesian method that tracks belief updates over semantic concepts . |
| Outcome: | The proposed method is model-agnostic and supports post hoc analysis. |
Towards Informative Few-Shot Prompt with Maximum Information Gain for In-Context Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language models (LLMs) have the capability to engage In-context Learning (ICL) however, this particular learning paradigm suffers from high instability stemming from factors such as input distribution, order and prompt formats. |
| Approach: | They propose to quantify the information gain obtained in prediction after observing a given example candidate and to sample those with maximum IG. |
| Outcome: | The proposed method can yield an average relative improvement of 14.3% across six classification tasks using three LLMs. |
Can Large Language Models Infer Human Actions and Motives? Evaluation in Social Prediction and Inspection Games (2026.findings-acl)
Copied to clipboard
| Challenge: | Game theory provides a framework for studying human behaviors through incentivized games that simulate social situations. |
| Approach: | They used two validated games from the cognitive science literature to study how well several recent open- and closed-source LLMs predict player actions with underlying human motives. |
| Outcome: | The results show that state-of-the-art LLMs can achieve accuracy close to human levels in predicting players’ actions with underlying human motives in SPGs, but failed to recognize statistical patterns in players’ action. |