LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Models (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Transformer-based language models (LMs) are opaque and unexplained, causing problems for endusers and developers who wish to debug or fix their behaviour. |
| Approach: | They propose an interactive debugger tool for transformer-based LMs that provides a fine-grained interpretation of the model's internal prediction process and a powerful framework for intervening in LM behavior. |
| Outcome: | The proposed tool provides a fine-grained interpretation of the model's internal prediction construction process, and a powerful framework for intervening in LM behavior. |
Similar Papers
LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Models (2024.acl-demos)
Copied to clipboard
| Challenge: | Existing tools focus on isolated parts of the decision-making process, but LM-TT makes the entire prediction process transparent. |
| Approach: | They present an open-source toolkit for analyzing the internal workings of Transformer-based language models. |
| Outcome: | The LM Transparency Tool makes the entire prediction process transparent . it shows the importance of specific component at each step . |
Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step by Step (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are leading progress in code generation, but they are underutilized in the literature. |
| Approach: | They propose a debugging framework that allows LLMs to refine their generated programs with the runtime execution information. |
| Outcome: | The proposed framework improves the baseline performance by 9.8% across the HumanEval, MBPP, and TransCoder benchmarks. |
InterpreT: An Interactive Visualization Tool for Interpreting Transformers (2021.eacl-demos)
Copied to clipboard
Vasudev Lal, Arden Ma, Estelle Aflalo, Phillip Howard, Ana Simoes, Daniel Korat, Oren Pereg, Gadi Singer, Moshe Wasserblat
| Challenge: | Using Transformer-based models for NLU/NLP tasks is a growing interest . but there are many open questions regarding the behavior of these models . |
| Approach: | They present an interactive visualization tool for interpreting Transformer-based models. |
| Outcome: | The tool can track and visualize token embeddings through each layer of a Transformer, highlight distances between certain token embeds, and identify task-related functions of attention heads using new metrics. |
Language Model Transformers as Evaluators for Open-domain Dialogues (2020.coling-main)
Copied to clipboard
| Challenge: | Computer-based systems for communication with humans are a cornerstone of AI research since the 1950s. |
| Approach: | They propose to use transformer neural networks to predict one or more words based on an already given context to provide an efficient, automatic indication of dialogue quality. |
| Outcome: | The proposed language models show that human evaluators have a positive correlation between the output of the models and scores. |
XMD: An End-to-End Framework for Interactive Explanation-Based Debugging of NLP Models (2023.acl-demo)
Copied to clipboard
Dong-Ho Lee, Akshen Kadakia, Brihi Joshi, Aaron Chan, Ziyi Liu, Kiran Narahari, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, Xiang Ren
| Challenge: | Existing models are susceptible to learning spurious biases that do not reflect the underlying task. |
| Approach: | They propose an open-source framework for explanation-based model debugging that allows users to provide various forms of feedback on model explanations. |
| Outcome: | The proposed framework improves model’s OOD performance on text classification tasks by up to 18%. |
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space (2022.emnlp-main)
Copied to clipboard
| Challenge: | Fig. 1 shows how feed-forward network (FFN) layers are utilized to build LMs. |
| Approach: | They reverse-engineer the operation of feed-forward network layers to find out how they work . they show that each update can be decomposed to sub-updates corresponding to single parameter vectors . |
| Outcome: | The proposed model reduces the toxicity of GPT2 by almost 50% and improves computation efficiency with a simple early exit rule, saving 20% of computation on average. |
Explanation-Based Human Debugging of NLP Models: A Survey (2021.tacl-1)
Copied to clipboard
| Challenge: | In this paper, we review literature that exploits explanations to enable humans to fix bugs in NLP models. |
| Approach: | They review literature that exploits explanations to enable humans to fix bugs in NLP models. |
| Outcome: | The proposed approach is described in detail in this paper and is based on three dimensions of the problem explanation-based human debugging (EBHD). |
VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work in interpretability suggests we can project weights and hidden states of transformer-based language models (LMs) to their vocabulary space, a transformation that makes them more human interpretable. |
| Approach: | They propose a tool to visualize a forward pass of Generative Pre-trained Transformers as an interactive flow graph with nodes representing neurons or hidden states and edges representing interactions between them. |
| Outcome: | The proposed visualization simplifies huge amounts of data into easy-to-read graphs that can reflect the models’ internal processing, uncovering the contribution of each component to the models' final prediction. |
Ecco: An Open Source Library for the Explainability of Transformer Language Models (2021.acl-demo)
Copied to clipboard
| Challenge: | Existing models that use the Transformer architecture are lag behind our ability to scale them. |
| Approach: | They propose an open-source library for the explainability of Transformer-based NLP models that captures, analyzes, visualizes, and interactively explores the inner mechanics of these models. |
| Outcome: | The proposed tools capture, analyze, visualize, and explore the inner workings of Transformer-based language models. |
Transformers: State-of-the-Art Natural Language Processing (2020.emnlp-demos)
Copied to clipboard
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, Alexander Rush
| Challenge: | Transformers is an open-source library that aims to open up advances in natural language processing to the wider machine learning community. |
| Approach: | they propose an open-source library that aims to open up advances in machine learning to the wider community. |
| Outcome: | Transformers is an open-source library with the goal of opening up these advances to the wider machine learning community. |