Papers by Lior Wolf
Caption Enriched Samples for Improving Hateful Memes Detection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for classifying memes are difficult to perform, with human accuracy only about 85% . recent state-of-the-art models perform considerably less accurately, achieving up to 64.73% accuracy. |
| Approach: | They propose to use an off-the-shelf caption generator to capture the first image and overlayed text. |
| Outcome: | The proposed tool improves classification accuracy for unimodal and multimodal models . the proposed tool can be used to model the contrast between image content and overlayed text . |
Faithful Serum: Mitigating the Faithfulness Gap in Textual Explanations of LLM Decisions via Attribution Guidance (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has focused on generating convincing rationales that appear to be subjectively faithful, but it remains unclear whether these explanations are epistemic faithful. |
| Approach: | They propose a method that enhances epistemic faithfulness by guiding explanation generation through attention-level interventions, informed by token-level heatmaps. |
| Outcome: | The proposed method significantly improves epistemic faithfulness across multiple models, benchmarks, and prompts. |
Non-Adversarial Unsupervised Word Translation (D18-1)
Copied to clipboard
| Challenge: | Unsupervised word translation from non-parallel inter-lingual corpora has attracted much research interest. |
| Approach: | They propose a method that aligns two words in two languages and iteratively refines the alignment. |
| Outcome: | The proposed method achieves better performance than state-of-the-art deep adversarial approaches on word translation of European and Non-European languages. |
Safeguarding Language Models via Self-Destruct Trapdoor (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing mechanisms to restrict behavior of language models (LMs) are vulnerable to misuse and misalignment. |
| Approach: | They propose a mechanism to restrict specific behaviors in language models by exploiting hardware properties. |
| Outcome: | The proposed mechanism can be applied to trigger overflows for specific behaviors or target hardware malfunctions. |
Knowledge Editing in Language Models via Adapted Direct Preference Optimization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can become outdated over time due to lack of updated world knowledge. |
| Approach: | They propose to use weight updates to improve LLM alignment without retraining . they propose a method that continually updates the knowledge stored in the model . |
| Outcome: | The proposed method is more effective than existing methods on large datasets and models. |
PRILoRA: Pruned and Rank-Increasing Low-Rank Adaptation (2024.findings-eacl)
Copied to clipboard
| Challenge: | Several approaches to parameter-efficient fine-tuning have been proposed . low-rank Adaptation (LoRA) does not consider the varying importance of each layer . |
| Approach: | They propose a method that allocates a different rank for each layer and performs pruning throughout the training process. |
| Outcome: | The proposed method is based on eight GLUE benchmarks and is currently the state of the art. |
Describing Sets of Images with Textual-PCA (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to describe images using a common theme is needed to describe the images . a grammatical phrase is not sufficient to describe an image set, since captioning engines are not general enough. |
| Approach: | They propose a method to capture attributes of images and variations within a set . they use a pretrained vision-language model to generate a centroid phrase with the largest average similarity . |
| Outcome: | The proposed method captures the essence of image sets and describes them in a semantically meaningful way . it is easy for humans to identify and describe a common theme, but it is not generic enough . |
LLM Questionnaire Completion for Automatic Psychiatric Assessment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Psychiatric evaluations are heavily based on patient verbal reports of disturbed feelings, thoughts, behaviors, and their changes over time. |
| Approach: | They employ a Large Language Model to convert unstructured psychological interviews into structured questionnaires spanning various psychiatric and personality domains. |
| Outcome: | The proposed model improves diagnostic accuracy compared to baselines. |
Maximal Multiverse Learning for Promoting Cross-Task Generalization of Fine-Tuned Language Models (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent studies suggest the use of general language models for improving natural language processing tasks. |
| Approach: | They propose a method that leverages the second phase to its fullest by applying an extensive number of parallel classifier heads, which are enforced to be orthogonal, while adaptively eliminating the weaker heads during training. |
| Outcome: | The proposed method improves the generalization ability of BERT, sometimes leading to a +9% gain in accuracy. |
TensorLens: End-to-End Transformer Analysis via High-Order Attention Tensors (2026.acl-long)
Copied to clipboard
| Challenge: | Existing attention-aggregation methods focus on individual attention heads or layers, failing to account for the model’s global behavior. |
| Approach: | They propose a unified attention representation that captures the entire transformer as a single, input-dependent linear operator expressed through a high-order attention-interaction tensor. |
| Outcome: | The proposed model encapsulates the entire transformer as a single, input-dependent linear operator expressed through a high-order attention-interaction tensor. |
MTAdam: Automatic Balancing of Multiple Training Loss Terms (2021.emnlp-main)
Copied to clipboard
| Challenge: | In supervised and unsupervised learning, adding loss terms often leads to improved performance. |
| Approach: | They propose an algorithm that balances the gradient magnitude of loss terms across all layers . they use Adam to add loss terms to neural models, but add more terms as they are added . |
| Outcome: | The proposed method improves performance and improves training outcomes. |
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent interpretability methods project weights and hidden states obtained from the forward pass to the models’ vocabularies, helping to uncover how information flows within LMs. |
| Approach: | They propose to cast a gradient matrix as a low-rank linear combination of forward and backward passes’ inputs and then to project these gradients into vocabulary items. |
| Outcome: | The proposed method can be cast as a low-rank linear combination of forward and backward passes’ inputs and project these gradients into vocabulary items. |
Mitigating Copy Bias in In-Context Learning through Neuron Pruning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models have shown impressive few-shot in-context learning abilities, but they are prone to a ‘copying bias’, where they copy answers from provided examples instead of learning the underlying patterns. |
| Approach: | They propose a method to prune neurons that prioritize copying over generalization and adopt a task-recognition perspective on ICL and examine task vectors induced by the model. |
| Outcome: | The proposed method improves performance across a diverse set of ICL tasks while maintaining or improving the model’s general capabilities. |
Focus Your Attention (with Adaptive IIR Filters) (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to deal with long-range data processing are implicit convolutions and regularized parameterization. |
| Approach: | They propose a new layer where dynamic (i.e., input-dependent) IIR filters are used to process the input sequence prior to applying conventional attention. |
| Outcome: | The proposed layer performs on-par with state-of-the-art networks with a fraction of their parameters and time complexity that is sub-quadratic with input size. |
Reversed Attention: On The Gradient Descent Of Attention Layers In GPT (2025.naacl-long)
Copied to clipboard
| Challenge: | In this work, we examine the attention maps obtained from the backward pass of attention, which we call "Reversed Attention" (RA). |
| Approach: | They propose to use a method called "attention patching" to alter the forward pass of attention without modifying the model's weights. |
| Outcome: | The proposed method enables the model to alter the forward pass of attention without altering the model’s weights. |
Segment-Based Attention Masking for GPTs (2025.acl-long)
Copied to clipboard
| Challenge: | Causal masking is a fundamental component in Generative Pre-Trained Transformers (GPT) models, playing a crucial role during training. |
| Approach: | They propose to apply causal masking to all input tokens step-by-step, mimicking the generation process. |
| Outcome: | The proposed model can process the entire user prompt at once, but it is applied to all input tokens step-by-step, mimicking the generation process. |
The Hidden Attention of Mamba Models (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Mamba models can be used for multiple domains, including NLP, long-range sequence processing, and computer vision. |
| Approach: | They add a third view and show that Mamba models can be viewed as attention-driven models. |
| Outcome: | The proposed model can be viewed as attention-driven and empirically compare it to the attention-based models of transformers. |