Papers by Barun Patra
Invariant Language Modeling (2022.emnlp-main)
Copied to clipboard
Maxime Peyrard, Sarvjeet Ghotra, Martin Josifoski, Vidhan Agarwal, Barun Patra, Dean Carignan, Emre Kiciman, Saurabh Tiwary, Robert West
| Challenge: | Existing methods to remove spurious correlations and biases involve expensive domain alignment. |
| Approach: | They propose a framework for learning invariant representations that generalize better across environments . they adapt a game-theoretic implementation of IRM to language models . |
| Outcome: | The proposed framework can remove structured noise, ignore correlations and achieve better generalization across environments. |
EfficientXLang: Towards Improving Token Efficiency Through Cross-Lingual Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Despite recent advances in Reasoning Language Models, most research focuses solely on English, even though many models are pretrained on multilingual data. |
| Approach: | They evaluate three open-source RLMs: DeepSeek R1, Qwen 2.5, and Qwend 3 across four math datasets and seven typologically diverse languages. |
| Outcome: | The proposed model reduces token usage and preserves accuracy even after translation into English. |
To Schedule or not to Schedule: Extracting Task Specific Temporal Entities and Associated Negation Constraints (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for date-time entity extraction from text are task agnostic, resulting in insufficient results for task specific date-timing extraction. |
| Approach: | They propose a model for extracting subset of date-time entities from text and their negation constraints. |
| Outcome: | The proposed model achieves an absolute gain of 19% f-score points compared to baseline methods in detecting date-time entities relevant to scheduling meetings and a 4% improvement over baseline methods for detecting negation constraints over date-timing entities. |
ScopeIt: Scoping Task Relevant Sentences in Documents (2020.coling-industry)
Copied to clipboard
| Challenge: | a problem faced by conversational agents working with large documents is the frequent presence of information that is irrelevant to the agent. |
| Approach: | They propose a neural model for scoping relevant information from a large document . they show that the model performs better with emails than existing baselines . |
| Outcome: | The proposed model improves intent detection and entity extraction tasks without drop in recall. |
Weakly Supervised Attention Networks for Entity Recognition (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to entity recognition require large amounts of token-level data, which can be expensive and cumbersome to obtain. |
| Approach: | They propose a weakly supervised model that can be annotated at word level from a corpus containing binary presence/absence labels. |
| Outcome: | The proposed model performs reasonably well on the task of entity recognition despite not having access to token-level ground truth data. |
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia (2024.acl-long)
Copied to clipboard
Giovanni Monea, Maxime Peyrard, Martin Josifoski, Vishrav Chaudhary, Jason Eisner, Emre Kiciman, Hamid Palangi, Barun Patra, Robert West
| Challenge: | Large language models (LLMs) have an impressive ability to draw on novel information supplied in their context, yet the mechanisms underlying contextual grounding remain unknown. |
| Approach: | They propose a method to study grounding abilities using a counterfactual dataset constructed to clash with a model's parametric knowledge using Fakepedia. |
| Outcome: | The proposed method evaluates grounding abilities when the internal parametric knowledge clashes with the contextual information. |
A Practical Analysis of Human Alignment with *PO (2025.findings-naacl)
Copied to clipboard
| Challenge: | Prior research focused on identifying the best-performing method to varying hyperparameters . prior research focused primarily on a grid search, which can be impractical for general practitioners . |
| Approach: | They propose a preference optimization method that is more stable across hyperparameters and reduces the average response length. |
| Outcome: | The proposed method increases likelihood of achieving better results through various metrics, such as KL divergence and response length. |
Language Model Decoding as Likelihood–Utility Alignment (2023.findings-eacl)
Copied to clipboard
Martin Josifoski, Maxime Peyrard, Frano Rajič, Jiheng Wei, Debjit Paul, Valentin Hartmann, Barun Patra, Vishrav Chaudhary, Emre Kiciman, Boi Faltings
| Challenge: | Existing studies only compare decoding algorithms in narrow scenarios, and their findings do not generalize across tasks. |
| Approach: | They propose a taxonomy of misalignment mitigation strategies to provide a unifying view of decoding as a tool for alignment. |
| Outcome: | The proposed taxonomy combines likelihood and utility assumptions to provide general statements about decoding as a tool for alignment across tasks. |
Scaling Laws for Multilingual Language Models (2025.findings-acl)
Copied to clipboard
Yifei He, Alon Benhaim, Barun Patra, Praneetha Vaddamanu, Sanchit Ahuja, Parul Chopra, Vishrav Chaudhary, Han Zhao, Xia Song
| Challenge: | Existing scaling laws for language models are limited to a limited number of languages, but they can be applied to arbitrary number of different languages. |
| Approach: | They propose a scaling law for general-purpose decoder-only language models trained on multilingual data that shifts focus from individual languages to language families. |
| Outcome: | The proposed scaling law can be applied to models trained on multilingual data . it can be used to predict performance across multiple languages and models . |
On the Adaptation of Unlimiformer for Decoder-Only Transformers (2024.lrec-main)
Copied to clipboard
| Challenge: | despite efforts in the community, most common models have a context length of 4k or less. |
| Approach: | They propose to adapt a vector-retrieval augmentation method to decoder-only transformers . they also expand the experimental setup on summarization to include a new task and an instruction-tuned model . |
| Outcome: | The proposed model performs on par with a model with 2x the context length. |
Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces (P19-1)
Copied to clipboard
| Challenge: | Recent work on bilingual lexicon induction (BLI) relies on an assumption about the isometry of two embedding spaces. |
| Approach: | They propose a semi-supervised approach that relaxes the isometric assumption while leveraging limited aligned bilingual lexicons and a larger set of unaligned word embeddings. |
| Outcome: | The proposed method obtains state-of-the-art results on 15 of 18 language pairs on the MUSE dataset and does particularly well when the embedding spaces don’t appear isometric. |
Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning (2023.acl-long)
Copied to clipboard
Barun Patra, Saksham Singhal, Shaohan Huang, Zewen Chi, Li Dong, Furu Wei, Vishrav Chaudhary, Xia Song
| Challenge: | XY-LENT: X-Y bitext enhanced Language ENcodings achieves state-of-the-art performance over 5 cross-lingual tasks within all model size bands. |
| Approach: | They propose a method for building multilingual representation models that are competitive with existing models and more parameter efficient. |
| Outcome: | The proposed model outperforms XLM-R XXL and is 5x and 6x smaller respectively. |
A Length-Extrapolatable Transformer (2023.acl-long)
Copied to clipboard
Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, Furu Wei
| Challenge: | Existing Transformers can only deal with the in-distribution size of inputs. |
| Approach: | They propose a relative position embedding to explicitly maximize attention resolution . they also use blockwise causal attention during inference for better resolution a . |
| Outcome: | The proposed model achieves strong performance in interpolation and extrapolation settings. |
Everything you need to know about Multilingual LLMs: Towards fair, performant and reliable models for languages of the world (2023.acl-tutorials)
Copied to clipboard
| Challenge: | Responsible AI issues such as fairness, bias and toxicity will be discussed in this tutorial . |
| Approach: | This tutorial will describe various aspects of scaling up language technologies to many of the world’s languages by describing the latest research in Massively Multilingual Language Models (MMLMs). |
| Outcome: | This tutorial will cover various aspects of scaling up language technologies to many of the world's languages by describing the latest research in multilingual models. |
On Efficiently Acquiring Annotations for Multilingual Models (2022.acl-short)
Copied to clipboard
| Challenge: | a recent study shows that joint learning across multiple languages performs better than the aforementioned approaches . traditional approaches to support NLP tasks require a lot of annotations to perform . a new approach is to train a model for each language with annotation budget divided equally among them . |
| Approach: | They propose a method for joint learning across multiple languages using a single model . they show that active learning provides additional, complementary benefits . |
| Outcome: | The proposed method outperforms other models on a diverse set of tasks . it can arbitrate its annotation budget to query languages it is less certain on . |