Papers by Robert West
An Ordinal Latent Variable Model of Conflict Intensity (2023.acl-long)
Copied to clipboard
| Challenge: | Advances in automated event extraction yield massive data sets of “who did what to whom” micro-records that enable data-driven approaches to monitoring conflict. |
| Approach: | They propose a probabilistic generative model that assumes each observed event is associated with a latent intensity class. |
| Outcome: | The proposed model obtains comparatively good held-out predictive performance on a conflictual to cooperative scale. |
Activation Scaling for Steering and Interpreting Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a successful intervention should flip the correct with the wrong token, while remaining sparse. |
| Approach: | They propose to use activation scaling to flip the correct with the wrong token . they use gradient-based optimization to learn and evaluate a specific kind of efficient intervention . |
| Outcome: | The proposed method performs comparable with steering vectors but is much less minimal. |
Invariant Language Modeling (2022.emnlp-main)
Copied to clipboard
Maxime Peyrard, Sarvjeet Ghotra, Martin Josifoski, Vidhan Agarwal, Barun Patra, Dean Carignan, Emre Kiciman, Saurabh Tiwary, Robert West
| Challenge: | Existing methods to remove spurious correlations and biases involve expensive domain alignment. |
| Approach: | They propose a framework for learning invariant representations that generalize better across environments . they adapt a game-theoretic implementation of IRM to language models . |
| Outcome: | The proposed framework can remove structured noise, ignore correlations and achieve better generalization across environments. |
Better than Average: Paired Evaluation of NLP systems (2021.acl-long)
Copied to clipboard
| Challenge: | Evaluation in NLP is usually done by comparing the scores of competing systems . averaging scores independently and declaring the best system is difficult . |
| Approach: | They examine the use of averages to aggregate evaluation scores into a final number . they argue that the average ignores the pairing arising from the fact that systems are evaluated on the same test instances. |
| Outcome: | The proposed method ignores the pairing arising from the fact that systems are evaluated on the same test instances. |
Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectification (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a cost-effective and time-consuming way to capture public opinion and behavior, but their outputs are often biased and yield invalid estimates. |
| Approach: | They propose to use large language models to generate survey responses and rectification methods that debias population estimates to find out how human responses are best allocated between them. |
| Outcome: | The proposed methods reduce bias below 5% and increase sample size by up to 14% under a fixed budget. |
Can Language Models Recognize Convincing Arguments? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have found that large language models can generate persuasive content without engaging in human experimentation. |
| Approach: | They extend a dataset with debates, votes, and user traits to measure LLMs' ability to distinguish between strong and weak arguments, predict stances based on beliefs and demographic characteristics, and determine appeal of argument to individual based upon their traits. |
| Outcome: | The proposed tasks outperform human predictions in detecting convincing arguments in debates, votes, and user traits. |
Efficient Entity Candidate Generation for Low-Resource Languages (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches for cross-lingual entity linking are not suitable for English. |
| Approach: | They propose a candidate generation problem in cross-lingual entity linking with a focus on low-resource languages. |
| Outcome: | The proposed solution outperforms the state-of-the-art approach on 9 real-world datasets and query types. |
Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have great potential for synthetic data generation. |
| Approach: | They show that large language models can generate useful data even for complex tasks . they use a symmetric task difficulty asymmetry to prompt an LLM to generate plausible input text for a target output structure. |
| Outcome: | The proposed approach outperforms existing models by a substantial margin on closed information extraction tasks with 1.8M data points and 770M parameters. |
Strong Heuristics for Named Entity Linking (2022.naacl-srw)
Copied to clipboard
| Challenge: | Named entity linking (NEL) is a challenging task due to the frequency of unseen and emerging entities, which necessitates the use of unsupervised or zero-shot methods. |
| Approach: | They propose to map speaker-attributed quotes to a unique identifier in a referent knowledge base and then use it to resolve the ambiguity. |
| Outcome: | The proposed method disambiguates 94% and 63% of the mentions on Quotebank and the AIDA-CoNLL benchmark, respectively. |
Self-Recognition in Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for language modeling are very capable, but depend on commercial providers to build them. |
| Approach: | They propose a model-generated security question to assess self-recognition in LMs . they find no evidence of general or consistent self-reason in any examined LM . |
| Outcome: | The proposed approach can be externally administered to keep track of frontier models as it does not require access to internal model parameters or output probabilities. |
Low-Rank Subspaces for Unsupervised Entity Linking (2021.emnlp-main)
Copied to clipboard
| Challenge: | Entity linking is an important problem with many applications. |
| Approach: | They propose a method that exploits the fact that entities that are truly mentioned in a document tend to form a semantically dense subset of all candidate entities in the document. |
| Outcome: | a new method that outperforms existing methods on real-world datasets outperformed existing methods. |
Entity Insertion in Multilingual Linked Corpora: The Case of Wikipedia (2024.emnlp-main)
Copied to clipboard
| Challenge: | a new task for entity insertion in information networks is needed to integrate entities into multilingual linked corpora . text spans in the source and target entities are not available to insert a link to the target entity . a benchmark dataset in 105 languages is used to study the problem of entity inserted in information systems . |
| Approach: | They propose a framework for entity insertion that integrates entities into linked corpora . they compile a benchmark dataset in 105 languages and test it in a zero-shot manner . |
| Outcome: | The proposed framework outperforms baseline models on languages not seen during training with minimal performance drop. |
GenIE: Generative Information Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to open information extraction only work with unrealistically small numbers of entities and relations. |
| Approach: | They propose to use a transformer encoder-decoder model to extract triplets from unstructured text . they use 'generative information extraction' to generate triplet representations of information . |
| Outcome: | The proposed model is state-of-the-art on closed information extraction and generalizes from fewer training data points than baselines. |
GRAD: Generative Retrieval-Aligned Demonstration Sampler for Efficient Few-Shot Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) enriches prompts with external information, but its reliance on static databases constrains adaptability and can result in irrelevant demonstrations. |
| Approach: | They propose a Generative Retrieval-Aligned Demonstrator (GRAD) that trains an LLM model to generate input-specific concise demonstrations. |
| Outcome: | The proposed model outperforms strong baselines on Qwen2.5-14B across mathematical reasoning and advanced STEM questions. |
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia (2024.acl-long)
Copied to clipboard
Giovanni Monea, Maxime Peyrard, Martin Josifoski, Vishrav Chaudhary, Jason Eisner, Emre Kiciman, Hamid Palangi, Barun Patra, Robert West
| Challenge: | Large language models (LLMs) have an impressive ability to draw on novel information supplied in their context, yet the mechanisms underlying contextual grounding remain unknown. |
| Approach: | They propose a method to study grounding abilities using a counterfactual dataset constructed to clash with a model's parametric knowledge using Fakepedia. |
| Outcome: | The proposed method evaluates grounding abilities when the internal parametric knowledge clashes with the contextual information. |
Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing grammar-constrained decoding methods are limited to specific tasks . a grammar constraint is used to control the generation of LMs, but it is limited to a few tasks a task is not performed. |
| Approach: | They propose grammar-constrained decoding to control the generation of large language models . they demonstrate that grammars can describe the output space for a wider range of tasks . |
| Outcome: | The proposed grammars outperform unconstrained models on information extraction, entity disambiguation, and constituency parsing. |
Classifying Dyads for Militarized Conflict Analysis (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing research examines the origins of militarized conflict by examining bi-lateral relationships between entity pairs and multi-lateral relations among multiple entities. |
| Approach: | They propose to use Wikipedia to model dyadic and systemic causes to compare their correlations with conflict between two entities. |
| Outcome: | The proposed graphs show that Wikipedia articles of allies are semantically more similar than enemies. |
Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers (2025.acl-long)
Copied to clipboard
| Challenge: | A central question in multilingual language modeling is whether large language models develop a universal concept representation, disentangled from specific languages. |
| Approach: | They analyze latent representations during a word-translation task in transformer-based LLMs and extract the residual stream of the last token of the word to be translated and insert the mean at the corresponding positions in the forward pass. |
| Outcome: | The proposed model can translate a word in multiple languages without changing the language and vice versa. |
Tandem Training for Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | As language models improve, their actions and reasoning will become difficult or impossible for weaker agents and humans to follow, undermining interpretability and oversight. |
| Approach: | They propose a tandem training paradigm that allows models to adapt their language to weaker partners by intermittently and randomly sampling a frozen weak model instead of the strong model being trained. |
| Outcome: | The proposed model is able to produce solutions that are intelligible to weaker agents and humans while keeping task accuracy high. |
A Logical Fallacy-Informed Framework for Argument Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | Argument generation is crucial in daily life and has numerous online and offline applications. |
| Approach: | They propose a fallacy-informed preference optimization that includes a classification loss to capture the fine-grained information on fallacy types to help LLMs generate logically sound arguments. |
| Outcome: | The proposed method reduces fallacy errors by 17.5% on argument generation tasks and outperforms fine-tuned baselines and other preference optimization methods, such as DPO. |
Combining Constrained and Unconstrained Decoding via Boosting: BoostCD and Its Application to Information Extraction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to structured NLP tasks use autoregressive models trained on pairs of unstructured input text and structured output targets. |
| Approach: | They propose a model that combines constrained and unconstrained decoding in two phases to achieve two weak predictions. |
| Outcome: | The proposed model outperforms previous approaches both in and out of distribution, addressing several common errors identified in those approaches. |
Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are shown to perform better when asked to reason step-by-step before generating a final answer. |
| Approach: | They propose a framework to tailor small-sized LMs to generate correct reasoning steps and robustly reason over these steps. |
| Outcome: | The proposed framework outperforms four competitive baselines and improves the robustness and generalization ability of the reasoning LM, yielding higher performance on out-of-distribution test sets. |
On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation (2020.acl-main)
Copied to clipboard
| Challenge: | a standard evaluation setup for supervised machine learning tasks does not hold for natural language generation tasks. |
| Approach: | They propose to use reference-free machine translation evaluation to compare source texts to system translations to find key limitations. |
| Outcome: | The proposed metrics perform poorly as semantic encoders for reference-free machine translation evaluation. |
Localized Cultural Knowledge is Conserved and Controllable in Large Language Models (2026.findings-acl)
Copied to clipboard
Veniamin Veselovsky, Berke Argın, Benedikt Stroebl, Chris Wendler, Robert West, James Evans, Thomas L. Griffiths, Arvind Narayanan
| Challenge: | Large language models (LLMs) display language patterns influenced by their native tongue when learning new languages. |
| Approach: | They propose to quantify the explicit-implicit localization gap in large language models by using a new cultural localization benchmark and find large gaps in the majority of models. |
| Outcome: | The proposed model can generate culturally localized responses in multiple languages while maintaining language accuracy and task diversity. |
KLearn: Background Knowledge Inference from Summarization Data (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing summarization approaches focus on relevance, but background knowledge is not modeled by simple aggregation of large background corpora. |
| Approach: | They propose to define summary scoring functions that explicitly model background knowledge and compare them to baselines to see if they fit human judgments. |
| Outcome: | The proposed framework fits human judgments significantly better than baselines. |
Sketch-Guided Constrained Decoding for Boosting Blackbox Large Language Models without Logit Access (2024.acl-short)
Copied to clipboard
| Challenge: | Constrained decoding is a technique for enforcing constraints on language model outputs. |
| Approach: | They propose a technique for enforcing constraints on language model outputs . they propose auxiliary model that refines the initial output without constraints . |
| Outcome: | The proposed approach is able to refine the output of an unconstrained blackbox LLM without access to logits. |
REFINER: Reasoning Feedback on Intermediate Representations (2024.eacl-long)
Copied to clipboard
Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, Boi Faltings
| Challenge: | Language models (LLMs) have shown remarkable performance by explicitly generating intermediate inferences,e.g., chain-of-thought prompting. |
| Approach: | They propose a framework for finetuning LMs to generate intermediate reasoning steps while interacting with a critic model that provides automated feedback on the reasoning. |
| Outcome: | Empirical evaluations of REFINER on three diverse reasoning tasks show that it significantly improves over baseline models. |