Papers by Maxime Peyrard
Invariant Language Modeling (2022.emnlp-main)
Copied to clipboard
Maxime Peyrard, Sarvjeet Ghotra, Martin Josifoski, Vidhan Agarwal, Barun Patra, Dean Carignan, Emre Kiciman, Saurabh Tiwary, Robert West
| Challenge: | Existing methods to remove spurious correlations and biases involve expensive domain alignment. |
| Approach: | They propose a framework for learning invariant representations that generalize better across environments . they adapt a game-theoretic implementation of IRM to language models . |
| Outcome: | The proposed framework can remove structured noise, ignore correlations and achieve better generalization across environments. |
Better than Average: Paired Evaluation of NLP systems (2021.acl-long)
Copied to clipboard
| Challenge: | Evaluation in NLP is usually done by comparing the scores of competing systems . averaging scores independently and declaring the best system is difficult . |
| Approach: | They examine the use of averages to aggregate evaluation scores into a final number . they argue that the average ignores the pairing arising from the fact that systems are evaluated on the same test instances. |
| Outcome: | The proposed method ignores the pairing arising from the fact that systems are evaluated on the same test instances. |
Objective Function Learning to Match Human Judgements for Optimization-Based Summarization (N18-2)
Copied to clipboard
| Challenge: | In previous work on summarization, the objective function is based on ad-hoc assumptions about which quality aspects of a summary are relevant. |
| Approach: | They learn a summary-level scoring function including human judgments as supervision and automatically generated data as regularization. |
| Outcome: | The proposed method performs well across automatic and manual evaluations. |
What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Search (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work has demonstrated the promise of orchestrating large language models (LLMs) within evolutionary and agentic optimization systems. |
| Approach: | They present a large-scale study of LLM-guided evolutionary search . they find strong LLMs behave as local refiners, producing frequent improvements . weaker LLM optimizers exhibit large semantic drift, they say . |
| Outcome: | The results highlight the importance of trajectory analysis for understanding and improving LLM-based optimization systems. |
Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have great potential for synthetic data generation. |
| Approach: | They show that large language models can generate useful data even for complex tasks . they use a symmetric task difficulty asymmetry to prompt an LLM to generate plausible input text for a target output structure. |
| Outcome: | The proposed approach outperforms existing models by a substantial margin on closed information extraction tasks with 1.8M data points and 770M parameters. |
GenIE: Generative Information Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to open information extraction only work with unrealistically small numbers of entities and relations. |
| Approach: | They propose to use a transformer encoder-decoder model to extract triplets from unstructured text . they use 'generative information extraction' to generate triplet representations of information . |
| Outcome: | The proposed model is state-of-the-art on closed information extraction and generalizes from fewer training data points than baselines. |
Live Blog Corpus for Summarization (L18-1)
Copied to clipboard
| Challenge: | Live blogs are increasingly popular news format to cover breaking news and live events. |
| Approach: | They propose to collect corpora for automatic live blog summarization by a web-based system . they make the tools publicly available to encourage the research community . |
| Outcome: | The proposed method improves the accuracy of live blog summarization by allowing for public access to the corpus. |
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia (2024.acl-long)
Copied to clipboard
Giovanni Monea, Maxime Peyrard, Martin Josifoski, Vishrav Chaudhary, Jason Eisner, Emre Kiciman, Hamid Palangi, Barun Patra, Robert West
| Challenge: | Large language models (LLMs) have an impressive ability to draw on novel information supplied in their context, yet the mechanisms underlying contextual grounding remain unknown. |
| Approach: | They propose a method to study grounding abilities using a counterfactual dataset constructed to clash with a model's parametric knowledge using Fakepedia. |
| Outcome: | The proposed method evaluates grounding abilities when the internal parametric knowledge clashes with the contextual information. |
Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing grammar-constrained decoding methods are limited to specific tasks . a grammar constraint is used to control the generation of LMs, but it is limited to a few tasks a task is not performed. |
| Approach: | They propose grammar-constrained decoding to control the generation of large language models . they demonstrate that grammars can describe the output space for a wider range of tasks . |
| Outcome: | The proposed grammars outperform unconstrained models on information extraction, entity disambiguation, and constituency parsing. |
Language Model Decoding as Likelihood–Utility Alignment (2023.findings-eacl)
Copied to clipboard
Martin Josifoski, Maxime Peyrard, Frano Rajič, Jiheng Wei, Debjit Paul, Valentin Hartmann, Barun Patra, Vishrav Chaudhary, Emre Kiciman, Boi Faltings
| Challenge: | Existing studies only compare decoding algorithms in narrow scenarios, and their findings do not generalize across tasks. |
| Approach: | They propose a taxonomy of misalignment mitigation strategies to provide a unifying view of decoding as a tool for alignment. |
| Outcome: | The proposed taxonomy combines likelihood and utility assumptions to provide general statements about decoding as a tool for alignment across tasks. |
What Matters to an LLM? Behavioral and Computational Evidences from Summarization (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly entrusted with the management of information. |
| Approach: | They combine behavioral and computational analyses to find out what LLMs prioritize . they generate length-controlled summaries and derive empirical importance distributions . |
| Outcome: | The proposed model converges on consistent importance patterns and clusters more by family than by size. |
A Simple Theoretical Model of Importance for Summarization (P19-1)
Copied to clipboard
| Challenge: | Existing empirical summarization frameworks only identify signals correlating with the vague human intuition of Importance. |
| Approach: | They propose to define several concepts that were previously used only intuitively in summarization: redundancy, Relevance, and Informativeness. |
| Outcome: | The proposed models will improve summarization systems and improve their performance on standard datasets, while the notion of Importance remains latent. |
On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation (2020.acl-main)
Copied to clipboard
| Challenge: | a standard evaluation setup for supervised machine learning tasks does not hold for natural language generation tasks. |
| Approach: | They propose to use reference-free machine translation evaluation to compare source texts to system translations to find key limitations. |
| Outcome: | The proposed metrics perform poorly as semantic encoders for reference-free machine translation evaluation. |
KLearn: Background Knowledge Inference from Summarization Data (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing summarization approaches focus on relevance, but background knowledge is not modeled by simple aggregation of large background corpora. |
| Approach: | They propose to define summary scoring functions that explicitly model background knowledge and compare them to baselines to see if they fit human judgments. |
| Outcome: | The proposed framework fits human judgments significantly better than baselines. |
Date Fragments: A Hidden Bottleneck of Tokenization for Temporal Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | a tokeniser that splits "2025-03-14" into "20", "25", "-0", "3", "-1", "4" obscures temporal cues and obscures structure . excessive fragmentation correlates with accuracy drops of up to 10 points on uncommon dates . |
| Approach: | They propose a date fragmentation ratio measure that measures how faithfully a tokeniser preserves multi-digit date components. |
| Outcome: | The proposed method shows that excessive fragmentation correlates with accuracy drops of up to 10 points on uncommon dates like historical and futuristic dates. |
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance (D19-1)
Copied to clipboard
| Challenge: | Existing evaluation metrics are not capable of evaluating text quality. |
| Approach: | They propose a metric that compares system output against reference texts based on semantics rather than surface forms. |
| Outcome: | The proposed metric shows a high correlation with human judgment of text quality on a number of text generation tasks. |
REFINER: Reasoning Feedback on Intermediate Representations (2024.eacl-long)
Copied to clipboard
Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, Boi Faltings
| Challenge: | Language models (LLMs) have shown remarkable performance by explicitly generating intermediate inferences,e.g., chain-of-thought prompting. |
| Approach: | They propose a framework for finetuning LMs to generate intermediate reasoning steps while interacting with a critic model that provides automated feedback on the reasoning. |
| Outcome: | Empirical evaluations of REFINER on three diverse reasoning tasks show that it significantly improves over baseline models. |
Studying Summarization Evaluation Metrics in the Appropriate Scoring Range (P19-1)
Copied to clipboard
| Challenge: | Existing evaluation metrics are compared based on their ability to correlate with humans, but they disagree in the higher-scoring range in which current systems operate. |
| Approach: | They show that evaluation metrics which behave similarly on these datasets strongly disagree in the higher-scoring range in which current systems operate. |
| Outcome: | The evaluation metrics which behave similarly on these datasets strongly disagree in the higher-scoring range in which current systems operate. |