Papers by Damien Sileo
Mining Discourse Markers for Unsupervised Sentence Representation Learning (N19-1)
Copied to clipboard
| Challenge: | Current state of the art systems in NLP heavily rely on manually annotated datasets, which are expensive to obtain and are ineffective to extract. |
| Approach: | They propose to automatically discover sentence pairs with relevant discourse markers and apply it to massive amounts of data. |
| Outcome: | The proposed method can learn transferable sentence embeddings from 174 discourse markers even for rare markers such as “coincidentally” or “amazingly”. |
Generating Multiple-choice Questions for Medical Question Answering with Distractors and Cue-masking (2024.lrec-main)
Copied to clipboard
| Challenge: | Medical multiple-choice question answering (MCQA) requires high accuracy to be useful in practice. |
| Approach: | They propose to focus masked language modeling on disease name prediction when using medical encyclopedic paragraphs as input. |
| Outcome: | The proposed model outperforms the masked language model on disease name prediction and masks the cues to the answers. |
Logic Haystacks: Probing LLMs’ Long-Context Logical Reasoning (Without Easily Identifiable Unrelated Padding) (2026.eacl-short)
Copied to clipboard
| Challenge: | Recent large language models claim long context windows, but evaluations often involve simple retrieval tasks or synthetic tasks padded with irrelevant text. |
| Approach: | They use grammars to generate simplified English with logical representations to create long input text while controlling its semantics. |
| Outcome: | The proposed model performs better with realistic distractors than with standard models. |
DiscSense: Automated Semantic Analysis of Discourse Markers (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing models for predicting discourse markers have been used to study link between markers and semantic relations . |
| Approach: | They use a model trained to predict discourse markers between sentence pairs to predict plausible markers between sentences with a known semantic relation. |
| Outcome: | The proposed method predicts markers between sentence pairs with a known semantic relation . the resulting dataset, named DiscSense, is publicly available . |
tasksource: A Large Collection of NLP tasks with a Structured Dataset Preprocessing Framework (2024.lrec-main)
Copied to clipboard
| Challenge: | Several initiatives release harmonized datasets or provide harmonization codes to preprocess datasets into a consistent format. |
| Approach: | They propose an annotation framework that enables concise, readable, and reusable annotations. |
| Outcome: | The proposed framework outperforms all publicly available text encoders on all tasks. |
Tau-Eval: A Unified Evaluation Framework for Useful and Private Text Anonymization (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Existing studies on text anonymization prioritize privacy preservation at the expense of utility, relying on reference-based metrics like ROUGE, BERTScore, or METEOR to measure textual fidelity. |
| Approach: | They propose an open-source framework for benchmarking text anonymization methods through the lens of privacy and utility task sensitivity. |
| Outcome: | The proposed framework is open-source and provides a Python library, documentation and tutorials. |
Probing neural language models for understanding of words of estimative probability (2023.starsem-1)
Copied to clipboard
| Challenge: | Words of Estimative Probability (WEP) are phrases used to express the plausibility of a statement. |
| Approach: | They propose to use a UNLI dataset to assess language models' ability to process WEPs. |
| Outcome: | The proposed model can accurately capture the consensual probability level associated with each WEP. |
MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logic (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Theory of Mind (ToM) is a critical component of intelligence but its assessment remains the subject of heated debates. |
| Approach: | They propose to use dynamic epistemic logic to isolate a particular component of ToM and generate controlled problems in English natural language. |
| Outcome: | The proposed language model scales from 70M to 6B and 350M to 174B do not consistently yield better results than random chance. |
Analysis and Prediction of NLP Models via Task Embeddings (2022.lrec-1)
Copied to clipboard
| Challenge: | Pretrained transformer-based encoders can be used to acquire rich text representations but need additional task supervision to be useful for downstream tasks. |
| Approach: | They propose a transformer to each MetaEval task and a neural network with a weighted encoder to perform the embeddings. |
| Outcome: | The proposed model outperforms baselines on GLUE tasks and can be used as a benchmark for future transfer learning research. |
Scaling Synthetic Logical Reasoning Datasets with Context-Sensitive Declarative Grammars (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing proof generation algorithms bias reasoning toward specific proof traces and limit extensibility. |
| Approach: | They propose a framework with flexible context-sensitive rules binding multiple languages . they propose to use English verbalization of predicates to enhance logical reasoning . |
| Outcome: | The proposed framework surpasses GPT-4 in accuracy on a human-authored logic dataset by 12%. |
A Pragmatics-Centered Evaluation Framework for Natural Language Understanding (2022.lrec-1)
Copied to clipboard
| Challenge: | a number of studies have suggested that models induce universal text representations . current benchmarks focus on semantic phenomena, so pragmatics needs to be the focus . |
| Approach: | They propose a benchmark that unites 11 pragmatics-focused evaluation datasets for English. |
| Outcome: | The proposed benchmark shows that natural language inference does not result in genuinely universal representations. |
Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for anonymizing textual documents lack flexibility to adapt to diverse requirements. |
| Approach: | They propose a task formulation in which anonymization strategies are automatically adapted to specific privacy–utility requirements. |
| Outcome: | The proposed framework achieves better privacy–utility trade-off than existing baselines on open-source language models while remaining computationally efficient and effective on larger closed-source models. |
DISRPT: A Multilingual, Multi-domain, Cross-framework Benchmark for Discourse Processing (2024.lrec-main)
Copied to clipboard
Chloé Braud, Amir Zeldes, Laura Rivière, Yang Janet Liu, Philippe Muller, Damien Sileo, Tatsuya Aoyama
| Challenge: | DISRPT is a multilingual, multi-domain, and cross-framework benchmark dataset for discourse processing. |
| Approach: | They present a multilingual, multi-domain, and cross-framework benchmark dataset for discourse processing that includes 13 languages and 24 corpora covering about 4 millions tokens and around 250,000 discourse relation instances from 4 discourse frameworks. |
| Outcome: | The DISRPT dataset includes data from 24 corpora covering about 4 millions tokens and around 250,000 discourse relation instances from 4 discourse frameworks. |