Papers by Benoît Sagot
Explicit Learning and the LLM in Machine Translation (2025.emnlp-main)
Copied to clipboard
| Challenge: | a growing number of researchers are examining whether large language models can learn to translate a "new" language using grammar books. |
| Approach: | They examine an LLM's ability to learn new languages using grammar books . authors suggest alternative fine-tuning strategies to improve explicit learning . |
| Outcome: | The proposed model can learn low-resource languages described in grammar books but lacking extensive corpora. |
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT (2021.eacl-main)
Copied to clipboard
| Challenge: | Multilingual pretrained language models have demonstrated remarkable zero-shot cross-lingual transfer capabilities. |
| Approach: | They propose to use a layer ablation technique to create a multilingual model that is viewed as a stacking of two sub-networks: a language-agnostic encoder and a task-specific predictor. |
| Outcome: | The proposed model can perform zero-shot cross-lingual transfer for many languages. |
Tree of Problems: Improving structured problem solving with compositionality (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable performance across multipletasks through in-context learning. |
| Approach: | They propose a Tree of Problems (ToP) that is a simpler version of Tree of Thoughts (toT) they propose 'in-context learning' is the ability of Large Language Models (LLMs) to perform a task with the help of a few demonstrations within their context. |
| Outcome: | The proposed approach outperforms ToT and GoT and performs better on complex reasoning tasks. |
Towards Zero-Shot Multimodal Machine Translation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Current multimodal machine translation systems rely on fully supervised data, which is costly to collect and prevents extension of MMT to language pairs with no such data. |
| Approach: | They propose a method to bypass the need for fully supervised data to train MMT systems . they adapt a strong text-only machine translation model to a visually conditioned language model and a divergence test set to evaluate how well models use images to disambiguate translations. |
| Outcome: | The proposed method can generalize to languages with no fully supervised training data. |
Methodological Aspects of Developing and Managing an Etymological Lexical Resource: Introducing EtymDB-2.0 (2020.lrec-1)
Copied to clipboard
| Challenge: | Diachronic lexical information is increasingly used in historical linguistics and in NLP . etymological resources need to be fine-grained, large-coverage and accurate . |
| Approach: | They propose guidelines to generate etymological lexical resources for each step of the life-cycle of an ethymology . they introduce EtymDB 2.0, an 'etiological database' generated from the Wiktionary . |
| Outcome: | The proposed resources are generated for each step of the life-cycle of an etymological lexicon: creation, update, evaluation, dissemination, and exploitation. |
In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that in-context examples for machine translation are beneficial for high-resource languages. |
| Approach: | They propose to use in-context examples for machine translation (MT) they argue that similarity-based selection can improve MT . |
| Outcome: | The proposed approach improves machine translation (MT) and low-resource languages. |
Establishing a New State-of-the-Art for French Named Entity Recognition (2020.lrec-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a task consisting in identifying text spans that denote named entities such as person, location and organization names. |
| Approach: | They manually annotated the French TreeBank with information related to named entities . they sketch the underlying annotation guidelines and provide a few figures about the annotations . |
| Outcome: | The French TreeBank is the main source of morphosyntactic and syntactical annotations for French. |
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing web crawling pipelines are used to collect large corpora raw data, but the main way to collect such data is through manual data extraction. |
| Approach: | They propose to use a web crawler to extract and classify data from a multilingual web corpus and an automated annotation pipeline to improve it. |
| Outcome: | The proposed version of OSCAR could be used to pre-train large generative language models and other applications in Natural Language Processing and Digital Humanities. |
Building a User-Generated Content North-African Arabizi Treebank: Tackling Hell (2020.acl-main)
Copied to clipboard
Djamé Seddah, Farah Essaidi, Amal Fethi, Matthieu Futeral, Benjamin Muller, Pedro Javier Ortiz Suárez, Benoît Sagot, Abhishek Srivastava
| Challenge: | a treebank for a north-African Arabic dialect known for code-switching is made freely available . authors: geopolitical events are a factor highlighting a language deficiency in terms of natural language processing resources . |
| Approach: | They propose to make a treebank for a romanized user-generated content variety of Algerian . they supplement it with 50k unlabeled sentences from common crawl and web-crawled data . |
| Outcome: | The proposed treebank is made of 1500 sentences, fully annotated in morpho-syntax and universal dependency syntax, with full translation at both the word and sentence levels. |
CoNLL-UL: Universal Morphological Lattices for Universal Dependency Parsing (L18-1)
Copied to clipboard
Amir More, Özlem Çetinoğlu, Çağrı Çöltekin, Nizar Habash, Benoît Sagot, Djamé Seddah, Dima Taji, Reut Tsarfaty
| Challenge: | Using the universal dependencies framework, we address the need for a universal representation of morphological analysis that can capture alternative morphology of surface tokens and is compatible with the segmentation and morphologic annotation guidelines prescribed for UD treebanks. |
| Approach: | They propose a new annotation format for word lattices that represent morphological analyses and a resource that obeys this format for a range of typologically different languages. |
| Outcome: | The proposed model can capture alternative morphological analyses of surface tokens and is compatible with the segmentation and morphology guidelines prescribed for UD treebanks. |
MANTa: Efficient Gradient-Based Tokenization for End-to-End Robust Language Modeling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Subword tokenization algorithms have been an essential component of language modeling but their static nature results in important flaws that degrade the models’ downstream performance and robustness. |
| Approach: | They propose a module for Adaptive Neural TokenizAtion that is differentiable and trained end-to-end with the language model. |
| Outcome: | The proposed tokenizer improves robustness to character perturbations and out-of-domain data. |
PatentEval: Understanding Errors in Patent Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | a patent is a legal instrument that grants inventors or entities exclusive rights over their invention for a designated period. |
| Approach: | They propose a typology specifically designed for evaluating two distinct tasks in machine-generated patent texts. |
| Outcome: | The proposed approach provides valuable insights into the capabilities and limitations of current language models in the specialized field of patent text generation. |
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data (2026.acl-long)
Copied to clipboard
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett, Rafael Mosquera, Sara Hincapié Monsalve, Thom Vaughan, Damian Stewart, Malte Ostendorff, Idris Abdulmumin, Vukosi Marivate, Shamsuddeen Hassan Muhammad, Atnafu Lambebo Tonja, Hend Al-Khalifa, Nadia Ghezaiel Hammouda, Verrah Akinyi Otiende, Tack Hwa Wong, Jakhongir Saydaliev, Melika Nobakhtian, Muhammad Ravi Shulthan Habibi, Chalamalasetti Kranti, Carol Muchemi, Khang Nguyen, Faisal Muhammad Adam, Luis Frentzen Salim, Reem Alqifari, Cynthia Jayne Amol, Joseph Marvin Imperial, Ilker Kesen, Ahmad Mustafid, Pavel Stepachev, Leshem Choshen, David Anugraha, Hamada Nayel, Seid Muhie Yimam, Vallerie Alexandra Putra, My Chiffon Nguyen, Azmine Toushik Wasi, Gouthami Vadithya, Rob Van Der Goot, Lanwenn ar C’horr, Karan Dua, Andrew Yates, Mithil Bangera, Yeshil Bangera, Hitesh Laxmichand Patel, Shu Okabe, Fenal Ashokbhai Ilasariya, Dmitry Gaynullin, Genta Indra Winata, Yiyuan Li, Juan Pablo Martínez, Amit Agarwal, Ikhlasul Akmal Hanif, Raia Abu Ahmad, Esther Adenuga, Filbert Aurelian Tjiaranata, Weerayut Buaphet, Michael Anugraha, Sowmya Vajjala, Benjamin L Rice, Azril Hafizi Amirudin, Jesujoba Oluwadara Alabi, Srikant Panda, Yassine Toughrai, Bruhan Kyomuhendo, Daniel Ruffinelli, null Akshata, Manuel Goulão, Ej Zhou, Ingrid Gabriela Franco Ramirez, Cristina Aggazzotti, Konstantin Dobler, Jun Kevin, Quentin Pagès, Nicholas Andrews, Nuhu Ibrahim, Mattes Ruckdeschel, Amr Keleg, Mike Zhang, Casper Rufaro Muziri, Saron Samuel, Sotaro Takeshita, Kun Kerdthaisong, Luca Foppiano, Rasul Dent, Tommaso Green, Ahmad Mustapha Wali, Kamohelo Makaaka, Vicky Feliren, Inshirah Idris, Hande Celikkanat, Abdulhamid Abubakar, Jean Maillard, Benoît Sagot, Thibault Clérice, Kenton Murray, Sarah K. K. Luger
| Challenge: | Language identification (LID) is a fundamental step in curating multilingual corpora. |
| Approach: | They introduce CommonLID, a community-driven, human-annotated LID benchmark for the web domain, covering 109 languages. |
| Outcome: | The proposed benchmark covers 109 languages and shows that existing evaluations overestimate accuracy for many languages in the web domain. |
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets (2022.tacl-1)
Copied to clipboard
Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro Ortiz Suarez, Iroro Orife, Kelechi Ogueji, Andre Niyongabo Rubungo, Toan Q. Nguyen, Mathias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaventure F. P. Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Ballı, Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, Mofetoluwa Adeyemi
| Challenge: | Lower-resource corpora have systematic issues, including mislabeled or nonstandard/ambiguous language codes. |
| Approach: | They manually audit the quality of 205 language-specific corpora released with five major public datasets. |
| Outcome: | The results show that lower-resource corpora have systematic issues even for non-proficient speakers. |
From FreEM to D’AlemBERT: a Large Corpus and a Language Model for Early Modern French (2022.lrec-1)
Copied to clipboard
Simon Gabay, Pedro Ortiz Suarez, Alexandre Bartz, Alix Chagué, Rachel Bawden, Philippe Gambette, Benoît Sagot
| Challenge: | Anguage models for historical states of language are becoming more complex to process and more scarce in the corpora available. |
| Approach: | They propose to use a contextualised language model to analyse historical states of language in French. |
| Outcome: | The proposed model is based on a corpus of historical texts and is evaluated with an NLP task. |
ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for sentence simplification are focused on a single transformation, such as lexical paraphrasing or splitting. |
| Approach: | They propose a dataset for assessing sentence simplification in English using a crowdsourced multi-reference corpus. |
| Outcome: | The proposed dataset shows that it captures characteristics of simplicity better than other datasets. |
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations (2023.acl-long)
Copied to clipboard
Paul-Ambroise Duquenne, Hongyu Gong, Ning Dong, Jingfei Du, Ann Lee, Vedanuj Goswami, Changhan Wang, Juan Pino, Benoît Sagot, Holger Schwenk
| Challenge: | SpeechMatrix is a large-scale multilingual corpus of speech-to-speech translations mined from real speech of European Parliament recordings. |
| Approach: | They present a large-scale multilingual corpus of speech-to-speech translations mined from real speech of European Parliament recordings. |
| Outcome: | The proposed model can train bilingual models on 136 language pairs with 418 thousand hours of speech. |
SpiRit-LM: Interleaved Spoken and Written Language Model (2025.tacl-1)
Copied to clipboard
Tu Anh Nguyen, Benjamin Muller, Bokai Yu, Marta R. Costa-jussa, Maha Elbayad, Sravya Popuri, Christophe Ropers, Paul-Ambroise Duquenne, Robin Algayres, Ruslan Mavlyutov, Itai Gat, Mary Williamson, Gabriel Synnaeve, Juan Pino, Benoît Sagot, Emmanuel Dupoux
| Challenge: | SpiRit-LM is a foundation multimodal language model that freely mixes text and speech. |
| Approach: | They propose a multimodal language model that freely mixes text and speech . they extend the model to the speech modality by continuously training it on text and language units. |
| Outcome: | The proposed model can learn new tasks in a few-shot fashion across modalities. |
mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus (2025.findings-acl)
Copied to clipboard
Matthieu Futeral, Armel Randy Zebaze, Pedro Ortiz Suarez, Julien Abadji, Rémi Lacroix, Cordelia Schmid, Rachel Bawden, Benoît Sagot
| Challenge: | Existing studies show that multimodal large language models can learn from text-image data. |
| Approach: | They propose to train multimodal large language models on large amounts of text-image data . they also show a boost in few-shot learning performance across various multilingual tasks . |
| Outcome: | The proposed dataset is not public and is only in English . it is the first large-scale multilingual and multimodal document corpus crawled from the web. |
TopXGen: Topic-Diverse Parallel Data Generation for Low-Resource Machine Translation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning and similarity search have been shown to improve LLMs' performance in machine translation, but they lag behind when dealing with low-resource languages. |
| Approach: | They propose a method that uses an LLM to generate topic-specific target-side data in the LRL. |
| Outcome: | The proposed approach boosts LLM translation performance during in-context learning and fine-tuning. |
A multilingual collection of CoNLL-U-compatible morphological lexicons (L18-1)
Copied to clipboard
| Challenge: | Existing morphological lexicons are limited in scope and are not universally accepted . morphology lexical information is encoded into morphologists or gathered in lexiconics . |
| Approach: | They propose a multilingual collection of morphological lexicons that follow the Universal Dependencies initiative. |
| Outcome: | The proposed collection of 53 morphological lexicons covers 38 languages . they have been shown to improve part-of-speech tagging and parsing accuracy . |
Cheating a Parser to Death: Data-driven Cross-Treebank Annotation Transfer (L18-1)
Copied to clipboard
| Challenge: | Using annotated corpus for linguistic purposes is no longer justified . hand-crafted syntactic resources such as grammars and lexicons can be used as sources of features to guide data driven systems. |
| Approach: | They propose an efficient method for transferring annotations between two different treebanks of the same language. |
| Outcome: | The proposed method is based on the Universal Dependency annotation scheme and was evaluated on the gold standard (94.75% of LAS, 99.40% UAS on the test set). |
Automatic Normalisation of Early Modern French (2022.lrec-1)
Copied to clipboard
| Challenge: | Spelling normalisation is a useful step in the study and analysis of historical language texts, whether it is manual analysis by experts or automated analysis using downstream natural language processing (NLP) tools. |
| Approach: | They propose a new benchmark for the normalisation of Early Modern French into contemporary French using ABA, alignment-based approach and MT-approaches. |
| Outcome: | The proposed method homogenises the variable spelling in historical documents and reduces the gap between the historical state of the language and the contemporary state. |
Gaperon: A Peppered English-French Generative Language Model Suite (2026.findings-acl)
Copied to clipboard
Nathan Godey, Wissam Antoun, Rian Touchent, Rachel Bawden, Éric Villemonte de la Clergerie, Benoît Sagot, Djamé Seddah
| Challenge: | Standardized benchmarks have become the dominant metric for measuring progress in large language models, but their validity is compromised by data contamination and unclear relationship between benchmark scores and genuine language understanding. |
| Approach: | They propose to use GAPERON to investigate evaluation dynamics under realistic training conditions. |
| Outcome: | The proposed model outperforms models that excel on benchmarks in qualitative text generation and vice versa. |
DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexicon (2022.tacl-1)
Copied to clipboard
Robin Algayres, Tristan Ricoul, Julien Karadayi, Hugo Laurençon, Salah Zaiem, Abdelrahman Mohamed, Benoît Sagot, Emmanuel Dupoux
| Challenge: | Existing nonparametric models for text segmentation use a Dirichlet process to jointly segment sentences and build a lexicon of word types. |
| Approach: | They propose a Bayesian nonparametric model that uses a Dirichlet process to jointly segment sentences and build a lexicon of word types. |
| Outcome: | The proposed model improves on the Zero Resource Speech Benchmark 2017 and can learn semantic and syntactic representations as assessed by a new spoken word embedding benchmark. |
Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve Question Answering performance on non-English data are expensive and limited to evaluation set. |
| Approach: | They propose a method to improve Question Answering performance without additional annotations by leveraging Question Generation models to produce synthetic samples in a cross-lingual fashion. |
| Outcome: | The proposed method outperforms baselines on four datasets in English significantly . the proposed model outperformed baselines in english and is comparable to the validation set of the original SQuAD. |
Probing Multilingual Cognate Prediction Models (2022.findings-acl)
Copied to clipboard
| Challenge: | linguistic interpretations of cognate prediction have been based on external analysis (accuracy, raw results, errors). |
| Approach: | They propose to use character-based machine translation models to store linguistic and diachronic information but not in previously assumed ways. |
| Outcome: | The proposed model stores linguistic and diachronic information but does not achieve it in previously assumed ways. |
Anisotropy Is Inherent to Self-Attention in Transformers (2024.eacl-long)
Copied to clipboard
| Challenge: | despite their success, Transformers models suffer from a representation degeneration problem . anisotropy is a property of hidden representations that makes them unexpectedly close to each other . |
| Approach: | They investigate the representation degeneration problem in a self-supervised learning model based on Transformers . anisotropy is a property of hidden representations which makes them unexpectedly close to each other . |
| Outcome: | The representation degeneration problem is a phenomenon widely observed among self-supervised learning methods based on Transformers. |
BERTrade: Using Contextual Embeddings to Parse Old French (2022.lrec-1)
Copied to clipboard
| Challenge: | a growing interest in digital humanities for automatic processing and annotation of historical texts is generating new models for historical languages. |
| Approach: | They use POS-tagging and dependency parsing to evaluate contextual word embedding models . Old French is one of the historical languages for which they have the largest amount of syntactically annotated data . |
| Outcome: | The proposed model can be used to improve performance in Old French, the authors show . they use POS-tagging and dependency parsing to evaluate the model's quality . |
OFrLex: A Computational Morphological and Syntactic Lexicon for Old French (2020.lrec-1)
Copied to clipboard
| Challenge: | Using heterogeneous language resources, we extract structured and exploitable information from a large-coverage morphological and syntactic Old French lexicon. |
| Approach: | They propose to use a large-coverage morphological and syntactic Old French lexicon to extract structured and exploitable information from heterogeneous language resources. |
| Outcome: | The proposed extension technique will be validated manually in the near future and take advantage of OFrLex’s viewing, searching and editing interface. |
From Text to Source: Results in Detecting Large Language Model-Generated Content (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate human-like text, but have ethical and misuse concerns. |
| Approach: | They evaluate whether a classifier trained to distinguish between source and target LLMs can detect text from an LLM without further training. |
| Outcome: | The proposed method detects text from target LLMs without further training. |
MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases (2022.lrec-1)
Copied to clipboard
| Challenge: | MUSS trains strong models using sentence-level paraphrase data instead of labeled simplification data. |
| Approach: | They propose a multilingual unsupervised sentence simplification system that does not require labeled simplification data. |
| Outcome: | The proposed model outperforms the previous best supervised models on English, French, and Spanish benchmarks despite not using labeled simplification data. |
CamemBERT: a Tasty French Language Model (2020.acl-main)
Copied to clipboard
Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, Benoît Sagot
| Challenge: | Pretrained language models are now ubiquitous in Natural Language Processing, but their use in other languages is limited. |
| Approach: | They propose to train monolingual Transformer-based model for other languages using web crawled data instead of Wikipedia data and a relatively small web crawl dataset leads to better results. |
| Outcome: | The proposed model performs as well as those obtained using larger datasets. |
On the Scaling Laws of Geographical Representation in Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Language models embed geographical information in their hidden representations, but larger models cannot mitigate this bias. |
| Approach: | They propose to extend this finding to Large Language Models by observing how geographical knowledge evolves when scaling language models. |
| Outcome: | The proposed model scales consistently with increasing model size, but smaller models cannot mitigate geographic bias inherent in training data. |
Making Sentence Embeddings Robust to User-Generated Content (2024.lrec-main)
Copied to clipboard
| Challenge: | NLP models struggle on user-generated content (UGC) due to high lexical variance and deviating from the standard texts. |
| Approach: | They propose a sentence embedding model that embeds non-standard sentences and their standard counterparts close to each other in the embeddable space. |
| Outcome: | The proposed model outperforms LASER on key typos and social media abbreviations while outperforming LASER in other tasks. |
Generative Spoken Dialogue Language Modeling (2023.tacl-1)
Copied to clipboard
Tu Anh Nguyen, Eugene Kharitonov, Jade Copet, Yossi Adi, Wei-Ning Hsu, Ali Elkahky, Paden Tomasello, Robin Algayres, Benoît Sagot, Abdelrahman Mohamed, Emmanuel Dupoux
| Challenge: | dGSLM is the first “textless” model able to generate audio samples of naturalistic spoken dialogues. |
| Approach: | They propose a model that generates speech, laughter, and other paralinguistic signals in two channels simultaneously and reproduces more naturalistic turn taking compared to a text-based cascaded model. |
| Outcome: | The proposed model reproduces more naturalistic and fluid turn taking than a text-based cascaded model. |
Generative Spoken Language Model based on continuous word-sized audio tokens (2023.emnlp-main)
Copied to clipboard
| Challenge: | Text-based language models outperform character-based models, but speech inputs are 20ms or 40ms-long discrete units. |
| Approach: | They propose a generative language model based on word-size continuous audio tokens . they replace lookup table for lexical types with a Lexical Embedding function . |
| Outcome: | The proposed model is five times more memory efficient than discrete unit GSLMs and is phonetically and semantically interpretable. |
Can Cognate Prediction Be Modelled as a Low-Resource Machine Translation Task? (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing work on cognate prediction based on similarities of two languages has not studied their differences or optimized architectural choices. |
| Approach: | They compare statistical and neural MT architectures to a bilingual setup to test their hypothesis . they use monolingual pretraining, backtranslation and multilinguality to test the hypothesis based on the results . |
| Outcome: | The proposed architectures can be used to generate cognates in a given language . the proposed architecture can be employed with monolingual pretraining, backtranslation and multilinguality . |
Identifying Rare Languages in Common Crawl Data is a Needles-in-a-Haystack Problem (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new pipeline can be used to create corpora for over-looked languages . |
| Approach: | We propose a new pipeline that can filter a single snapshot in twohours. |
| Outcome: | The proposed pipeline can filter a single snapshot in twohours. |
Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | generative large language models (LLMs) can perform in-context learning . machine translation (MT) has been shown to benefit from in-constitu examples . |
| Approach: | They propose a compositional translation paradigm that replaces naive few-shot MT with similarity-based demonstrations. |
| Outcome: | The proposed paradigm replaces naive few-shot MT with similarity-based demonstrations. |
What Does BERT Learn about the Structure of Language? (P19-1)
Copied to clipboard
| Challenge: | BERT is a language representation model that has performed well in diverse language understanding benchmarks. |
| Approach: | They perform experiments to unpack the elements of English language structure learned by BERT. |
| Outcome: | The proposed model outperforms state-of-the-art models in the GLUE benchmark by a significant margin. |
When Your Cousin Has the Right Connections: Unsupervised Bilingual Lexicon Induction for Related Data-Imbalanced Languages (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised bilingual lexicon induction depend on good quality static or contextual embeddings for both languages. |
| Approach: | They propose a method for unsupervised bilingual lexicon induction between a related LRL and a high-resource language that only requires inference on a masked language model of the HRL. |
| Outcome: | The proposed method performs well on low-resource languages with 5M tokens against Hindi . it is compared with existing methods on (mid-resourced) Marathi and Nepali . |
Complex Labelling and Similarity Prediction in Legal Texts: Automatic Analysis of France’s Court of Cassation Rulings (2022.lrec-1)
Copied to clipboard
Thibault Charmet, Inès Cherichi, Matthieu Allain, Urszula Czerwinska, Amaury Fouret, Benoît Sagot, Rachel Bawden
| Challenge: | Detecting divergences in the applications of the law is an important task . divergencies can occur at three levels: within the Cour de Cassation, between trial courts and, more rarely, between a trial court and the Cour of Cassion. |
| Approach: | They propose to provide automatic tools to facilitate the search for similar rulings . they provide automatic keyword sequence generation models and predict keyword sequences based on available texts . |
| Outcome: | The proposed tools improve correlations between the obtained similarities and human judgments of similarity. |
XLS-R fine-tuning on noisy word boundaries for unsupervised speech segmentation into words (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to segment speech into words is needed to overcome the lack of explicit word boundaries in the speech stream. |
| Approach: | They propose to fine-tune a self-supervised speech model to predict word boundaries . they use XLS-R to fine tune the models and infer new word boundary labels . |
| Outcome: | The proposed model outperforms existing models and sets a new state-of-the-art on five corpora with different languages. |
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Language models are a new standard to build state-of-the-art NLP systems. |
| Approach: | They compare multilingual and monolingual models on unseen languages . they show that some languages benefit from transfer learning whereas others don't . |
| Outcome: | The proposed model behaves in multiple ways on unseen languages, while others fail to transfer . the results provide a promising direction towards making multilingual models useful for a new set of unseense languages. |
Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work in multimodal machine translation (MT) has shown that ambiguity can be resolved using accompanying context such as images. |
| Approach: | They propose a multimodal machine translation approach based on a strong text-only MT model and a novel guided self-attention mechanism to train it. |
| Outcome: | The proposed model outperforms existing models on EnglishFrench, EnglishGerman and EnglishCzech benchmarks and is freely available. |
T-Modules: Translation Modules for Zero-Shot Cross-Modal Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to perform zero-shot cross-modal transfer between speech and text are limited to a very small number of language pairs. |
| Approach: | They propose a method to perform zero-shot cross-modal transfer between speech and text for translation tasks by using a speech decoder. |
| Outcome: | The proposed model significantly improves state-of-the-art for zero-shot speech translation on Must-C. |
Data-Efficient French Language Modeling with CamemBERTa (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in NLP have significantly improved the performance of language models on a variety of tasks. |
| Approach: | They introduce a French DeBERTa model that builds upon the DeBERTAV3 architecture and training objective and evaluate its performance on a variety of French downstream tasks and datasets. |
| Outcome: | The proposed model outperforms BERT-based models on most tasks given the same amount of training tokens and trained on 30% of its input tokens. |
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages (2020.acl-main)
Copied to clipboard
| Challenge: | a recent trend in neural NLP has been the introduction of feature-based and fine-tuning methods . we train monolingual contextualized word embeddings for five mid-resource languages . |
| Approach: | They use common Crawl corpus to train monolingual contextualized word embeddings . they compare performance of OSCAR-based and Wikipedia-based embeddables on part-of-speech tasks . |
| Outcome: | The results show that OSCAR-based and Wikipedia-based embeddings perform better than Wikipedia-style embedders on part-of-speech tagging and parsing tasks. |
Controllable Sentence Simplification (2020.lrec-1)
Copied to clipboard
| Challenge: | Text simplification is often considered an all-purpose generic task where the same simplifications are suitable for all but multiple audiences can benefit from simplified text in different ways. |
| Approach: | They propose a controllable simplification model that provides explicit control on simplification systems based on Sequence-to-Sequence models. |
| Outcome: | The proposed model outperforms standard models on simplification benchmarks. |