Papers by Reut Tsarfaty
RUN through the Streets: A New Dataset and Baseline Models for Realistic Urban Navigation (D19-1)
Copied to clipboard
| Challenge: | Existing work on map-based NL navigation relies on small artificial worlds with a fixed set of entities known in advance. |
| Approach: | They propose a task to interpret navigation instructions in natural language (NL) they use a dataset aligned with real, dense, urban maps to study neural architectures . |
| Outcome: | The proposed task is based on a dataset of 2515 navigation instructions aligned with real routes over three regions of Manhattan. |
From SPMRL to NMRL: What Did We Learn (and Unlearn) in a Decade of Parsing Morphologically-Rich Languages (MRLs)? (2020.acl-main)
Copied to clipboard
| Challenge: | a decade has passed since the establishment of SPMRL to address the peculiar challenges of Statistical Parsing for Morphologically-rich languages (MRLs). |
| Approach: | They propose a framework for parsing MRLs and propose implementing symbolic ideas into modern neural architectures. |
| Outcome: | The proposed strategies are based on the multi-tagging task in Hebrew, a morphologically-rich, high-fusion, language. |
Morphological Inflection with Phonological Features (2023.acl-short)
Copied to clipboard
| Challenge: | Recent advances in morphological tasks can be difficult to solve when little training data is available or when generalizing to previously unseen lemmas. |
| Approach: | They propose two methods to manipulate phonemic data to include phonological features instead of characters. |
| Outcome: | The proposed methods yield comparable results to baseline models, with minor improvements in some languages. |
Asking It All: Generating Contextualized Questions for any Semantic Role (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to question generation require conditioning on existing answers in text . previous work required human-curated templates, limiting coverage and question fluency . |
| Approach: | They propose a task of role question generation that produces a prototype and revises it to be contextually appropriate for the passage. |
| Outcome: | The proposed model generates diverse and well-formed questions for a large, broad-coverage ontology of predicates and roles. |
MoNaCo: More Natural and Complex Questions for Reasoning Across Dozens of Documents (2026.tacl-1)
Copied to clipboard
Tomer Wolfson, Harsh Trivedi, Mor Geva, Yoav Goldberg, Dan Roth, Tushar Khot, Ashish Sabharwal, Reut Tsarfaty
| Challenge: | Automated agents powered by large language models are becoming more ingrained into how people seek information . but evaluation benchmarks for LLMs rarely feature natural questions that are time-consuming . a new benchmark, MoNaCo, aims to address this gap by eliciting and manually answering time-wasting questions . |
| Approach: | They propose a benchmark of 1,315 natural and time-consuming questions that require dozens of intermediate steps to solve. |
| Outcome: | MoNaCo benchmarks achieve at least 61.2% F1 in real-world time-consuming questions hampered by low recall and hallucinations . Frontier LLMs evaluated on MoN achieving at least 61% F1, harmed by low memory and halluzinations. |
HeGeL: A Novel Dataset for Geo-Location from Hebrew Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets in English for textual geolocation are limited because of the location of the place is implicit. |
| Approach: | They propose to use a Hebrew place description corpus to analyze lingual geospatial reasoning. |
| Outcome: | The Hebrew Geo-Location corpus collects literal Hebrew place descriptions and analyzes lingual geospatial reasoning. |
Morphological Reinflection with Multiple Arguments: An Extended Annotation schema and a Georgian Case Study (2022.acl-short)
Copied to clipboard
| Challenge: | morphological annotations are a common problem in some languages, but the flat structure of the current schema makes it impossible to treat them. |
| Approach: | They propose a general solution for polypersonal agreement in Georgian language . they extend the existing UniMorph annotation schema to address this problem . |
| Outcome: | The proposed framework covers all possible variants of argument marking, and is accurate and balanced. |
The Possible, the Plausible, and the Desirable: Event-Based Modality Detection for Language Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies restrict modal expressions to a closed syntactic class . modal sense labels are vastly different across different studies, lacking an accepted standard . |
| Approach: | They propose a task where modal expressions can be words of any syntactic class and sense labels are drawn from a comprehensive taxonomy which harmonizes the modal concepts contributed by the different studies. |
| Outcome: | The proposed task is based on the Georgetown Gradable Modal Expressions corpus . it detects and classifies fine-grained modal concepts and associates them with modified events . |
Beyond N-Grams: Rethinking Evaluation Metrics and Strategies for Multilingual Abstractive Summarization (2025.acl-long)
Copied to clipboard
| Challenge: | n-gram-based metrics are considered indicative (even if imperfect) of human evaluation for English, but their suitability for other languages remains unclear. |
| Approach: | They systematically assess evaluation metrics for generation for languages and tasks using n-gram-based and neural-based metrics. |
| Outcome: | The proposed evaluation suite is based on eight languages from four typological families and shows that it is sensitivity to the language type at hand. |
Do Pretrained Contextual Language Models Distinguish between Hebrew Homograph Analyses? (2023.eacl-main)
Copied to clipboard
| Challenge: | Semitic morphologically-rich languages are characterized by extreme word ambiguity . many of the words are homographs with multiple possible analyses . |
| Approach: | They evaluate existing models for Hebrew homographs using word-piece embeddings . they find they are more effective when the number of word-part splits is limited . |
| Outcome: | The proposed models outperform non-contextualized embeddings on Hebrew homograph challenge sets. |
CoNLL-UL: Universal Morphological Lattices for Universal Dependency Parsing (L18-1)
Copied to clipboard
Amir More, Özlem Çetinoğlu, Çağrı Çöltekin, Nizar Habash, Benoît Sagot, Djamé Seddah, Dima Taji, Reut Tsarfaty
| Challenge: | Using the universal dependencies framework, we address the need for a universal representation of morphological analysis that can capture alternative morphology of surface tokens and is compatible with the segmentation and morphologic annotation guidelines prescribed for UD treebanks. |
| Approach: | They propose a new annotation format for word lattices that represent morphological analyses and a resource that obeys this format for a range of typologically different languages. |
| Outcome: | The proposed model can capture alternative morphological analyses of surface tokens and is compatible with the segmentation and morphology guidelines prescribed for UD treebanks. |
Multilingual Sequence-to-Sequence Models for Hebrew NLP (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work on pretrained language models for Hebrew is under-parameterized and under-trained . previous work on pretraining Hebrew LMs focused on encoder-only architectures . |
| Approach: | They propose to use sequence-to-sequence generative architectures to train large LMs in morphologically rich languages such as Hebrew. |
| Outcome: | The proposed model improves on all existing Hebrew NLP benchmarks. |
Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs (2025.findings-naacl)
Copied to clipboard
| Challenge: | Current LLMs are primarily trained on English data but also include data from other languages. |
| Approach: | They propose to use a pre-translation strategy to translate a task prompt into English before inference . they use 'a modular entity' that could be translated into four different languages . |
| Outcome: | The proposed strategies are based on a set of pre-trained data across 35 languages covering both low and high-resource languages. |
Draw Me a Flower: Processing and Grounding Abstraction in Natural Language (2022.tacl-1)
Copied to clipboard
| Challenge: | Abstraction is a core tenet of human cognition and communication. yet, interpreting and grounding abstraction expressed in natural language (NL) has not been systematically studied in NLP. |
| Approach: | They propose a 2D instruction-following game that elicits abstract instructions from 4k natural language instructions. |
| Outcome: | The proposed method elicits 4k natural language instructions rich with diverse types of abstractions and assesses neural models. |
Effective QA-Driven Annotation of Predicate–Argument Relations Across Languages (2026.eacl-long)
Copied to clipboard
| Challenge: | Explicit representations of predicate-argument relations are a cornerstone of natural language understanding. |
| Approach: | They propose a cross-linguistic projection approach that reuses an English QA-SRL parser within a constrained translation and word-alignment pipeline to automatically generate question-answer annotations aligned with target-language predicates. |
| Outcome: | The proposed approach outperforms strong multilingual LLMs in Hebrew, Russian, and French. |
HeQ: a Large and Diverse Hebrew Reading Comprehension Benchmark (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Current benchmarks for Hebrew Natural Language Processing (NLP) focus mainly on morpho-syntactic tasks, neglecting the semantic dimension of language understanding. |
| Approach: | They propose to use Hebrew machine reading comprehension (MRC) as extractive Question Answering to address this problem. |
| Outcome: | The proposed benchmark features 30,147 question-answer pairs derived from both Hebrew Wikipedia articles and Israeli tech news. |
(Un)solving Morphological Inflection: Lemma Overlap Artificially Inflates Models’ Performance (2022.acl-short)
Copied to clipboard
| Challenge: | Inflection tasks have gained a lot of traction in recent years, mostly via SIGMORPHON's shared-tasks. |
| Approach: | They propose to use split-by-lemma to challenge the generalization capacity of morphological inflection models by employing harder train-test splits. |
| Outcome: | The proposed method is based on a split-by-lemma method that challenges the generalization capacity of the models. |
Representations and Architectures in Neural Sentiment Analysis for Morphologically Rich Languages: A Case Study from Modern Hebrew (C18-1)
Copied to clipboard
| Challenge: | Existing sentiment analyzers for MRLs that use tokens and morpheme-based representations have no empirically studied effects of representation choices on neural sentiment analysis. |
| Approach: | They develop a sentiment analysis benchmark for Hebrew based on 12K social media comments and provide two instances of data. |
| Outcome: | The proposed benchmarks show that representation choices have measurable effects on task perfromance and that they vary depending on architecture type. |
Design Choices for Crowdsourcing Implicit Discourse Relations: Revealing the Biases Introduced by Task Design (2023.tacl-1)
Copied to clipboard
| Challenge: | Disagreement in natural language annotation has been studied from a perspective of biases introduced by the annotators and the annotation frameworks. |
| Approach: | They propose to analyze task design bias in crowdsourced annotations where lay annotators are used to elicit interpretations. |
| Outcome: | The proposed methods can push annotators towards certain relations and some discourse relation senses can be better elicited with one or the other approach. |
A Novel Challenge Set for Hebrew Morphological Disambiguation and Diacritics Restoration (2020.findings-emnlp)
Copied to clipboard
| Challenge: | morphological parsers face a formidable challenge with unbalanced ambiguities in homographs . case of unbalanciated ambiguity is difficult to disambiguate, especially in cases of unbalancing . a new dataset improves the overall average F1 score for Hebrew homograph . |
| Approach: | They propose a challenge set for Hebrew homographs with substantial attestation of each analysis of 21 Hebrew homographies. |
| Outcome: | The proposed set improves the average F1 score for Hebrew homographs by 0.67 . the annotated datasets are made publicly available for further research. |
Where Do We Go From Here? Multi-scale Allocentric Relational Inferencefrom Natural Spatial Descriptions (2024.eacl-long)
Copied to clipboard
| Challenge: | Current NLP navigation studies focus on egocentric local descriptions that require reasoning over the agent’s local perception. |
| Approach: | They propose to use a dataset to analyse English geospatial instructions to find locations and paths from natural language descriptions. |
| Outcome: | The proposed task and dataset includes 10,404 examples of English geospatial instructions for reaching a target location using map-knowledge. |
AlephBERT: Language Model Pre-training and Evaluation from Sub-Word to Sentence Level (2022.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that large pre-trained language models are not sufficient for Hebrew. |
| Approach: | They propose a large pre-trained language model for Hebrew that recovers morphological segments encoded in contextualized embedding vectors. |
| Outcome: | The proposed model obtains state-of-the-art on all tasks beyond contemporary Hebrew baselines. |
Superlatives in Context: Modeling the Implicit Semantics of Superlatives (2025.naacl-long)
Copied to clipboard
| Challenge: | a study of superlatives shows that the semantics of superlations in context can be challenging for contemporary models. |
| Approach: | They propose a unified account of superlative semantics which allows for a broad-coverage annotation schema. |
| Outcome: | The proposed schema allows for interpreting superlative expressions and their semantic interpretations. |
Morphology Without Borders: Clause-Level Morphology (2022.tacl-1)
Copied to clipboard
| Challenge: | Morphological tasks use large multi-lingual datasets that organize words into inflection tables . lack of a clear linguistic and operational definition of what is a word impairs universality of tasks . |
| Approach: | They propose to view morphology as a clause-level phenomenon, rather than word-level . they propose to use a dataset for clause- level morphological tasks in 4 different languages . |
| Outcome: | The proposed dataset for clause-level morphology covers 4 typologically different languages: English, German, Turkish, and Hebrew. |
Dyna-bAbI: unlocking bAbI’s potential with dynamic synthetic benchmarking (2022.starsem-1)
Copied to clipboard
Ronen Tamari, Kyle Richardson, Noam Kahlon, Aviad Sar-shalom, Nelson F. Liu, Reut Tsarfaty, Dafna Shahaf
| Challenge: | Controlled synthetic tasks are an important resource for diagnosing model behavior. |
| Approach: | They propose a framework that provides fine-grained control over task generation in bAbI. |
| Outcome: | The proposed framework provides fine-grained control over task generation in the bAbI benchmark. |
Breaking the Language Barrier: Can Direct Inference Outperform Pre-Translation in Multilingual LLM Applications? (2024.naacl-short)
Copied to clipboard
Yotam Intrator, Matan Halfon, Roman Goldenberg, Reut Tsarfaty, Matan Eyal, Ehud Rivlin, Yossi Matias, Natalia Aizenberg
| Challenge: | Existing studies have focused on pre-translation, but there is still need for it . authors say that it is not universally necessary to translate large language models . |
| Approach: | They re-evaluate the need for pre-translation in the context of PaLM2 models . authors found that PaLM2-L consistently outperforms pre-translated in 94 out of 108 languages . |
| Outcome: | The proposed model outperforms pre-translation in 94 out of 108 languages and 6 benchmarks . authors argue that pre-translated inputs can be used to improve performance . |
Breakpoint Transformers for Modeling and Tracking Intermediate Beliefs (2022.emnlp-main)
Copied to clipboard
| Challenge: | a framework for language understanding models to track and improve beliefs through intermediate points in text is needed . breakpoint modeling is an efficient and end-to-end learning approach that trains models to train beliefs . understanding the behavior of models remains a formidable challenge for model safety, authors say . |
| Approach: | They propose a framework that trains models to track beliefs through intermediate points in text . their framework allows for efficient and robust learning of this type of model . |
| Outcome: | The proposed model outperforms strong representation learning approaches on a variety of NLU tasks. |
What’s Wrong with Hebrew NLP? And How to Make it Right (D19-3)
Copied to clipboard
| Challenge: | Sub-optimal performance of many morphologically rich languages (MRLs) is due to errors in early morphology disambiguation decisions, that cannot be recovered later on in the pipeline, yielding incoherent annotations on the whole. |
| Approach: | They propose to use a joint morpho-syntactic infrastructure for processing Modern Hebrew texts to provide rich and expressive annotations. |
| Outcome: | The proposed pipelines are based on a morpho-syntactic infrastructure for processing Modern Hebrew texts. |
Into the Unknown: Generating Geospatial Descriptions for New Environments (2024.findings-acl)
Copied to clipboard
| Challenge: | Similar to vision-and-language navigation tasks, the Rendezvous (RVS) task requires reasoning over allocentric spatial relationships using non-sequential navigation instructions and maps. |
| Approach: | They propose a large-scale augmentation method for generating high-quality synthetic data for new environments using readily available geospatial data. |
| Outcome: | The proposed method improves accuracy on unseen and seen environments by 45.83% on the Rendezvous (RVS) task. |
Beyond Word Boundaries: A Hebrew Coreference Benchmark and an Evaluation Protocol for Morphologically Complex Text (2026.acl-long)
Copied to clipboard
| Challenge: | CR methods originally designed for English struggle with Morphologically Rich Languages (MRLs) a single token in Hebrew may consist of multiple anaphors, and word/morpheme boundary discrepancies make mention detection and coreference resolution difficult in MRLs. |
| Approach: | They propose a CR dataset that identifies mentions at word, sub-word and multi-word levels and an evaluation protocol that directly addresses word/morpheme boundary discrepancies. |
| Outcome: | The proposed evaluation protocol directly addresses word/morpheme boundary discrepancies in Modern Hebrew, an MRL rich with complex words and pronominal clitics. |
Is Probing All You Need? Indicator Tasks as an Alternative to Probing Embedding Spaces (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing probing tasks are designed to evaluate the information existing in representations by training a simple classification model. |
| Approach: | They propose to use indicators to query embedding spaces for the existence of certain properties to determine whether a property exists in an embeddable space. |
| Outcome: | The proposed indicators provide a more accurate picture of the information captured and removed compared to probes. |
Text-based NP Enrichment (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing NLP tasks and benchmarks do not cover all NP-mediated relations . we aim to enrich each NP in a text with all the preposition-mediated relationships that hold between it and other NPs in the text. |
| Approach: | They propose a task to enrich NPs with preposition-mediated relations that hold between them . they build a large-scale dataset and analyze the data to test the task . |
| Outcome: | The proposed task is based on a large-scale dataset and fine-tuned language models. |
Covering Uncommon Ground: Gap-Focused Question Generation for Answer Assessment (2023.acl-short)
Copied to clipboard
Roni Rabin, Alexandre Djerbetian, Roee Engelberg, Lidan Hackmon, Gal Elidan, Reut Tsarfaty, Amir Globerson
| Challenge: | Human communication often involves information gaps between the interlocutors. |
| Approach: | They propose a model that generates such gap-focused questions automatically . they propose an evaluation by human annotators of the generated questions . |
| Outcome: | The proposed model outperforms human generated questions in a competitive environment. |
Conjunct Resolution in the Face of Verbal Omissions (2023.acl-long)
Copied to clipboard
| Challenge: | Verbal omissions occur when verbs and arguments are omitted from subsequent clauses . state-of-the-art models struggle with this task, but have limited results . |
| Approach: | They propose a conjunct resolution task that uses a split-and-rephrase paradigm to recover verbal omissions . they propose omitted words in bold and omitted words in red . |
| Outcome: | The proposed method performs decently, but leaves ample room for improvement. |
Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020.findings-emnlp)
Copied to clipboard
Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou
| Challenge: | Standard test sets for supervised learning evaluate in-distribution generalization but are misleading when a dataset has systematic gaps. |
| Approach: | They propose a more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data. |
| Outcome: | The proposed model performs significantly lower on contrast sets than on the original test sets—up to 25% in some cases. |
Minimal Supervision for Morphological Inflection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Neural models for morphological reinflection tasks have proved to be extremely accurate given ample labeled data, yet labele d data may be slow and costly to obtain. |
| Approach: | They exploit orthographic and semantic regularities in morphological systems to exploit the orthographic regularities on their own to achieve respectable accuracy. |
| Outcome: | The bootstrapping method outperforms hallucination-based methods for morphological reinflection tasks. |
Design Choices in Crowdsourcing Discourse Relation Annotations: The Effect of Worker Selection and Training (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent methods have obtained promising results by extracting relation labels from participants . obtaining linguistic annotations from novice crowdworkers is difficult . crowdsourcing allows for fast and cost-effective collection of labelled data, but because tasks need to be intuitive, crowdworker cannot be asked to perform them. |
| Approach: | They propose to use a selection-only approach to obtain linguistic annotations from novices . current study shows that the method is cost- and time-intensive . |
| Outcome: | The current study shows that selection and training improves the agreement between workers and gold labels, but the method is cost- and time-intensive. |
A Pointer Network Architecture for Joint Morphological Segmentation and Tagging (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Morphological Disambiguation (MD) is a task of decomposing tokens into morphemes . a simple pipeline is used to segment and tagging raw tokens . |
| Approach: | They propose a new pointer network model that combines symbolic knowledge of morphemes with the learning capacity of neural end-to-end modeling. |
| Outcome: | The proposed model outperforms all previous reported results on Hebrew and Turkish . it uses morphological knowledge and the learning capacity of neural end-to-end modeling . |
Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP (2024.emnlp-main)
Copied to clipboard
| Challenge: | Improvements in language models’ capabilities have pushed their applications towards longer contexts, making long-context evaluation and development an active research area. |
| Approach: | They propose to unpack the taxonomy of long-context based on the properties that make them more difficult with longer contexts. |
| Outcome: | The proposed taxonomy is based on the properties that make them more difficult with longer contexts. |
COHESENTIA: A Novel Benchmark of Incremental versus Holistic Assessment of Coherence in Generated Texts (2023.emnlp-main)
Copied to clipboard
| Challenge: | linguistics has been used to assess the coherence of generated texts . a benchmark of coherency scores is developed to measure the quality of generated text . |
| Approach: | They propose a benchmark to assess coherence of automatically generated texts . they use global and incremental methods to score sentences for coherency . |
| Outcome: | The proposed benchmark measures human-perceived coherence of automatically generated texts . it uses global and incremental scoring, and shows that the models are unsatisfactory . |
pyBART: Evidence-based Syntactic Transformations for IE (2020.acl-demos)
Copied to clipboard
| Challenge: | Syntactic dependencies are designed to accurately reflect syntactical relations, but they do not make semantic relations explicit. |
| Approach: | They propose a Python library for converting English Enhanced UD trees to Enhanced or Enhanced representations. |
| Outcome: | The proposed representations are linguistically sound and make lexical relations explicit . the proposed representation scores higher than Enhanced UD graphs, while requiring fewer patterns. |
QADiscourse - Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines (2020.emnlp-main)
Copied to clipboard
| Challenge: | Discourse relations describe how two propositions relate to one another . annotating discourse relations requires expert annotators . |
| Approach: | They propose a new representation of discourse relations as question-and-answer pairs that crowd-sources wide-coverage data annotated with discourse relations. |
| Outcome: | The proposed representation of discourse relations as QA pairs allows crowd-sourcing wide-coverage datasets annotated with discourse relations. |
UniMorph 4.0: Universal Morphology (2022.lrec-1)
Copied to clipboard
Khuyagbaatar Batsuren, Omer Goldman, Salam Khalifa, Nizar Habash, Witold Kieraś, Gábor Bella, Brian Leonard, Garrett Nicolai, Kyle Gorman, Yustinus Ghanggo Ate, Maria Ryskina, Sabrina Mielke, Elena Budianskaya, Charbel El-Khaissi, Tiago Pimentel, Michael Gasser, William Abbott Lane, Mohit Raj, Matt Coler, Jaime Rafael Montoya Samame, Delio Siticonatzi Camaiteri, Esaú Zumaeta Rojas, Didier López Francis, Arturo Oncevay, Juan López Bautista, Gema Celeste Silva Villegas, Lucas Torroba Hennigen, Adam Ek, David Guriel, Peter Dirix, Jean-Philippe Bernardy, Andrey Scherbakov, Aziyana Bayyr-ool, Antonios Anastasopoulos, Roberto Zariquiey, Karina Sheifer, Sofya Ganieva, Hilaria Cruz, Ritván Karahóǧa, Stella Markantonatou, George Pavlidis, Matvey Plugaryov, Elena Klyachko, Ali Salehi, Candy Angulo, Jatayu Baxi, Andrew Krizhanovsky, Natalia Krizhanovskaya, Elizabeth Salesky, Clara Vania, Sardana Ivanova, Jennifer White, Rowan Hall Maudslay, Josef Valvoda, Ran Zmigrod, Paula Czarnowska, Irene Nikkarinen, Aelita Salchak, Brijesh Bhatt, Christopher Straughn, Zoey Liu, Jonathan North Washington, Yuval Pinter, Duygu Ataman, Marcin Wolinski, Totok Suhardijanto, Anna Yablonskaya, Niklas Stoehr, Hossep Dolatian, Zahroh Nuriah, Shyam Ratan, Francis M. Tyers, Edoardo M. Ponti, Grant Aiton, Aryaman Arora, Richard J. Hatcher, Ritesh Kumar, Jeremiah Young, Daria Rodionova, Anastasia Yemelina, Taras Andrushko, Igor Marchenko, Polina Mashkovtseva, Alexandra Serova, Emily Prud’hommeaux, Maria Nepomniashchaya, Fausto Giunchiglia, Eleanor Chodroff, Mans Hulden, Miikka Silfverberg, Arya D. McCarthy, David Yarowsky, Ryan Cotterell, Reut Tsarfaty, Ekaterina Vylomova
| Challenge: | The Universal Morphology project provides broad-coverage instantiated morphological inflection tables for hundreds of diverse languages. |
| Approach: | They propose a language-independent feature schema for rich morphological annotation and a type-level resource of annotated data in diverse languages realizing that schema. |
| Outcome: | The proposed schema has added 66 new languages, including 24 endangered languages. |
A Truly Joint Neural Architecture for Segmentation and Parsing (2024.eacl-long)
Copied to clipboard
| Challenge: | Contemporary multilingual dependency parsers can parse a diverse set of languages, but performance is lower for Morphologically Rich Languages. |
| Approach: | They propose a joint neural architecture where a lattice-based representation is provided to an arc-factored model and solves the morphological segmentation and syntactic parsing tasks at once. |
| Outcome: | The proposed architecture is language-agnostic and language-based to improve on Hebrew . it shows that the proposed model can parse morphological segmentation and syntactic parsing tasks at once. |
Location Not Found: Exposing Implicit Local and Global Biases in Multilingual LLMs (2026.acl-long)
Copied to clipboard
Guy Mor-Lan, Omer Goldman, Matan Eyal, Adi Mayrav Gilady, Sivan Eiger, Idan Szpektor, Avinatan Hassidim, Yossi Matias, Reut Tsarfaty
| Challenge: | Multilingual large language models have minimized the fluency gap between languages, but they are exposed to the risk of biases as knowledge and norms may propagate across languages. |
| Approach: | They propose a test set with 2,156 questions in 12 languages to quantify models' biases . they show a global bias towards answers relevant to the US-locale . |
| Outcome: | The proposed model can answer locale-ambiguous questions in 12 languages. |
Neural Modeling for Named Entities and Morphology (NEMO2) (2021.tacl-1)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a fundamental NLP task, commonly formulated as classification over a sequence of tokens. |
| Approach: | They develop a morphologically rich-and-ambiguous language with a token-level and morpheme-level NER annotation framework to address Named Entity Recognition (NER) a novel hybrid architecture precedes and prunes morphology and outperforms the standard pipeline for Hebrew NER and Hebrew morphologies. |
| Outcome: | The proposed architecture outperforms the standard pipeline for Hebrew NER and Hebrew morphological decomposition tasks. |
HeSum: a Novel Dataset for Abstractive Text Summarization in Hebrew (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models excel in various natural language tasks in English, but their performance in low-resource languages like Hebrew remains unclear. |
| Approach: | They propose a benchmark dataset specifically designed for Hebrew abstractive text summarization that combines 10,000 article-summary pairs from Hebrew news websites. |
| Outcome: | The proposed dataset shows that it presents distinct difficulties even for state-of-the-art LLMs. |
ZEST: Zero-shot Learning from Text Descriptions using Textual Similarity and Visual Summarization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Specifically, given birds’ images with free-text descriptions of their species, we learn to classify images of previously-unseen species based on specie descriptions. |
| Approach: | They propose to leverage the similarity between species and extract visual summaries from the texts to match visual features to the parts of the text that discuss them. |
| Outcome: | The proposed model outperforms the state-of-the-art on the largest benchmarks for text-based zero-shot learning. |
Mitigating Hallucinations in Large Vision-Language Models (LVLMs) via Language-Contrastive Decoding (LCD) (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) often produce object hallucinations due to their reliance on text cues and learned object co-occurrence biases. |
| Approach: | They propose a language-contrasting decoding algorithm that adjusts LVLM outputs based on LLM confidence levels to mitigate object hallucinations. |
| Outcome: | The proposed method shows up to %4 improvement in POPE F1 scores and %36 reduction in CHAIR scores on COCO validation set while improving captioning quality scores. |
Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance (2024.findings-acl)
Copied to clipboard
| Challenge: | Despite being the cornerstone of BPE, the importance of compression in the tokenization process is still unclear. |
| Approach: | They argue for the theoretical importance of compression in the tokenization process . they also demonstrate the empirical importance of compressing tokenizers for downstream success of pre-trained language models. |
| Outcome: | The proposed method can be viewed as 0-gram language modeling where equal probability is assigned to all tokens. |
MRL Parsing Without Tears: The Case of Hebrew (2024.findings-acl)
Copied to clipboard
| Challenge: | a new approach to parsing morphologically rich languages (MRLs) is needed to overcome the deficiencies of current approaches. |
| Approach: | They propose a "flipped pipeline" where multiple layers are predicted independently on whole-token basis and then synthesized. |
| Outcome: | The proposed model achieves near-SOTA performance on Hebrew NLP tasks. |
Multilingual Instruction Tuning With Just a Pinch of Multilinguality (2024.findings-acl)
Copied to clipboard
| Challenge: | Using multilingual instruction tuning, large language models can be used to follow instructions in multiple languages . a multilingual model can be tuned on a wide range of languages, yet most datasets are limited to English . |
| Approach: | They investigate how multilinguality during instruction tuning affects instruction-following across languages . they find that only 40 multilingual examples improve multilingual instruction- follow . |
| Outcome: | The results show that multilingual models perform better on multilingual mixtures compared to monolingual models . the results suggest that building multilingual instruction-tuned models can be done with only 2-4 languages . |
The Truth, The Whole Truth, and Nothing but the Truth: A New Benchmark Dataset for Hebrew Text Credibility Assessment (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a new dataset evaluates the credibility of statements made by Israeli public figures and politicians . a dataset of 1021 statements is used to assess the credibility and accuracy of statements . |
| Approach: | They propose a dataset to evaluate the credibility of statements by Israeli politicians . they use annotated statements manually annotating them for their credibility status . |
| Outcome: | The proposed model outperforms models based on statement and context, and achieves a 48.3 F1 score. |