Papers by Jonathan May
Macro-Average: Rare Types Are Important Too (2021.naacl-main)
Copied to clipboard
| Challenge: | MT metrics trained on segment-level human judgments are inherently non-transparent and reflect undesirable biases. |
| Approach: | They propose to use a type-based classifier metric to evaluate machine translation and compare it with a supervised and unsupervised one. |
| Outcome: | The proposed model outperforms other models in indicating cross-lingual information retrieval task performance and shows that it can be used to compare supervised and unsupervised neural machine translation. |
Many-to-English Machine Translation Tools, Data, and Pretrained Models (2021.acl-demo)
Copied to clipboard
| Challenge: | Commercial translation systems support only one hundred languages or fewer . commercial translation systems do not make these models available for transfer to low resource languages . |
| Approach: | They propose a multilingual neural machine translation model that can translate from 500 source languages to English. |
| Outcome: | The proposed model can translate from 500 source languages to English, or be used as a parent model for low-resource languages. |
Investigating the Benefits of Free-Form Rationales (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that crowdsourced rationales provide additional background knowledge to models . a qualitative study shows generated rationale is not as useful for humans as crowdsourced ones . |
| Approach: | They investigate whether crowdsourced rationales provide additional background knowledge to models . they find that ECQA rationale provides additional background information to understand a decision . |
| Outcome: | The results show that ECQA rationales provide additional background knowledge to understand a decision . compared to crowdsourced rationale, generated rationale is not as useful for humans . |
Bridging the Gap between Native Text and Translated Text through Adversarial Learning: A Case Study on Cross-Lingual Event Extraction (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent research in cross-lingual learning has found that combining large-scale pretrained multilingual language models with machine translation can yield good performance. |
| Approach: | They propose a model architecture that jointly encodes a source language input sentence with its translation to the target language during training and takes a target language sentence with it as input during evaluation. |
| Outcome: | The proposed model architecture can integrate machine translation to improve event extraction while adding machine-translated data yields unstable performance due to representational gap. |
Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning (2025.findings-naacl)
Copied to clipboard
Hyundong Justin Cho, Karishma Sharma, Nicolaas Paul Jedema, Leonardo F. R. Ribeiro, Jonathan May, Alessandro Moschitti
| Challenge: | Language models are biased towards generic outputs as they are trained to align to an aggregate preference to be generally useful. |
| Approach: | They propose a tuning-free method that personalizes language models for text generation tasks with fewer than 10 examples per user. |
| Outcome: | The proposed method achieves favorable win rates on pairwise comparisons with the previous state-of-the-art and outperforms competitive tuning-free baselines for personalized alignment tasks of writing emails, essays and news articles. |
Cross-lingual Structure Transfer for Relation and Event Extraction (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to identify complex semantic structures are difficult to train from under-annotated sources. |
| Approach: | They exploit relation- and event-relevant language-universal features to train relation or event extractors from source annotations and apply them to target languages. |
| Outcome: | The proposed approach achieves comparable performance to state-of-the-art models trained on 3,000 manually annotated mentions. |
CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing systems that negotiate with humans have broad applications in pedagogy and conversational AI. |
| Approach: | They propose to annotate persuasion strategies and perform correlation analysis to understand how dialogue behaviors are associated with the negotiation performance. |
| Outcome: | The proposed system improves negotiation performance for all strategies labeled as skewed . the proposed system is available on github.com/kushalchawla/ . |
Multitask Semi-Supervised Learning for Class-Imbalanced Discourse Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Discourse learning is a complex task, and schemas evolve across annotation efforts preventing compilation of smaller datasets into larger ones. |
| Approach: | They propose a multitask learning approach that can combine discourse datasets from similar and diverse domains to improve discourse classification. |
| Outcome: | The proposed approach improves on the NewsDiscourse dataset by 4.9% over current state-of-the-art benchmarks on one of the largest discourse datasets. |
Finding the Optimal Vocabulary Size for Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Class imbalance is said to exist when one or more classes are not of approximately equal frequency in data. |
| Approach: | They cast neural machine translation (NMT) as a classification task in an autoregressive setting and examine its limitations. |
| Outcome: | The proposed model performs better on multiple languages with large data sizes with different vocabulary sizes. |
Can Vision Language Models Understand Mimed Actions? (2025.findings-acl)
Copied to clipboard
Hyundong Justin Cho, Spencer Lin, Tejas Srinivasan, Michael Saxon, Deuksin Kwon, Natali T. Chavez, Jonathan May
| Challenge: | Nonverbal communication (NVC) is an integral part of human language, but it has been overlooked in natural language processing research. |
| Approach: | They propose a multimodal multimodal recognition task that uses a corpus of mimed gestures to evaluate their understanding of NVC. |
| Outcome: | The proposed task is based on 86 unique gestures with perturbations applied to avatar, background, and viewpoint for evaluating recognition robustness. |
RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to personalized dialogue generation rely on dialogue data paired with user traits, profiles or persona description sentences. |
| Approach: | They propose a hierarchical transformer retriever trained on dialogue domain data to perform personalized retrieval and a context-aware prefix encoder that fuses the retrieved information to the decoder more effectively. |
| Outcome: | The proposed model generates more fluent and personalized responses under a suite of human and automatic metrics and is superior to state-of-the-art baselines on English Reddit conversations. |
Grounding Conversations with Improvised Dialogues (2020.acl-main)
Copied to clipboard
| Challenge: | Effective dialogue involves grounding, the process of establishing mutual knowledge that is essential for communication between people. |
| Approach: | a new study uses a corpus of yes-and-turns to analyze improv dialogues . they find that dialogue is a collaborative process by which partners coordinate via turns or acts to jointly construct a common world state. |
| Outcome: | a new study fine-tunes chit-chat dialogue systems with their corpus to encourage more grounded, relevant conversation. |
LegalDiscourse: Interpreting When Laws Apply and To Whom (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in NLP and information retrieval have already enabled practical applications. |
| Approach: | They propose a 'discourse' taxonomy for span-and-relation parsing of legal texts . they use a dataset of 602 state-level law paragraphs with 3,715 discourse spans and 1,671 relations to investigate the increase in liquor licenses and decrease in applicable laws. |
| Outcome: | The proposed model performs poorly at span identification and relation classification, but lags far below human level. |
A Representation Sharpening Framework for Zero Shot Dense Retrieval (2026.eacl-long)
Copied to clipboard
| Challenge: | Zero-shot dense retrieval requires generic, pretrained DRs, which struggle to represent semantic differences between similar documents. |
| Approach: | They propose a training-free representation sharpening framework that augments a document’s representation with information that helps differentiate it from similar documents in the corpus. |
| Outcome: | The proposed framework is compatible with prior approaches to zero-shot dense retrieval and consistently improves their performance. |
Cross-lingual Continual Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing multi-lingual representations such as the one-hop transfer learning pipeline are difficult to adapt to new languages. |
| Approach: | They propose a cross-lingual continuum learning paradigm that evaluates continuous learning approaches that adapt to emerging data from different languages. |
| Outcome: | The proposed model can be used to adapt to new languages in a sequential manner. |
Analyzing Norm Violations in Live-Stream Chat (2023.emnlp-main)
Copied to clipboard
Jihyung Moon, Dong-Ho Lee, Hyundong Cho, Woojeong Jin, Chan Park, Minwoo Kim, Jonathan May, Jay Pujara, Sungjoon Park
| Challenge: | Existing methods for detecting toxic language and norm violations are limited to live-streaming platforms . existing methods are less effective when applied to live streaming platforms based on a limited time frame . |
| Approach: | They propose to use contextual information to automatically moderate toxic content on live streaming platforms. |
| Outcome: | The proposed model improves on live-streaming platforms by 35%. |
Continual Dialogue State Tracking via Example-Guided Question Answering (2023.emnlp-main)
Copied to clipboard
Hyundong Cho, Andrea Madotto, Zhaojiang Lin, Khyathi Chandu, Satwik Kottur, Jing Xu, Jonathan May, Chinnadhurai Sankar
| Challenge: | Dialogue systems are frequently updated to accommodate new services, but naively updating them by continually training with data for new services causes catastrophic forgetting. |
| Approach: | They propose to reformulate dialogue state tracking (DST) as a bundle of example-guided question answering tasks to minimize the task shift between services. |
| Outcome: | The proposed model achieves state-of-the-art performance on DST continual learning metrics without relying on any complex regularization or parameter expansion methods. |
NewsInterview: a Dataset and a Playground to Evaluate LLMs’ Grounding Gap via Informational Interviews (2025.acl-long)
Copied to clipboard
Alexander Spangher, Michael Lu, Sriya Kalyan, Hyundong Justin Cho, Tenghao Huang, Weiyan Shi, Jonathan May
| Challenge: | Existing large datasets (1k-10k transcripts) are generated via crowdsourcing and are inherently unnatural. |
| Approach: | They curate a dataset of 40,000 two-person informational interviews from NPR and CNN . they find that LLMs are significantly less likely than human interviewers to use acknowledgements and pivot to higher-level questions. |
| Outcome: | The proposed model is based on 40,000 interviews with journalists and CNN . |
ELISA-EDL: A Cross-lingual Entity Extraction, Linking and Localization System (N18-5)
Copied to clipboard
| Challenge: | ELISA-EDL is a cross-lingual entity extraction, linking and localization system for Wikipedia languages. |
| Approach: | They propose a cross-lingual entity extraction, linking and localization system for English speakers . it extracts entities from unstructured text in any of 282 Wikipedia languages and links them to English knowledge bases . |
| Outcome: | The proposed system extracts entity mentions from Wikipedia and links them to English knowledge bases and visualizes locations related to disaster topics on a world heatmap. |
BotEval: Facilitating Interactive Human Evaluation (2024.acl-demos)
Copied to clipboard
| Challenge: | Using language models to perform complex interactive tasks is becoming more common with the rapid progress in natural language processing (NLP) models. |
| Approach: | They develop an evaluation toolkit that enables human-bot interactions as part of the evaluation process. |
| Outcome: | The evaluation toolkit enables human-bot interactions as part of the evaluation process, rather than making judgements for a static input. |
Machine Translation Robustness to Natural Asemantic Variation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing machine translation models struggle with noisy data and tail-end words and phrases. |
| Approach: | They introduce and formalize a class of noise and variation that preserves meaning in the target language. |
| Outcome: | The proposed model can perform better on natural asemantic variation (NAV) the proposed model is robust to a variety of perturbations, but not all of them are achieved with organic variations. |
Salience-Aware Event Chain Modeling for Narrative Understanding (2021.emnlp-main)
Copied to clipboard
| Challenge: | Storytelling is the communication of interesting and related events that form a concrete process. |
| Approach: | They propose methods for extracting the principal chain from natural language text . they filter away non-salient events and supportive sentences to isolate them . authors propose novel methods for predicting and answering events from text based on event-based temporal question answering . |
| Outcome: | The proposed method improves narrative prediction and event-based temporal question answering tasks. |
Cross-lingual Structure Transfer for Zero-resource Event Extraction (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches for information extraction only use name tagging . Currently, most successful cross-lingual transfer learning methods are limited to sequence labeling . |
| Approach: | They propose a share-and-transfer framework to transfer graph structures across languages . they propose to convert sentences in any language to language-universal graph structures . |
| Outcome: | The proposed framework performs comparable to state-of-the-art models on three languages without annotations. |
Are Large Language Models Capable of Generating Human-Level Narratives? (2024.emnlp-main)
Copied to clipboard
Yufei Tian, Tenghao Huang, Miri Liu, Derek Jiang, Alexander Spangher, Muhao Chen, Jonathan May, Nanyun Peng
| Challenge: | a recent HCI study has pointed to gaps in machine storytelling ability at the global level . authors show that LLMs have less suspense and less tension than human stories . |
| Approach: | They propose a computational framework to analyze narratives through three discourse-level aspects. |
| Outcome: | The proposed framework analyzes narratives through three discourse-level aspects . it shows that LLMs fall short of human abilities in discourse understanding . |
A Grounded Unsupervised Universal Part-of-Speech Tagger for Low-Resource Languages (N19-1)
Copied to clipboard
| Challenge: | Unsupervised part of speech (POS) tagging is often framed as a clustering problem, but taggers need to ground their clusters as well. |
| Approach: | They propose an approach for low-resource unsupervised part of speech (POS) tagging that yields fully grounded output and requires no labeled training data. |
| Outcome: | The proposed method achieves reasonable performance across languages, including Sinhalese and Kinyarwanda, with no labeled training data. |
Experience Grounds Language (2020.emnlp-main)
Copied to clipboard
Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nisnevich, Nicolas Pinto, Joseph Turian
| Challenge: | aaron carroll: language understanding research is held back by a failure to relate language to the physical world it describes and to social interactions it facilitates. carroll says successful linguistic communication relies on a shared experience of the world. |
| Approach: | They propose to use a broader physical and social context to address communication problems . they argue that the current success of representation learning approaches is limited . |
| Outcome: | a new study suggests that the current success of representation learning requires a parallel tradition of research on the broader physical and social context of language to address the deeper questions of communication. |
Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a series of experiments show that fine-tuning only the cross-attention parameters is nearly as effective as fine-timing all parameters. |
| Approach: | They conduct experiments to fine-tune a translation model on data where either the source or target language has changed. |
| Outcome: | The proposed model can be trained to several new languages with reduced parameter storage overhead. |
Do Nuclear Submarines Have Nuclear Captains? A Challenge Dataset for Commonsense Reasoning over Adjectives and Objects (D19-1)
Copied to clipboard
| Challenge: | a dataset of human judgments is used to test the ability to construct models with an understanding of commonsense knowledge. |
| Approach: | They crowdsource sentences that answer a question about adjectives and their transitivity . they build strong baselines for the task using a classification approach . |
| Outcome: | The proposed model outperforms word-level models on commonsense reasoning tasks. |
Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Using a dictionary, given a rough, target language natives can uncover the latent, fully-fluent rendering of the translation. |
| Approach: | They propose a method that breaks translation into two steps by generating a dictionary and then ‘translating’ the resulting pseudo-translation into a fully fluent translation. |
| Outcome: | The proposed method 'gets better translation results on high-resource languages than previously published unsupervised MT studies' |
Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnostics (2022.findings-emnlp)
Copied to clipboard
Hyundong Cho, Chinnadhurai Sankar, Christopher Lin, Kaushik Sadagopan, Shahin Shayandeh, Asli Celikyilmaz, Jonathan May, Ahmad Beirami
| Challenge: | Recent studies have revealed the vulnerability of dialogue state tracking models to distributional shifts, resulting in poor performance. |
| Approach: | They present a toolkit for standardized and comprehensive dialogue state tracking diagnoses that provides a richer summary of strengths and weaknesses. |
| Outcome: | The proposed toolkit shows that different classes of DST models have clear strengths and weaknesses, while generation models are more promising for handling language variety and span-based classification models are robust to unseen entities. |
Augmenting Training Data for Massive Semantic Matching Models in Low-Traffic E-commerce Stores (2022.naacl-industry)
Copied to clipboard
Ashutosh Joshi, Shankar Vishwanath, Choon Teo, Vaclav Petricek, Vishy Vishwanathan, Rahul Bhagat, Jonathan May
| Challenge: | Existing methods to augment training data for e-commerce stores using behavioral data are limited in low-traffic stores . eXtreme multi-label classification systems require large amounts of customer behavior data . |
| Approach: | They propose a technique that augments behavioral training data via query reformulation . they use an example semantic matching model from the e-commerce store AL-XMC . |
| Outcome: | The proposed method improves quality of the AL-XMC model over a baseline model. |
Explaining Mixtures of Sources in News Articles (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that language models are essential for long-form article generation. |
| Approach: | They propose a generative process where a source-selection schema is first selected by a journalist, and then sources are chosen based on categories in that schema. |
| Outcome: | The proposed model can predict the most suitable schema given just the headline with reasonable accuracy. |
Identifying Informational Sources in News Articles (2023.emnlp-main)
Copied to clipboard
| Challenge: | Identifying sources of information in news articles is relevant to many tasks in NLP, including misinformation detection and argumentation. |
| Approach: | They propose a task to study compositionality of sources in news articles to understand how they are chosen to complement each other. |
| Outcome: | The proposed dataset can be used to train high-performing models for information detection and source attribution. |
NewsEdits: A News Article Revision Dataset and a Novel Document-Level Reasoning Challenge (2022.naacl-main)
Copied to clipboard
| Challenge: | a large dataset of news article revision histories provides clues to narrative and factual evolution in news articles. |
| Approach: | They propose tasks to predict edit-actions performed during version updates . they define article-level edit actions: Addition, Deletion, Edit and Refactor . |
| Outcome: | The proposed dataset is large-scale and multilingual and spans 15 years . it shows that edit-actions are predictable and are likely to be based on factual evolution . |
Connecting the Dots: Event Graph Schema Induction with Path Language Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to automate event extraction focus on uncertainty, re-occurring events and multiple hypotheses. |
| Approach: | They propose a new Event Graph Schema where two event types are connected through multiple paths involving entities that fill important roles in a coherent story. |
| Outcome: | The proposed model is highly effective at inducing salient and coherent schemas. |
Uncovering Intervention Opportunities for Suicide Prevention with Language Model Assistants (2026.acl-long)
Copied to clipboard
Jaspreet Ranjit, Hyundong Justin Cho, Claire J. Smerdon, Yoonsoo Nam, Myles Phung, Jonathan May, John R. Blosnich, Swabha Swayamdipta
| Challenge: | Using language models, annotators can help develop novel suicide interventions . 85% of cases where LM predictions disagree with existing annotations are analyzed . |
| Approach: | They propose a human-in-the-loop algorithm that leverages language models as an assistant to annotators and experts to facilitate data-driven insights from NVDRS data. |
| Outcome: | The proposed algorithm can be used to support the development of novel suicide interventions . it finds that LM predictions match existing data annotations about 85% of the time . |
Cross-lingual Joint Entity and Word Embedding to Improve Entity Linking and Parallel Sentence Mining (D19-61)
Copied to clipboard
| Challenge: | Entities can be used as effective signals to generate less ambiguous semantic representations and align multiple languages. |
| Approach: | They propose a method to generate cross-lingual data that is a mix of entities and contextual words based on Wikipedia. |
| Outcome: | The proposed method can generate cross-lingual data that is a mix of entities and contextual words based on Wikipedia . it provides reliable alignment on word/entity level and sentence level, and thus can be used for unsupervised cross-linguistic entity linking. |
GTA: Generating Long-horizon Tasks for Web Agents at Scale (2026.acl-long)
Copied to clipboard
Tenghao Huang, Kung-Hsiang Huang, Prafulla Kumar Choubey, Yilun Zhou, Muhao Chen, Jonathan May, Chien-Sheng Wu
| Challenge: | Existing benchmarks provide only coarse start–goal annotations without intermediate trajectories . Existing frameworks provide no supervision over the agent's latent decision process . |
| Approach: | They propose a framework that integrates crawling, retrieval-based seeding, in-context generation and automated quality control to produce realistic tasks paired with executable trajectories. |
| Outcome: | The proposed framework decouples crawling from generation for greater efficiency and ensures dense supervision through deterministic replays and systematic validation. |
Speechworthy Instruction-tuned Language Models (2024.emnlp-main)
Copied to clipboard
Hyundong Cho, Nicolaas Jedema, Leonardo Ribeiro, Karishma Sharma, Pedro Szekely, Alessandro Moschitti, Ruben Janssen, Jonathan May
| Challenge: | Current instruction tuned language models are trained on textual preference data and therefore not aligned to speech domain. |
| Approach: | They propose to use radio-industry best practices to prompt and learn speech-based preference data to improve speech-suitability of popular instruction tuned language models. |
| Outcome: | The proposed methods achieve the best win rates in head-to-head comparisons, resulting in preferred or tied to the base model in 76.2% of comparisons on average. |
Learning to Rewrite Negation Queries in Product Search (2025.coling-industry)
Copied to clipboard
| Challenge: | Negations in product search are often used to articulate unwanted product features or components. |
| Approach: | They propose a query rewriting approach to enhance product search performance . they use large language models to extract query reawrites from product text . their results pave the way for further research on enhancing search performance of queries with negations . |
| Outcome: | The proposed approach improves search performance by 3.17% for queries with negations. |
What Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition models are challenging for languages with little training data. |
| Approach: | They propose a simple and efficient neural architecture for cross-lingual named entity recognition models. |
| Outcome: | The proposed model achieves competitive performance with the state-of-the-art on two transferable factors: sequential order and multilingual embedding. |
Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL (2025.findings-acl)
Copied to clipboard
Wichayaporn Wongkamjan, Yanze Wang, Feng Gu, Denis Peskoff, Jonathan K. Kummerfeld, Jonathan May, Jordan Lee Boyd-Graber
| Challenge: | Future human-AI interaction tools can build on our methods for deception detection by triggering friction to give users a chance to interrogate suspicious proposals. |
| Approach: | They propose to use CTRL-D to detect deception in a board game called Diplomacy . CTRL is a counterfactual RL that has a good recall and almost perfect precision . future tools could build on this to reevaluate trust in suspicious negotiations . |
| Outcome: | The proposed method detects human deception with a high precision when compared to a Large Language Model approach that flags many true messages as deceptive. |
WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for measuring anti-LGBTQ+ bias are poorly defined and insufficiently grounded in real-world harms. |
| Approach: | They propose a bias benchmark that is community-sourced and generates a community survey. |
| Outcome: | The proposed method is community-sourced and improves on WinoQueer-v0. |
A Little Human Data Goes A Long Way (2025.acl-short)
Copied to clipboard
| Challenge: | Existing methods to replace human annotation are expensive and limited. |
| Approach: | They investigate the use of synthetic data in Fact Verification and Evidence-based Question Answering by replacing human-generated data with synthetic points on eight diverse datasets. |
| Outcome: | The proposed method shows promise but performance declines when replacing up to 90% of training data with synthetic data are severe . the proposed method can be used to improve models trained on purely synthetic data by including as few as 125 human-generated data points. |
Know Where You’re Going: Meta-Learning for Parameter-Efficient Fine-Tuning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on parameter-efficient fine-tuning methods require additional measures after pre-training and before fine-uning. |
| Approach: | They propose to take parameter-efficient fine-tuning into consideration after pre-training and before fine-uning and use meta-learning to prime a model specifically for parameter-efficiency. |
| Outcome: | The proposed method improves on a pre-trained model with certain modifications and achieves 4.96 points on cross-lingual NER fine-tuning. |
Leitner-Guided Memory Replay for Cross-lingual Continual Learning (2024.naacl-long)
Copied to clipboard
| Challenge: | Various continual learning approaches have proposed to mitigate catastrophic forgetting by restricting the data buffer or limiting the data size of a model. |
| Approach: | They propose to use a human-inspired spaced-repetition technique to prioritize examples for cross-lingual continual learning. |
| Outcome: | The proposed approach significantly and consistently decreases forgetting while maintaining accuracy across natural language understanding tasks, language orders, and languages. |
Cross-lingual Multi-Level Adversarial Transfer to Enhance Low-Resource Name Tagging (N19-1)
Copied to clipboard
| Challenge: | Low-resource language name tagging is an important but challenging task. |
| Approach: | They propose a neural architecture that leverages multi-level adversarial transfer to improve name tagging for low-resource languages. |
| Outcome: | The proposed approach outperforms previous approaches on CoNLL data sets. |
Learning to Generalize for Sequential Decision Making (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances show strong evidence of generalization in spatiotemporal modalities such as robotic manipulation. |
| Approach: | They propose a method for converting a reinforcement learning model into a natural language understanding model by a teacher-student imitation learning method. |
| Outcome: | The proposed model outperforms teacher performance on held-out decision problems by 7% and 24% on out-of-domain problems. |
Segmenting Numerical Substitution Ciphers (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for deciphering historical substitution ciphers are difficult to crack . cipheries that are not segmented are still difficult to deciphere . |
| Approach: | They propose automatic methods to segment historical substitution ciphers using BPE and unigram language models. |
| Outcome: | The proposed methods achieve an average segmentation error of 2% on 100 monoalphabetic ciphers and 27% on 3 real historical homophonic cipheries. |
WARP: Word-level Adversarial ReProgramming (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to transfer learning from pretrained language models are frozen and a task-specific head is trained on top of them. |
| Approach: | They propose an alternative approach that trains one or more task-specific layers on top of the language model. |
| Outcome: | The proposed approach outperforms existing methods on the GLUE leaderboard with just 32 training samples. |
Can VLMs Recall Factual Associations From Visual References? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a systematic deficiency in the multimodal grounding of Vision Language Models is identified . VLMs can recall factual associations when provided a textual reference to an entity . |
| Approach: | They identify a systematic deficiency in the multimodal grounding of Vision Language Models . they show that VLMs struggle to link their internal knowledge of an entity with its image representation . |
| Outcome: | The study shows that VLMs struggle to link internal knowledge with image representations . the findings provide recommendations for future research . |
Can Sequence-to-Sequence Models Crack Substitution Ciphers? (2021.acl-long)
Copied to clipboard
| Challenge: | Current methods for deciphering historical ciphers use beam search and a neural language model . but, this approach assumes that the target plaintext language is known . |
| Approach: | They propose an end-to-end multilingual decipherment model that can solve 1:1 substitution ciphers without explicit language identification. |
| Outcome: | The proposed model can decipher text without explicit language identification while still being robust to noise. |
Out-of-the-box Universal Romanization Tool uroman (P18-4)
Copied to clipboard
| Challenge: | uroman converts text in Chinese, Arabic and Cyrillic into a common Latin-script representation . the tool uses string similarity metrics to compare text from different scripts . |
| Approach: | They propose a tool that converts text in Chinese, Arabic and Cyrillic into a common Latin-script representation. |
| Outcome: | uroman converts text in Chinese, Arabic and Cyrillic into a common Latin-script representation . the tool is available as a Perl script and as an interactive demo web page . |
Recurrent Neural Networks as Weighted Language Recognizers (N18-1)
Copied to clipboard
| Challenge: | Recent experiments show that RNNs outperform other methods in assigning high probability to held-out English text. |
| Approach: | They focus on the single-layer, ReLU-activation, rational-weight RNNs with softmax . they show that most problems for such RNN are undecidable . |
| Outcome: | The proposed model outperforms other methods in assigning high probability to held-out English text. |
Style Transfer with Multi-iteration Preference Optimization (2025.naacl-long)
Copied to clipboard
| Challenge: | Numerous recent techniques for text style transfer characterize their approaches as variants of reinforcement learning and preference optimization. |
| Approach: | They propose to use a pseudo-parallel data generation method and a dynamic weighted reward aggregation method to improve upon established preference optimization techniques. |
| Outcome: | The proposed model outperforms existing models on two commonly used text style transfer datasets and is compared with state-of-the-art models. |
GPT is Not an Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction (2024.acl-long)
Copied to clipboard
| Challenge: | Current benchmarks for social biases have limitations in scope, grounding, quality and human effort required. |
| Approach: | They propose to use a language model to help with the development of bias benchmarks . they extend previous work to a new community and set of biases: the Jewish community and antisemitism . |
| Outcome: | The proposed LLM does not perform well on the Jewish community and antisemitism task. |
Personalized Help for Optimizing Low-Skilled Users’ Strategy (2025.naacl-short)
Copied to clipboard
Feng Gu, Wichayaporn Wongkamjan, Jordan Lee Boyd-Graber, Jonathan K. Kummerfeld, Denis Peskoff, Jonathan May
| Challenge: | a natural language agent generates moves and messages based on player intentions . a dozen games with novice and experienced players generate useful advice . |
| Approach: | a team of researchers augment a natural language agent to generate move and message advice . they use a game to simulate the intentions of novice and experienced players . |
| Outcome: | The enhanced agent generates move and message advice based on player intentions . the agent helps novices compete with experienced players and even surpass them . |
Tracking the Newsworthiness of Public Documents (2024.acl-long)
Copied to clipboard
| Challenge: | a new method to model news coverage of local government is needed . we show that newsworthiness predictions can be useful for journalists seeking to keep abreast of local governments. |
| Approach: | They propose a method that explicitly models when and why stories get press attention . they use an annotated corpus of news articles to build models that predict if a policy item will get covered . |
| Outcome: | The proposed model outperforms retrieval-based methods with limited annotated data and language use between corpora. |
R2D2: Remembering, Replaying and Dynamic Decision Making with a Reflective Agentic Memory (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for web agents struggle with efficient navigation and action execution due to limited visibility and understanding of web structures. |
| Approach: | They propose a framework that integrates memory-enhanced navigation and reflective learning to improve web agents' performance. |
| Outcome: | The proposed framework shows significant improvements over existing methods, including 50% reduction in navigation errors and threefold increase in task completion rates. |
Translating a Language You Don’t Know In the Chinese Room (P18-4)
Copied to clipboard
| Challenge: | In a corruption of John Searle’s famous AI thought experiment, the Chinese Room, we enable humans to translate text even if they don’t have any prior knowledge of the source language. |
| Approach: | They propose a tool that allows humans to translate text without prior knowledge of the source language. |
| Outcome: | The Chinese Room tool can create fluent translations with human expertise required only for the target language. |
The Million Authors Corpus: A Cross-Lingual and Cross-Domain Wikipedia Dataset for Authorship Verification (2025.findings-acl)
Copied to clipboard
| Challenge: | Authorship verification (AV) is a crucial task for identity verification, accountlinking, historical linguistics, and AI-generated text identification. |
| Approach: | They propose to use Wikipedia's Million Authors Corpus to examine authorship verification models on a broad scale. |
| Outcome: | The proposed dataset includes 60.08M textual chunks, contributed by 1.29M Wikipedia authors. |
Opponent Modeling in Negotiation Dialogues by Related Data Adaptation (2022.findings-naacl)
Copied to clipboard
| Challenge: | In a multi-issue negotiation, it involves inferring the relative importance that the opponent assigns to each issue under discussion, which is crucial for finding high-value deals. |
| Approach: | They propose a ranker for inferring the priority order of the opponent from partial dialogues without needing additional annotations for training. |
| Outcome: | The proposed model performs better than baselines while accessing fewer utterances from the opponent. |
Challenges in Context-Aware Neural Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | despite well-reasoned intuitions, most context-aware neural machine translation models show only modest improvements over sentence-level systems. |
| Approach: | They propose a more realistic setting for document-level translation called paragraph-to-paragraph (PARA2PARA) they collect a dataset of Chinese-English novels to promote future research . |
| Outcome: | The proposed model improves translation quality across document-level metrics and discourse phenomena. |
SARAL: A Low-Resource Cross-Lingual Domain-Focused Information Retrieval System for Effective Rapid Document Triage (P19-3)
Copied to clipboard
Elizabeth Boschee, Joel Barry, Jayadev Billa, Marjorie Freedman, Thamme Gowda, Constantine Lignos, Chester Palen-Michel, Michael Pust, Banriskhem Kayang Khonglah, Srikanth Madikeri, Jonathan May, Scott Miller
| Challenge: | a new cross-lingual information retrieval system for low-resource languages is available in less-frequently-taught languages . a multilingual system can search for relevant information in a haystack of documents in swahili or Somali . human-driven approaches to this problem are complicated in 'low-resourced' languages aaron sagar: "the key role played by humans in triaging results is complicated" |
| Approach: | They propose an end-to-end cross-lingual information retrieval system for low-resource languages . the system enables English speakers to search foreign language repositories using English queries . it summarizes the retrieved documents in English with respect to a particular information need . |
| Outcome: | The proposed system achieves top performance in the most recent IARPA MATERIAL CLIR+summarization evaluations. |
Can Language Model Moderators Improve the Health of Online Discourse? (2024.naacl-long)
Copied to clipboard
Hyundong Cho, Shuai Liu, Taiwei Shi, Darpan Jain, Basem Rizk, Yuyang Huang, Zixun Lu, Nuan Wen, Jonathan Gratch, Emilio Ferrara, Jonathan May
| Challenge: | Existing efforts to automate conversational moderation have focused on banning harmful comments or deleting them, but such efforts can inadvertently push users towards echo chambers that exacerbate polarization. |
| Approach: | They propose a framework to assess models’ moderation capabilities independently of human intervention and propose 'conversational moderation' they propose to use language models as conversational moderators to provide specific feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation. |
| Outcome: | The proposed framework assesses models’ moderation capabilities independently of human intervention and shows that appropriately prompted models provide specific and fair feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation. |
More Victories, Less Cooperation: Assessing Cicero’s Diplomacy Play (2024.acl-long)
Copied to clipboard
Wichayaporn Wongkamjan, Feng Gu, Yanze Wang, Ulf Hermjakob, Jonathan May, Brandon Stewart, Jonathan Kummerfeld, Denis Peskoff, Jordan Boyd-Graber
| Challenge: | Diplomacy is a boardgame that offers a challenge for communicative and cooperative AI. |
| Approach: | They run two dozen games with Cicero and annotate in-game communication with abstract meaning representation to separate in- game tactics from general language. |
| Outcome: | The proposed method can outperform Cicero in communicating with humans, but it's difficult to deceive and persuade AI. |
X-METRA-ADA: Cross-lingual Meta-Transfer learning Adaptation to Natural Language Understanding and Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Multilingual models have gained popularity for their zero-shot cross-lingual transfer learning capabilities, but their generalization ability is inconsistent for typologically diverse languages. |
| Approach: | They propose a meta-learning approach that adapts MAML to learn to adapt to new languages . they extensively evaluate two cross-lingual NLU tasks using English as source and spanish as target . |
| Outcome: | The proposed approach outperforms naive fine-tuning on cross-lingual tasks for most languages. |
Teaching Language Models To Gather Information Proactively (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models are often defaulted to passive responses or narrow clarifications when faced with incomplete or under-specified prompts. |
| Approach: | They propose a new task paradigm where LLMs must identify gaps in context and strategically elicit implicit user knowledge through targeted questions. |
| Outcome: | The proposed framework outperforms o3-mini on evaluation metrics and human annotators favor clarification questions and final outlines. |